The first castle, now in ruins, probably dates back to the time of the Longobard Autari. Built for strategic purposes (in particular to control the Traiana route ways) on the top of Santi Quaranta Martiri hill, this castle was used and reinforced during both Byzantine and Norman domination.

It was among the properties held by the Suevian Emperor Frederick II and then owned by Angioinian sovereigns. Subsequently, the building passed through the hands of other nobles, the Orsini, Affaitati and Capece Minutolo (who bought it at the start of the XVIII century), until its possession by the town council in 1956 when it was in practical ruin with only its towers in stone and tufa surviving.