The castel “Castelnau de Bonnafous” was built at the beginning of the thirteenth century (1235) by Sicard Alaman (the mighty vassal of the Comte de Toulouse Raymond VII), It represented an advanced bastion of the Toulouse possessions (north of the Tarn River) in front of the royalty (southern Tarn). Built on a rocky Piton named “Puy de Bonnafous” hence its original name of Castelnau de Bonnafous.
It was rebuilt by the Lévis in the fifteenth century and renamed Castelnau de Levis, A village was built at the foot of the walls at the same time. when the seigneurie came into the possession of Hugues d’ Amboise, baron d’Aubijoux; the fief remained with his descendants until the seventeenth century. Hugues’ grandson, Louis d’Amboise, comte d’Aubijoux and baron de Castelnau-de Bonnafous, restored the castle and dwelt in it. The narrow square watchtower (tour de guet) is 40m tall.
Apart from the watchtower, all that remains today are remnants of other buildings. It owes its pitiful state due to being used as a quarry during in the 19th and 20th centuries. From the chateau tower, there is a good view of Albi and the Tarn valley. The chateau is privately owned. It has been listed since 1909 as a monument historique by the French Ministry of Culture.