A castle is attested at La Ferté-Vidame from 985. In 1374, the domain is acquired by the Vendôme family who rebuilt the Castle. This family holds the prestigious title of Vidame de Chartres. It is this castle which is acquired on 19 may 1635 by Claude de Rouvroy, Duc de Saint-Simon, favourite of Louis XIII.
His son, Louis de Rouvroy, Duc de Saint-Simon, stayed in this castle which has preserved its fortress medieval aspect of eight towers, painted by Louis-Nicolas van Blarenberghe around 1750, preserved at the Museum of Boston. He wrote most of his famous memoirs. In 1718-1719, he constructed the building of the stables (current “small chateau”).
Louis de Rouvroy de Saint-Simon died in 1755. The Castle passes to his granddaughter, Marie-Christine-Chrétienne de Rouvroy de Saint-Simon, wife of Charles-Maurice de Monaco, count of Valentinois. On 21 June 1764, it yields the Castle and the 900 hectares of the estate to the financier Jean-Joseph de Laborde. At the same time, he acquires the title of Vidame of Chartres, attached to the seigneury.
Laborde entrusts the architect Antoine Matthieu Le Carpentier to completely rebuild the Castle, which he retains only a part of the feudal keep, but rendered unrecognizable under additions. The work, completed in 1771, lasted three years, which is short for building a huge three-storey building, which included, say, 167 pieces.
The building is built of brick and stones, as the achievements of the first half of the XVIIth century, but in an original style. The reception rooms were located on the ground floor, as in most country houses, the apartments of guests occupied the first and the second floor. The central Pavilion and two pavilions located at the end of the two wings were covered with square dome roofs. Oval protrusion of the central body was clearly inspired on the garden of the Chateau de Vaux-le-Vicomte. The old moat was transformed into ditches with grass.
Laborde spent at La Ferté-Vidame the senseless sum of 14 million livres. He received Louis XV, the future Joseph II of Austria and the Duc de Choiseul.
In 1783, Louis XVI forced the Duc de Penthièvre to relinquish the Chateau de Rambouillet. In return, Penthièvre, who has already vast estates in Normandy and Perche, loses La Ferté-Vidame, Laborde was forced to sell on the 4 January 1784 for 5.5 million livres. Laborde keeps only the title of Vidame of Chartres.
On the death of the Duc de Penthièvre in 1793, the land passes to his daughter, the Duchesse d’Orleans. But, having emigrated, her property was confiscated. Already trashed by looters, the Chateau de La Ferté-Vidame is sold on 27 March 1798 to Sir Cardot-Villers who, heavily in debt, retrieves all the materials that he can, trashing the forest cutting 31 000 trees. Unable to pay the price of his acquisition, he is deprived of his rights. The area is relisted in June 1803, but it does not find takers and rests in the domain of the State.
It is returned to the Duchesse d’Orléans. When she died, in 1826, the estate passed to her eldest son Louis-Philippe, Future King of the French. He reconstructs the lands, refacing walls, rehabilitating parts of water canals, restore and expand the small castle. But the revolution of 1848 interrupted his restoration.
The goods of the Maison d’Orléans was confiscated under Napoleon III. In 1872, selling La Ferté-Vidame to baron Léon de Dordolot, who engaged in his passion for hunting. He sold it in 1879 to a rich stockbroker from Paris, Charles Laurent. he moved to the small chateau, enlarged the area, began restoration work. His son, Roger Laurent, also into hunting, expands the domain to about 6,000 hectares, nearly 1,000 acres surrounded by walls and forming the Park.
His sister Thérèse Laurent married the Marquis de Lestrade. In 1913, his heirs sell the Chateau and the Park to the Société forestière de Bretagne operates the forest until 1921, before selling the land to Mr. Carpentier, industrialist from Villers-Cotterêts. Again in 1923 to Christian Vieljeux.
In 1945, the remaining part is sold to the Ministère de la Justice that is installs the Reverend Father Courtois, founder of the Sainte-Marie-Madeleine, social reintegration of women prisoners. The work ceases its activity in 1979, after the death of its founder.
In 1991, the State sold the castle to the Departement d’Eure-et-Loir, which undertakes work to make it accessible to the public.