Firstly, the ruling family sold the manor in 1713 to Dr John Radcliffe. Thereafter, due to his death a year later, it was managed by a Trust, essentially a committee, none of whom were resident. They employed land agents to manage their affairs and until the arrival of Thomas Harrison, whom I will come to in a moment, none of them were resident either. Thus, the only people of significance on the manor were the four or five farmers and the vicar. This small group made up the middle class with no upper class family above them.
The early "great house" must have been that built by Baron Mainou inside the motte and bailey castle at Old Wolverton. We have no idea what it looked like and no archaeological excavations have ever been undertaken to give us any clue. It seems likely that after 300 years the castle would have been abandoned in favour of a more comfortable house. We have no idea what sort of building was there in the 14th century, but we can assume, given the relative wealth of the de Wolvertons, that it was a splendid enough manor house for the period. From surviving knowledge of domestic architecture of that era, the house at Wolverton probably had a great hall, where all the activities of the house took place, including providing sleeping places for the servants, and attached to it, possibly at right angles, was the private chamber for the lord and his immediate family, often known as the solar. Kitchens were usually housed in a separate building set away from the great hall, as a precaution against the all too frequent risk of fire. We do not know if this house was of timber or stone construction; all we can presume is that there was a house there and it was of a standard befitting its occupants. This must have been followed by a succession of medieval houses on or near the site occupied by the de Wolverton and de Longueville families and it is fair to guess that they became progressively larger. There is some evidence that the house was largely rebuilt in 1586 and it probably ranked as a fine Elizabethan mansion. One visitor, Thomas Hearne in 1711, was impressed with the building.
“It stood near a large mount, thrown up East of the Church, & it was a magnificent Edifice, being 145 Feet in length & built with good Free-stone.”