Brampton Bryan came into the possession of the Harley family in 1309 through marriage with the Brampton family - the powerful March Earls - who had lived there since Domesday. The core of the estate remains in the ownership of the Harley family. The castle was built gradually between the early fourteenth and late sixteenth centuries, and was made famous by its siege during the Civil War when it was defended by the Parliamentarian Lady Brilliana Harley. The family now live a little to the west in a Georgian brick house.

Edward Harley, 4th Earl of Oxford and Mortimer (1726-90), was MP (Tory) for Herefordshire (1747-55); High Steward of Hereford (1755-90); Lord of the Bedchamber (1760-90); and Lord Lieutenant of Radnorshire (1766-90). In 1777 he commissioned Robert Adam's designs for a stable and entrance gates in a castle style, and a large ruinous folly. None of these were executed.

Literature:
A. T. Bolton, The architecture of Robert and James Adam, 1922, Volume II, Index p. 4; N. Pevsner, The buildings of England: Herefordshire, 1963, p. 81; A. Tait, Robert Adam: the creative mind: from the sketch to the finished drawing, 1996, p. 29; S. Astley, Robert Adam's castles, 2000, p. 34; D. King, The complete works of Robert & James Adam and unbuilt Adam, 2001, Volume II, pp. 229, 232, 243

Frances Sands, 2011