The Abbey of SS Mary and Benedict at Ramsey, Huntingdonshire, was sold by the crown in 1540 after the Dissolution of the monasteries. The new owner Sir Richard Williams used its ancient buildings as a source of stone for new buildings in the locality and his son built a small Tudor house on the site of the Abbey church in 1600.

In 1737 the property was acquired by the Fellowes family and in the early 19th century came into the possession of William Henry Fellowes, the nephew of Soane's client Robert Fellowes of Shotesham. He commissioned Soane to work on the house in 1804 and as a result extensive alterations costing a total of £10,848 were carried out in the Gothic style. Soane's designs for Ramsey Abbey were concurrent with those for his Gothic Libary at Stowe and his biographer Gillian Darley notes that Soane's alterations at Ramsey were 'dressed to suit the existing monastic building'.

Soane must have taken advantage of having workmen on site at the Abbey to have moulds taken for the three casts in his collection. Although there is no specific record of these casts being made the final reference to Ramsey Abbey in Soane's notebooks is on 22 May 1807 when he pays the sum of £1.14.0 for '... car:[riage] of boxes from Ramsey Abbey' which could be a reference to them being brought to London.

For Soane's work at Ramsey Abbey see Dorothy Stroud, Sir John Soane Architect, 1984, p.189; Ptolemy Dean, Sir John Soane and the Country Estate, 1999, p.190 and Gillian Darley John Soane: An Accidental Romantic, 1999, p.163.