The library is located on the principal storey of Robert Adam's central block, in the eastern half of the garden (south) front, between the dining room (later the music room) and the breakfast room. This room is widely held to be one of Adam's finest of the 1770s.

The room contains the original bookcases designed by Adam, as well as the portrait busts and plasterwork above, and his green and white marble chimneypiece. Harris has noted that the room also contains pier glasses which comprise an elongated version of those in Adam's library at Kenwood, and which were illustrated in The works in architecture of Robert and James Adam (Volume I, part II, pl. viii).