Maps of 1682 refer to this site as Potts Fields and excavations in 1965 revealed that it was the site of the earliest delftware kilns in England, which were established here c.1618. It was also the site of the graveyard of St Olave's church and a few gravestones remain.
It was laid out as a recreation ground in 1888 by St Olave's Board of Works, later converted to a public garden in 1909 by the London County Council. In the 21st century it became part of the landscaped public open space adjoining the river adjacent to the new GLA City Hall designed by Foster Architects and at the south-west foot of Tower Bridge, of which it provides excellent views. In May 2007 an extensive re-landscaping scheme was completed, with a new layout of paths, lawns, planting of trees and beds, fixed seating; the gravestones from the old churchyard now exposed on wall; new gate screens on Tooley Street. The park has been the venue for various outdoor events.