The first mention of the church was in the late 12th century. It was named "Staining", which means stone, to distinguish it from the other churches of All Hallows in the City of London, which were wooden.
The old church survived the Great Fire of London in 1666 but collapsed five years later in 1671. It was thought its foundations had been weakened by too many burials in the churchyard close to the church walls. The church was rebuilt in 1674.
The parishes of All Hallows Staining and nearby St Olave Hart Street were combined in 1870. At that time All Hallows Staining was demolished, leaving only the tower.
St Olave Hart Street was badly damaged by bombs in 1941, during the Second World War. Between 1948 and 1954, when the restored St Olave's was reopened, a prefabricated church stood on the site of All Hallows Staining. This was known as St Olave Mark Lane. The tower of All Hallows Staining was used as the chancel of the temporary church.
The tower of All Hallows Staining is maintained by the Worshipful Company of Clothworkers, one of the livery companies of the City of London. In 1957 the Clothworkers' Company built a church hall for St Olave Hart Street on the site of All Hallows Staining. The old tower now stands at the back of a small courtyard next to the new hall.