It was first established in 675 by the now-defunct Saxon Barking Abbey and was for many years named after the abbey, as All Hallows Barking. The church was built on the site of a former Roman building, traces of which have been discovered in the crypt. It was expanded and rebuilt several times between the 11th century and 15th century. Its proximity to the Tower meant that it acquired royal connections, with Edward IV making it a royal chantry and the beheaded victims of Tower executions being sent for temporary burial at All Hallows.
The church was badly damaged by a nearby explosion in 1649, which demolished its west tower, and only narrowly survived the Great Fire of London in 1666. It owed its survival to Admiral William Penn, father of William Penn of Pennsylvania fame, who saved it by having the surrounding buildings demolished to create firebreaks. In 1926 a Roman pavement together with many artifacts was discovered many feet below the church. Restored in the late 19th century, All Hallows was gutted by The Blitz in World War II and required extensive reconstruction, only being rededicated in 1957.
Many portions of the old church survived the war and have been sympathetically restored. Its outer walls are 15th century, with a 7th century Saxon doorway surviving from the original church. Many brasses remain in the interior (where one of London's brass rubbing centres is now located). Three outstanding wooden statues of saints dating from the 15th and 16th centuries can also be found in the church, as can an exquisite font cover believed to have been carved in 1682 by Grinling Gibbons, which is regarded by many as one of the finest pieces of carving in London. The church has a museum called the Undercroft Museum.