The earliest reference to a church at Charlton is in a fragment of manuscript, dating from between 1147 and 1182, refering to the dedication of a chapel to St Peter at Charlton. The nature and history of this early building are unclear until 1827 when the church was rebuilt as it was too small. The population of Charlton was rising rapidly in the early 19th Century from 279 in 1801, rising to 600 in 1811 and 791 in 1821. The church was enlarged by the novel expedient of cutting it in half and building a new section through the middle. How much of the building that was rebuilt in 1827 dated back to the 12th century is not known.The population of Charlton continued to grow from 2513 in 1841 to 7839 in 1891. By the late 19th century even the enlarged church was found to be too small for the growing community of Charlton which by then was incorporated into Dover. It was decided to build a new, much larger, church slightly to the north of the old church on the site of the old Charlton School and Master's House. The architect for the new church was James Brookes and his design was published in the “The Builder” magazine in May 1891. This original design included a rather plain tower on the St Alphege Road side of the building. This was later replaced with a design for a much more grandiose tower with a spire on the churchyard side of the building. Neither tower was built when it was found that the available funds were not sufficient. It was intended that the the tower with a spire would be built when funds allowed but it was never built.The new church was consecrated in 1893 and the old church demolished in 1895. A stone in the churchyard marks the site of the old high altar. The rubble from the old church was used partly to construct the new boundary wall, and partly for the footings of the large tower on the churchyard side. The church is still without its tower but he square mound on which the tower was to stand is still visible just outside the south porch.The church was damaged during the Second World War. On 15th September 1944 a 15 inch shell exploded after burrowing under the foundations of the south west corner. The nave was severely damaged, the west wall was split and the blast lifted a pillar and its base moved one and a half inches, though the pillar itself settled absolutely upright. This displacement is still visible today. Two weeks later a second shell fell on the eastern wall of the churchyard blowing out the east windows and causing further extensive structural damage.After the war extensive repairs were undertaken and the church was re hallowed by the Archbishop of Canterbury in September 1952.