Bishop Gray's application for a site for a church in Somerset East was approved in 1849.

​

Six years later this small Anglican Church in Beaufort Street, along with an Anglican Burial Ground in Paulet Street, was consecrated in 1855. The chancel and vestry were added in 1881 and the bell installed in 1883.

​

The Anglican "All Saints" Church is noted for its beautiful Memorial windows.