Born on 29 August 1849 at Greenwich, Kent, England, third son of John Sulman, jeweller, and his wife Martha, née Quinton. He was educated at Greenwich Proprietary School and in 1863 passed the Oxford junior examination. After his family moved to Croydon next year, he was articled to Thomas Allom, a London architect; he learned the use of oils and water-colour, and executed perspective drawings for Sir George Gilbert Scott.
He was Pugin travelling scholar in 1871. After travelling through England and western Europe Sulman began practising as an architect in London and designed among other buildings a large number of churches. He emigrated to Sydney, Australia in 1885. From 1921 to 1924 he was chairman of the Federal Capital Advisory Committee and influenced the development of Canberra. On 2 January 2008 it was announced that a suburb in the future Canberra district of Molonglo would be named Sulman.