Completed in 1903, the mosque sits on a piece of wakaf land within the bustling Central Business District (CBD). The mosque is flanked to the left by the elevated East Coast Parkway (ECP) and on its right by Mount Palmer on which sits two important shrines of Habib Noh and Habib Abdul Rahman.
The mosque was built by Haji Muhammad Salleh a merchant from Batavia, now known as Jakarta, who lived in Singapore in the mid 19th to the early 20th century. Haji Muhammad Salleh was a friend of Habib Noh, whom he highly revered. Habib Noh frequented Mount Palmer to pray in seclusion and Haji Muhammad Salleh intended to build a surau there for his beloved friend to pray in ease. However, Habib Noh passed away before his wish was fulfilled.
Haji Muhammad Salleh donated a piece of land at the foot of Mount Palmer and began building the mosque to serve the visitors of the shrine. Today it serves mostly workers in and around the CBD area.