Built in 1921 on a land bequeathed by philanthropist Ahmad Mohamed Salleh Angullia at the junction of Upper Serangoon Road and Hillside Drive, the mosque caters to residents living in Serangoon, Toa Payoh and surrounding areas. The mosque also saw the birth of Madrasah Al-Arabiah Al-Islamiah (MAI) which began operation sometime between 1930 and 1934 in a small building in the mosque’s compound.
In 1973, the mosque was expanded to cater to increasing demand. A dome was also added to the mosque structure. In 1994, the mosque was closed for a major redevelopment which saw the old building torn down, and a new building erected. It reopened a year later in 1995.