The mosque was officially opened in 1962. However, plans to build the mosque started in the 1950s for the Muslim community in the area which used to be known as Alexandra City Council Quarters, Henderson Hill, Redhill Road.
With the help of Jamiyah Singapore and the support from Haji Syed Ali Redha Alsagoff and Haji Baharuddin, a piece of land between Tiong Bharu Road and Henderson Road was obtained and approved by the Government for the site of a new mosque. Funds were then raised, including a huge donation from Jeddah-based charity, Rabitah Al-Alam Al-Islamiah. To commemorate the two organisations for the important role they played in the building of the mosque, the mosque was named “Jamiyah Ar-Rabitah”.
In 1983, to better meet the growing needs of Muslims in the area, the mosque was demolished and rebuilt. It was officially reopened on 9 August 1985, in conjunction with the celebration of Singapore’s 20th National Day, by then Mufti of Singapore Syed Isa Semait.