The history of the mosque began in 1959 when Muslim residents gathered to discuss the need for a surau (Muslim prayer hall) in Kebun Limau. They requested for land on Kim Keat Road and the government granted them in 1961.
The first surau was a simple 2-storey concrete and wood building, with a main prayer hall on the second floor and an office and a store on the ground floor. It was then known as Masjid Kebun Limau. As the Muslim community expanded in the 1970s, fund raising began for the expansion of the surau. In 1981, Hajjah Rahimabi Ahmad Angullia donated $1.6 million from the sale of a land inheritance from her late father to the building of a new mosque in the memory of her daughter who passed away in 1976.
A mosque was then built on the existing plot and renamed Masjid Hajjah Rahimabi Kebun Limau in honour of the generosity of Hajjah Rahimabi. The new building was completed in 1984.