Ruling on a petition filed by Muslim women's rights activists who demanded entry to the men-only inner sanctum of the Haji Ali Dargah, a Mumbai High Court bench said the restriction violated women's fundamental right to equality.
"Today the court is ruled in our favour and I am very thankful. It has been a hard fight over many years and we lost momentum as many women were concerned of what society would think of us," Bibi Khatoon from the Bharatiya Muslim Mahila Andolan, or Indian Muslim Women's Movement, told reporters.
"But I say let them say what they want, we will do what we need to do. The Sufi saint Haji Ali also came from woman's womb, so why are not being allowed to visit his tomb?"
The 15th century shrine, built on an islet 500 metres from the city's coast, can only be reached at low tide and draws tens of thousands of worshippers every year.
In 2012, the trust which manages the shrine imposed a ban on women entering its inner sanctum saying that it would a "grievous sin" to allow women near the tomb of Sufi saint Haji Ali who is buried there.
However, court officials said women would not be able to immediately enter the shrine as the judgment has been suspended for six weeks after the Haji Ali Dargah Trust said it would appeal against the decision.
The fight to allow women into the shrine gained momentum after a petition was lodged earlier this year with the Supreme Court demanding access for women to the Hindu Sabarimala Ayyappa temple in southern state of Kerala.
Hindu women in the state of Maharashtra, where Mumbai is the capital, are also protesting for entry to the Shani Shingnapur temple, one of a handful in the country which deny women entry.
The protests this year have also played out on social media, with #RighttoPray and #LetWomenPray trending on Twitter with campaigners saying the restrictions are based on patriarchy rather than on religion.