Sri Lankan President Maithripala Sirisena's office said the decision to release the fishermen had been taken.
Prime Minister Modi, who is due in Colombo on Friday, will be the first Indian Prime Minister since Rajiv Gandhi in 1987 to make a bilateral visit to the island.
Former Sri Lankan president Mahinda Rajapaksa had also ordered the release of Indian fishermen in Sri Lankan custody as a goodwill measure to mark the swearing-in of Prime Minister Modi in May last year.
The Prime Minister would be visiting the Mahabodhi Society in Colombo, and travel to Anuradhapura, Talaimannar and Jaffna during his trip.
He will be the first Indian Prime Minister and the second world leader after British Prime Minister David Cameron to visit Jaffna.
In Jaffna, the Prime Minister will lay the foundation stone for the iconic Jaffna Cultural Centre that will come up adjacent to the historic Jaffna Public Library.