Police sources said that the process of Ghosh's transfer back to the Presidency jail was done amid tight security and in close coordination with the hospital authorities and security personnel.
Neither the hospital authorities, nor the Presidency Jail officials were available for comment.
On Friday, Ghosh allegedly tried to commit suicide by consuming sleeping pills in the jail, following which the jail superintendent, a doctor and a staff-on-duty were suspended.
Presidency Correctional Home authorities had filed an FIR at the Hastings Police Station against Ghosh for allegedly attempting to commit suicide.
When he was produced before a city court on November 10, Ghosh had threatened to commit suicide if CBI did not take "proper action" against those allegedly involved in the chit fund scam within three days.