Assistant Superintendent of Parappana Agrahara jail D S Hatti said Madani has been admitted to Soukhya hospital in Whitefield near here for the treatment of diabetes and related eye complication.
His wife and son would be allowed to attend him at the hospital, Hatti said.
The Karnataka High Court had in November last year rejected Madani's bail plea but permitted him to avail treatment in a hospital of his choice under police escort.
Kerala Chief Minister Oommen Chandy had met his Karnataka counterpart Jagadish Shettar last week and urged him to provide medical treatment to Madani.
Ruling UDF and opposition LDF members had last month asked the Kerala government to intervene to ensure proper medical care for Madani. Chandy had said it was not just to keep a person in prison indefinitely "without trial".
Madani was arrested on August 17, 2010 from his camp at Anwarassery in Kerala's Kollam district. He remains lodged in the jail since then.
The Peoples Democratic Party (PDP) leader is among 31 people charge sheeted in the case relating to the July 2008 bomb blasts that killed one person and left 20 others injured here.