Mani, to whom police served the notice yesterday, said he could not comply with the demand of the police on "medical and legal" grounds.
The CPI-M leader said he had forwarded his reply to the special investigation team (SIT) re-investigating a 30-year-old political murder case.
Mani said he could not undergo the polygraph test as he was an asthma patient and had been under treatment for long. He also said his lawyers had advised him that polygraph test was not legally maintainable.
Police had registered a criminal case and started a fresh probe into the murder of an INTUC activist in Idukki district 30 years ago in the light of Mani's speech.
While addressing a public meeting at Manakkad near Thodupuzha in the district a few months ago, Mani said CPI-M had in the 1980s drawn up a list of political rivals to be eliminated and executed that. It sparked a controversy within and outside the state.
With Mani's speech causing deep embarrassment to even the central leadership of the CPI(M), he was removed as the CPI-M Idukki district secretary and suspended from the state committee in July.