Shinde was asked to respond to reported remarks by Singh in an interview in which he had claimed that he had quarrels with his boss during his stint in the Home Ministry.
Singh, who joined BJP last month, had also said a complaint was lodged against him with the Cabinet Secretary.
Also Read
Asked to reply "truthfully" at a news conference, Shinde said "one needs to run administration very well and if anybody fails to do his duty, I am bound to take action against him in the capacity as Home Minister."
"I have to scold police and secretary and anybody who fails to do his duty," the Minister said with the security breach in which Jat protesters had barged into his residence last year apparently in the back of his mind.
And as he concluded his reply, he turned to the questioner and asked, "I hope you are satisfied with my truthful response."
Singh, during his media interview, had claimed that he had many quarrels with Shinde over various issues and also alleged that Congress-led UPA government was "clueless" about governance and that its ministers were steeped in corruption.
In what was seen as a snub to Singh, Anil Goswami was also named as his successor weeks before the end of the incumbent's superannuation.
Singh was also criticised for not making any attempt to inform the family of Afzal Guru, the death row convict in Parliament attack case, who was hanged on February nine last year.