Bhogiram Dhanak (45), the victim, was undergoing sentence in a rape case, said the jail official Jyoti Tiwari.
Rajesh Adivasi (26), a murder accused, asked for a bucket when Dhanak was washing clothes, and when he refused, Adivasi hit him on the head with another bucket, she said.
Dhanak was immediately rushed to the hospital where he succumbed during the treatment, she said.
Sub Divisional Magistrate and Jail Superintendent D K Singh said he was conducting probe into the incident, and measures to prevent such incidents will be taken.
Dhanak's relatives, meanwhile, alleged that his enemies had him killed by hatching a conspiracy.