The incident took place in Baya village.
The woman, identified as Reena Devi, jumped into a pond with her three minor children in order to end their lives, the Deputy Superintendent of Police (DSP) M N Upadhyaya said.
While two of her children drowned, the locals rescued the woman and her remaining child, he said.
The bodies have been sent for post-mortem, Upadhyaya said.
Frequent quarrel in the family is stated to be the motive behind the extreme step taken by the woman, he said, adding the matter is being investigated.