The incident is of Saurbhar village under Parihar police station in the district and the deceased have been identified as Rinku Devi (28) and her son Ajit Kumar.
Family dispute is said to be behind the suicide, the official said.
Sitamarhi Deputy Superintendent of Police (DSP) M N Upadhyaya said, "The woman's husband works with a private firm in Delhi, and she was living with her parents-in-law. There was a simmering dispute going on between them for some time, and she took this extreme step."
The police has sent the bodies for postmortem.
Upadhyaya said investigations have started in the case to find whether there was any foul play involved in it.