Sources said the building housing the Evangelical Church of West Africa (ECWA) in Peri village near Potiskum in Yobe state where a service was going on was also set ablaze by the attackers.
The attackers killed six persons, including the congregation's pastor.
A police source confirmed the incident.
Militants from a fundamentalist Islamic sect, Boko Haram has killed thousands of people in northern Nigeria since 2009.
They have repeatedly targeted Christian worshippers in churches. Christmas celebrations too have come under attack by the terrorists.
The sect wants to install an Islamic government in northern Nigeria where Muslims are in a majority.