The attack took place late Thursday in the northern Tanzanian town of Mwanza, on the shores of Lake Victoria.
"Police are holding three people in connection with the incident," local police chief Ahmed Msangi said, but gave no details for any possible motivation for the killings.
Around a third of Tanzania's 51 million people are believed to be Muslim and around a half Christian although there are no official figures.
Communities traditionally live peacefully side by side, but there have been sporadic attacks on both mosques and churches in recent years.