Magufuli, 55 and currently the east African nation's minister of works, will be widely expected to succeed President Jakaya Kikwete, who will be stepping down after his second and final term.
"I am proud of our candidate, he is a very hardworking man and I am certain he will be the best president," Kikwete said after ballots were counted in Tanzania's administrative capital of Dodoma.
"He is a no nonsense man. We hope he will help the country to conquer poverty, fight graft and indiscipline."
The party said Magufuli scored 87 percent of the votes, beating off Justice Minister Asha-Rose Migiro, a former UN deputy secretary general and ex-foreign minister, and Amina Salum Ali, currently the African Union's ambassador to Washington.
More From This Section
Magufuli had not initially been seen as a front runner, but was among three shortlisted from a field of nearly 40 other candidates that included Vice-President Mohamed Bilal, Prime Minister Mizengo Pinda, and former prime ministers Edward Lowassa and Frederick Sumaye.
Magufuli also named Samia Suluhu Hassan, a state minister, as his running mate. She comes from semi-autonomous Zanzibar, in line with the country's constitution that stipulates if the president hails from the mainland, the vice-president must be from the semi-autonomous archipelago.
Tanzania, with over 50 million people, is east Africa's most populous country, with economic growth of more than seven percent, according to the World Bank.