The suspension of Rished Baden came after newly elected Prime Minster Kassim Majaliwa visited the main port in Dar es Salaam, and found 349 shipping containers listed as having arrived in the port had vanished.
Senior officials in the revenue authority and customs have also been suspended pending investigations, according to official statements quoted in Tanzania media.
Tanzania's President John Magufuli, who took power earlier this month after winning October 25 elections, has introduced a swathe of austerity cuts and crackdowns on public corruption.