Duminy forced an edge from Mominul Haque for 40, ending the batsman's 69-run second wicket stand with Imrul Kayes.
Kayes followed Mominul, who hit six fours off 87 balls, when Duminy struck in his next over, trapping the left-hander leg-before for 30.
Captain Mushfiqur Rahim and Mahmudullah Riyad then steadied the innings, sharing a 68-run unbeaten partnership to be 42 and 25 at the break, after Bangladesh won the toss and elected to bat at the Sher-e-Bangla National Stadium.
Steyn struck early in the day, taking opener Tamim Iqbal's wicket, and claiming his 400th scalp in Test cricket.
More From This Section
The 32-year-old Steyn almost reached the landmark in his second over of the day but Dean Elgar dropped Tamim at second slip.
Skipper Hashim Amla made no mistake in Steyn's next over when he took a chest-high catch at first slip.
Shaun Pollock is the only other South African to pass the 400-mark, taking 421 wickets before retiring in 2008.
He is one of only three currently active cricketers to achieve the feat, joining India spinner Harbhajan Singh and England paceman James Anderson.
South Africa handed wicketkeeper-batsman Dane Vilas a Test debut in the match, replacing Quinton de Kock.
The 30-year-old became South Africa's sixth new cap in the last 18 months.
Bangladesh brought back Nasir Hossain in place of left-arm spinner Taijul Islam.
The first Test ended in a draw after rain washed out the last two days in Chittagong.