He met External Affairs Minister Sushma Swaraj at her residence before taking the charge.
Akbar is the second Minister of State in the External Affairs Ministry after V K Singh.
"A welcome addition to MEA as @mjakbar takes charge as Minister of State (External Affairs)," MEA Spokesperson Vikas Swarup tweeted along with a photograph of Akbar taking charge.
65-year-old Akbar, who was recently elected to Rajya Sabha from Madhya Pradesh, is seen as an articulate and modern Muslim voice in BJP.
He was a party spokesperson and has often spoken on the government's foreign policies.