Yadav, who was lodged in Beur jail, was received by his former MP wife Ranjeeta Ranjan and a large number of his supporters outside the jail.
Ranjeeta Ranjan is now a Congress member.
After being convicted in the murder case by a lower court in 2008, Yadav was in Beur jail.
Last week a division bench of the Patna High Court comprising Justices V N Sinha and K K Lal had set aside the February 2008 conviction of Yadav and two other accused - Rajan Tiwari and Anil Kumar Yadav - by a lower court and acquitted them of all charges in connection with Sarkar's killing nearly 15 years ago.
Sarkar, a former CPI(M) legislator, was shot dead by unidentified persons in Purnia district on June 14, 1998.