A decision to this effect was taken during a meeting Kumar had with top state police officials and home department officials at his residence this morning.
State director general of police Abhayanand and principal secretary of home department Aamir Subhani also attended the meeting.
"The chief minister has decided to hand over the murder case to CBI during the meeting", an official spokesman told PTI.
RJD supremo Lalu Prasad, LJP president Ram Vilash Paswan and Bihar Congress had demanded a CBI inquiry to probe the killing.
The Ranvir Sena chief was shot dead by gunmen while he was taking a stroll near his house in Khopira village in Bhojpur district on Friday last.