Sultanganj Deputy Superintendent of Police Rakesh Kumar said the money was found inside an abandoned bag at the SBI ATM kiosk itself at Satsang Lane.
The security guard at the kiosk Manoj Yadav was arrested and his home was raided as his activities raised suspicion, Kumar said.
He said the money might have been kept back at the kiosk by Yadav to divert police attention from his role in the theft.
The theft had been reported to the police by the bank and its manager Abhishek Sinha had lodged an FIR against unidentified persons.