The guard, identified as Deepak Bahadur, had entered the house in Jamia Nagar on the intervening night of October 26-27, a senior police official said.
He was arrested after a complaint filed by one Nafees that his rifle, 20 live bullets, one LED TV, one LCD TV, two mobile phones and six watches had been stolen.
Nafees had told the police that he suspected Deepak in the incident following which they launched a manhunt to nab the accused and prevent him from escaping to Nepal.
He was apprehended last week at Nehru Place Bus terminal with stolen clothes and seven watches and he has allegedly confessed to his involvement.
Based on Deepak's confession, his relative Ramesh Bahadur was also taken into custody for aiding him to hide the stolen articles, the police official said.
The duo have been sent to judicial custody.