The incident at St Alphonsa's church came just five days ahead of the Delhi assembly elections on Saturday.
Police said the unidentified persons entered the church premises at around 1 AM by breaking open its main gate and damaged a number of sacred items. A case of theft has been registered, police said.
"We are investigating the incident and scanning the footage of CCTV installed near the church to identify the accused. We are probing all the angles in the case," said a senior police official.
"It has been happening for the last many months. It happened in St Sebastian's Church in Dilshad Garden, then in Vikaspuri and in Jasola. This is the fifth church," he said.
Also Read
The Union Home Ministry sought a report from the Delhi Police on the incident.
In a communication, the Home Ministry also told the Delhi Police to send a report as early as possible informing it what steps are being taken to ensure safety of religious places across the city.
The Delhi Police has also been asked to inform how many arrests have been made in attacks on religious places in the last six months, sources said.
"Another act of vandalism and targeted attack on our churches in Delhi is nothing but a reflection of hate campaign and false propaganda by groups whose sole aim is to break the religious harmony and social peace of this great nation.
"The attack within a week after celebration of Republic Day reflects a lot on the government and its failure to give protection to minorities and their religious structures," Couto said.
He alleged that the "growing" number of attacks on churches in different parts of the country was aimed at creating a fear psychosis among the community.
In December, the St Sebastian's Church in Dilshad Garden, was partly gutted and the Christian community had alleged foul play in the incident.