The activists reached a newly constructed house in Tiuri village that was being used as church and alleged that forcible conversion of Dalits to Christianity was going on there.
They also caught hold of a person, Ajmon Abraham from Kerala, for his alleged involvement in the conversion. When police reached there the activists handed over Abraham to them.
SHO of Aurai police station Onkar Singh Yadav said a probe into the matter was on and Abraham was being interrogated.
District in-charge of the HVY Surabh Sharma claimed that forcible conversion was going on in the house and administration was informed earlier about it.
Last month, police had stopped a church event attended by more than 150 people, including 10 American tourists, in Maharajganj after the HYV alleged that religious conversion was being carried out.
The right-wing group was set up in 2002 by Yogi Adityanath, now the Uttar Pradesh Chief Minister.