The four men were riding in a car when they were stopped by security personnel on duty outside Bilawal House in Clifton area and when questioned they pulled out weapons on the security officials refusing to come out of the car, a police official said.
"They have been detained and moved for further interrogation," the official said.
Television footages showed the luxurious car bearing a MV number plate indicating that the owner was a British national who had come to Pakistan on a visit and brought the car for personal use under the personal gift scheme allowed by custom authorities.
The footages showed one of the men in a white shirt arguing with a senior police official who than proceeded to slap him around after which the four men were taken away.
The suspicious car had tinted glasses.
Security around the Bilawal house remains tight all the time with one side of the road closed to regular traffic while a tall wall has also been built around the house as a security measure.
More From This Section
At the time the four men were arrested a meeting was going on in the Bilawal house with Interior Minister Rehman Malik and Sindh chief minister Syed Qaim Ali Shah also present.
Malik later told reporters that he had ordered a full scale inquiry into the incident and instructed the investigation agencies to find out the background of the four detained men.
Pakistan's financial hub has been hit by severe violence in recent months with target killings linked to ethnic, political and sectarian problems a regular occurrence as street crimes were also on the rise.