Swamy also attacked Lt Governor Najeeb Jung for "failing" to protect MPs from attack by "his government" and accused him of seeking guidance from Congress chief Sonia Gandhi's political secretary Ahmed Patel.
"Delhi government will survive if Kejriwal apologises... government should be dismissed in national interest," he said.
More From This Section
He claimed that in 1991 when he was the Union Law Minister, he "dismissed" four state governments and the decision of the then central government was upheld by the Supreme Court and supported in Parliament.
The Rajya Sabha MP charged that Kejriwal often uses the "shoot and scoot" tactic to attack political rivals. "He should either show documents (to corroborate his charges against Girri) or apologise," he said.
Asked why has he so far not sought the removal of Jung, Swamy said he was busy with the issue of RBI Governor Raghuram Rajan but now he will "talk to the government" on the issue.
Joining the dharna, BJP MP Manoj Tiwari dared the chief minister to face Girri and alleged that hurling baseless charges and going into hiding was an "old habit" of Kejriwal.
Girri has been sitting on a hunger strike since yesterday.
Kejriwal had, in a letter, accused Jung of trying to protect Girri in the murder of M M Khan, an official of the New Delhi Municipal Council, a body run by the central government.
Khan, an estate officer, was shot dead on May 16 in Jamia Nagar in south Delhi, a day before he was scheduled to clear an order on the lease terms of a hotel functioning on property leased out by the civic body.