Interim Prime Minister Arseniy Yatsenyuk vowed Ukraine would not cede "an inch" of its territory to Moscow after Russian forces and pro-Kremlin gunmen took over the Black Sea peninsula.
"This is our land," Yatsenyuk told a crowd of several thousand in the capital that was also attended by Russian former oligarch Mikhail Khodorkovsky, who spent a decade in prison in Russia and is a top critic of President Vladimir Putin.
Illustrating the divisions in Ukraine, interim president Oleksandr Turchynov led a minute of silence at the Kiev rally for demonstrators killed in three months of protests that led to the ouster of pro-Kremlin president Viktor Yanukovych.
In contrast, in the eastern city of Donetsk pro-Moscow activists paid tribute to a feared riot police unit accused of shooting at protesters in clashes in Kiev that left around 100 dead late last month.
More From This Section
"Russia! Russia!" the activists waving Russian flags shouted in Donetsk, the heartland of the former president, who fled to Russia after his overthrow.
"They cannot seize Crimea, (it's an) illegal occupation," said Svyatoslav Regushevsky, a 46-year-old who attended the unity rally with his two-year-old son, wearing a ski jacket in the colours of the Ukrainian flag.
Around 1,000 people were at that demonstration, while some 10,000 turned out for the rival one on Lenin Square across town where a sea of Russian tricolours could be seen.
"We don't want to be with those Ukrainian fascists any more," said Olga, 60, who declined to give her surname, while around her placards read "Crimea is not in Ukraine" and "With Russia, Peace in Crimea."
The rallies come after a new marathon round of phone calls by US President Barack Obama seeking to defuse the standoff with Russia, with each side imposing sanctions or warning of further prohibitive measures to force the other to back down.