More than 500 people gathered in Jerusalem in front of a screen reading in French "Jerusalem is Charlie", an AFP correspondent said.
Participants observed a minute's silence, holding signs reading in French "I am Charlie" and "I am a French Jew", and in Hebrew "Israel is Charlie".
"This is an attack on all of us -- on the Jewish people, on freedom of media and expression. We are all French Jews, Jerusalem is Charlie," Jerusalem Mayor Nir Barkat said.
But for some participants, the massive show of support in France following this week's attack on the offices of satirical weekly Charlie Hebdo was tinged with bitterness.
More From This Section
Pierre Besnainou, a longtime leader in the French-Jewish community who recently immigrated to Israel, said such an outpouring would have been welcome after previous attacks targeting French Jews.
"The mobilisation after Charlie leaves a bitter taste, since we felt very alone after Toulouse," he said, referring to attacks in which an Islamic extremist shot dead three children and a teacher at a Jewish school in 2012.
Dozens of Palestinians also held a rally in the West Bank city of Ramallah, as Palestinian president Mahmud Abbas participated too in the Paris march.
Waving Palestinian and French flags, they held up banners reading "Palestine stands with France against terrorism."
Gaza paid tribute to the victims during a candlelit vigil in the enclave. Hamas, the de facto rulers of Gaza, condemned the assault on Charlie Hebdo in a statement yesterday which failed to refer to the subsequent attack on the Jewish supermarket.