Mazen Faqha was released along with more than 1,000 other Palestinians in exchange for Gilad Shalit, an Israeli soldier Hamas had detained for five years.
Iyad al-Bozum, an interior ministry spokesman in the Hamas-ruled Gaza Strip, said that gunmen opened fire on Faqha in the Tell al-Hama neighbourhood.
"An investigation has been launched," he said, giving no further details.
Police spokesman Ayman al-Batniji said Faqha had "four bullets in his head" and said Israel and its "collaborators" were responsible for the killing.
Faqha was a senior Hamas official in the Israeli-occupied West Bank but after his release Israel transferred him to Gaza.
The Israeli army refused to comment. But Khalil al-Haya, a senior Hamas official in Gaza, said: "This assassination does not serve anybody but the occupiers (Israel), it is of no interest to the other parties.