"My troops had completed a planning session with senior Iraqi leaders several kilometers behind the front lines," Canadian special forces commander Brigadier General Michael Rouleau said.
"When they moved forward to confirm the planning at the front lines in order to visualize what they had discussed over a map, they came under immediate and effective mortar and machine gunfire."
The general said the Canadians used sniper fire to "neutralize both threats" and there were no Canadian injuries.
The United States has previously reported having launched an unsuccessful hostage-rescue operation against the IS group in Syria, but Western forces have not officially engaged in ground combat.
Canada has some 600 troops in the region participating in airstrikes against the Islamic State. About 60 of them are special forces training and advising Iraqi troops on the ground but theoretically not in combat.