Le Monde said special forces and members of the DGSE external security service were present in Libya for "clandestine operations" against IS jihadists, thought to number several thousand in the chaos-scarred country.
It also claimed that an air strike in November that killed the top IS leader in the country, Abu Nabil, was "initiated by Paris".
French officials said it had launched a probe into the leak.
"The investigation should establish if details covered by defence secrecy rules were revealed in this article," said a source close to Defence Minister Jean-Yves Le Drian.
Also Read
A violation of defence secrecy carries a penalty of up to three years in prison and a USD 50,000 fine.
France and other Western countries have agreed that military action is needed to dislodge IS in Libya, which has exploited the chaos caused by the overthrow of dictator Moamer Kadhafi in 2011.
But it wants a national unity government in place to request the assistance before making a formal intervention.