The capture came as UN Secretary General Ban Ki-moon expressed anger and shame at the world's failure to stop the conflict.
More than 215,000 people have been killed since anti-government protests, which erupted in March 2011, were brutally repressed by President Bashar al-Assad's regime civil war followed.
The country has been ravaged by warring factions, including jihadist groups.
The coalition that seized Idlib city is made up of Al-Nusra Front, the official Syrian affiliate of Al-Qaeda, and several Islamist factions.
More From This Section
"There is still a group of soldiers fighting in the security quarter of the city, but they will not be able to reverse the situation," Observatory director Rami Abdel Rahman added.
The jihadist group also announced the capture of the northwestern city on its official Twitter accounts.
"Thanks be to God, the city of Idlib has been liberated," it wrote.
The group posted photos of its fighters in front of the governorate building, the city council, a local prison and a police station in the city.
Official news agency SANA said only that "army troops were regrouping south of the city of Idlib in preparation to face an influx of thousands of terrorists coming from Turkey."
The government has regularly accused Turkey, a backer of the uprising against Assad, of providing support and sanctuary to "terrorists."
The first provincial capital to fall was Raqa, in the north, which was seized by rebels in March 2013.
The rebels were subsequently ousted by the Islamic State group, which has made Raqa the de factor Syrian capital of its self-proclaimed Islamic "caliphate" in Syria and Iraq.