Dozens of people were also wounded in the attacks, with continuing gunfire several hours after they began.
A security guard and one other person were killed when four gunmen burst into Tehran's parliament complex with rifles and a pistol, according to the ISNA news agency.
One of the attackers blew himself up on the fourth floor of the parliament office building as a standoff with police continued for several hours.
Two attackers -- at least one female -- blew themselves up outside the shrine, according to local media.
More From This Section
Iran's emergency services said they were dealing with 33 injured from the attacks and that two people had died from their injuries in hospital.
IS claimed the attacks in a report from its Amaq propaganda agency which said: "Fighters from the Islamic State attacked the Khomeini mausoleum and the parliament building in Tehran."
Parliament was in session as the attacks unfolded, with live footage showing members continuing with routine business even as gun battles were reported in surrounding office buildings and snipers took position on nearby rooftops.
Speaker Ali Larijani dismissed the attacks, saying they were a "trivial matter" and that security forces were dealing with them.
Intensified gunfire was heard from the neighbouring offices as Fars news agency reported police had launched an assault. A picture on social media showed police helping staff escape through windows.
An official at Khomeini's mausoleum in south Tehran said "three or four" people had entered via the western entrance and opened fire, killing the gardener and wounding several people, according to Fars.
The shrine lies around 20 kilometres away from parliament and houses the body of Khomeini, who led the Islamic revolution in 1979.
The ILNA news agency said security forces were dismantling a bomb inside the shrine and that firing was still going on around the site.
The intelligence ministry claimed there was a third "terrorist" team that had been neutralised before the attacks started.
The city was on lockdown, with streets blocked and parts of the metro closed. Journalists were kept away from the shrine by police.
Jihadist groups have clashed frequently with security forces along Iran's borders with Iraq and Afghanistan, but the country has largely escaped attacks within its urban centres.
Iran, the predominant Shiite power, has been helping both Iraq and President Bashar al-Assad's regime in Syria to battle IS.
The jihadist group is under increasing pressure in both countries, having lost significant territory in the face of offensives now targeting its last two major urban bastions, Raqa in Syria and Mosul in Iraq.
IS published a rare video in Persian in March, warning that it "will conquer Iran and restore it to the Sunni Muslim nation as it was before."
Militant groups are also known to operate in Iran's southeastern Sistan-Baluchistan province, which borders Pakistan and has a large Sunni community.
Jaish-ul Adl (Army of Justice), which Tehran accuses of links with Al-Qaeda, has carried out several armed attacks inside Iranian territory in recent years.
Iran's supreme leader Ayatollah Ali Khamenei said on Sunday that attacks by IS in Europe and elsewhere showed that Western policies in the Middle East have backfired.
"This is a fire that (Western powers) themselves ignited and now has backfired on them," he told a gathering of senior officials in Tehran.