Prime Minister Stephen Harper was not in the Langevin Building at the time, Royal Canadian Mounted Police Cpl Lucy Shorey said.
Ottawa Police Const. Chuck Benoit said there was commotion and a package was left after the suspect said he would do something.
"There were certain things that he mentioned. He was arrested at that point," Benoit said. He said he didn't have further details and could not say whether the man threatened the prime minister.
The historic Langevin Building, which dates to 1883, sits immediately across the street from Parliament Hill.
The incident comes as the government prepares for today's speech in Parliament, which sets out its parliamentary agenda for the forthcoming House of Commons session.