Police said this was "the first significant arrest" of the investigation into the killing of 25-year-old Yvonne Fletcher, which led Britain to sever diplomatic relations with Libya for 15 years.
The Libyan, who is in his 50s, is also suspected of money laundering, along with two other Libyans who were also arrested.
"Over the past 31 years we have never lost our resolve to solve this case," said Richard Walton, the head of counter terrorism at London's Metropolitan Police.
A rival demonstration of supporters of Libyan ruler Moamer Kadhafi had also turned out when "a number of shots were fired from within" the embassy, the Metropolitan Police said in a statement.
Also Read
Ten anti-Kadhafi protesters were wounded in the shooting.
Detectives also today appealed for eyewitnesses from that day and released 14 images of individuals they want to identify, as well as video footage of the demonstration and shooting.
"Any information, however small, can help bring closure," Fletcher's family said in a statement.
But the killer is presumed to have left Britain among the 30 staff who were then deported under diplomatic immunity.
"We believe that the incident was part of the so-called 'stray dogs campaign' being orchestrated from Libya to attack overseas dissidents and their interests during this time," the police said.
The killing led to Britain severing diplomatic relations with Libya until 1999 and has long been an obstacle in ties between London and Tripoli, along with the 1988 bombing of a passenger jet over the Scottish town of Lockerbie.
The overthrow of Libyan leader Moamer Kadhafi's regime in 2011 has given investigators new hope of solving the Fletcher and Lockerbie crimes.
In 2012, Libya's then prime minister Abdel Rahim al-Kib laid a wreath at the memorial to Fletcher, and Scotland Yard has sent detectives to Libya to continue their investigations into the killing.