Facebook AI bots developed their own language beyond human comprehension

The author debunks the notion that this is a cause for alarm

What happens if you put a bunch of children online, say on a closed WhatsApp group, and tell them to barter books, hats and balls? They’ll negotiate; they’ll develop personal “valuations” for the goods they’re trading; they’ll drive better bargains as they develop more negotiating skills.  Along the way, they’ll develop their own jargon. That’s exactly how financial traders and auctioneers operate, using complex hand-gestures (in physical auctions) and linguistic shortcuts to trade efficiently. 

The amazing thing is that artificial intelligence (AI) might develop negotiation systems along much the same trajectory. That’s indicated by the outcome of an interesting experiment run by Facebook’s Artificial Intelligence team (described at Deal or no deal? Training AI bots to negotiate). 

Facebook set two AIs to barter for hats, balls and books, giving each AI, or “agent”, pre-assigned different values for such objects and forcing them to negotiate in adversarial ways. The AIs learnt negotiation skills, they learnt to drive hard bargains, using tactics that included trying to fool each other. They also developed their own trading jargon that was incomprehensible to the researchers. 

Some of the dialogue between the AIs was funny, and baffling in equal parts (see illustration for part of the conversation). 

This is not gibberish. It made sense to the AIs. The proof of the pudding lies in the fact that the negotiations ended successfully. The AIs also learnt sneaky tactics like pretending that one good was valuable to them so that they could later pretend that they were sacrificing a lot while bartering it for some-thing else. 

Facebook eventually turned off the experiment. The researchers were hoping to create agents that could negotiate seamlessly with human beings. Facebook wanted to create a chatbot, which could negotiate deals fluently enough to pass the Turing Test — humans would not realise the negotiator was a robot. Facebook subsequently rebooted the experiment successfully using AIs bounded by one stipulation: they had to stick to conventional English. 

Fearmongers have gone to town over the experiment by saying that this development of language was a fearsome new capacity. Indeed, this misreporting feeds into fears that several well-known scientists and engineers including Stephen Hawking, Bill Gates and Elon Musk have expressed: one day AIs will be more intelligent than their human creators and, once that “Singularity” occurs, they might take over.    

Hawking once cautioned that AI could “take off on its own and re-design itself at an ever increasing rate. Humans, who are limited by slow biological evolution, couldn’t compete, and would be superseded.” That’s an informed, if alarming, opinion. 

But the misreporting turned a well-known computer ability into terrifying science fiction. In reality, as Facebook researcher and Georgia Tech professor, Dhruv Batra, pointed out that “while the idea of AI agents inventing their own language may sound alarming to people outside the field, it is a well-established sub-field of AI, with publications dating back decades. Agents will drift off understandable language and invent codewords. Like if I say ‘the’ five times, you may interpret that to mean I want five copies of this item. This isn’t so different from the way communities of humans create shorthands.”

There are many machine-constructed languages. Among the most intriguing examples is that invented by the Google translation tool. Google researchers wanted to know if a computer that could translate, say, Japanese to English and English to German and German to Russian could directly translate Japanese to Russian without any link-language. They discovered this was possible. The machine stored what it classified as similar linguistic “concepts” at some deep level. 

We’ve also seen the invention of shorthands multiple times in the cellphone-internet era. As cellphones came into vogue in the early 1990s, Finnish mobile-maker Nokia discovered the value of texting. Nokia gave phones to a bunch of kids and found that they preferred exercising fingers to exercising vocal cords. What’s more, the kids invented argot and abbreviations that left adults baffled. 

Those Nokia beta-testers were the first generation of native cellphone users. Nowadays everybody texts (or uses instant messengers) and everybody knows that kids will text in jargon incomprehensible to their older siblings, let alone parents. 

AI may eventually become more intelligent than humans and, of course, smart machines have already changed society in many ways. But the era of super-intelligent machines is far away. Worrying about it has been compared to worrying about overpopulation on Mars.