AI is already harming us — but not the way you think — Project AGI

As AI/ML researchers, we have obviously pondered the risks of AI. We even wrote about it. But what might surprise you is the risks that keep AI/ML folk awake at night aren’t the ones you hear about in the media. We’re not worried about runaway “paperclip maximizers” or “skynet”-style machine coups. Andrew Ng says: “worrying about the rise of evil killer robots is like worrying about overpopulation and pollution on Mars”.

Researchers are worried about far more immediate dangers, such as:

• The social consequences of improving AI — distribution of wealth, economics, and unemployment
• The political consequences of cheaper, faster, better AI — especially propaganda and conspiracy theories
• Bias, for practical or accidental reasons
• Poor reproducibility (because code, data, methods)
• Open access to scientific research, so that everyone can benefit and be protected
• Concentration of resources and talent in a few organizations, and simultaneously lack of diversity
• Siloing of research expertise & methods leading to stagnation
• Hype, and the impact of fallout on the industry

Anecdotally, there’s high awareness of community issues and social responsibility in AI/ML. As such, I believe we do not as a group fulfil the “mad scientist” stereotype, doggedly pursuing an intellectual goal without consideration of the costs.

Let’s focus on the social and political consequences.

The debate over AI safety hit the headlines again when OpenAI published their GPT-2 language model, including source code* and explanatory paper, but withholding the training data and big trained models. GPT-2 is very much in line with our current direction, i.e. using unsupervised next-input prediction as a training regime — so this is of intense interest to us.

* The code is stripped of debug outputs and control options which makes me think it’s actually a simplified version that would be hard to develop further — you’d have to restore all the extra stuff to actually monitor how it’s training and learning when applied to a new dataset. So there’s still some technical hurdle for abuse, beyond money.

OpenAI was created expressly to mitigate some of the AI risks listed above by being open — publishing open access research, code, models and methods. But in this case, they felt that releasing the trained models and data would be too risky and hence deviated from their normal procedures.

So what was OpenAI worried about? Next-word prediction doesn’t sound very scary… is it?

On the Internet, no one can tell if you’re a Bot

The model trained by OpenAI is a generative model. This means that it can generate text by repeatedly predicting the next word. It’s remarkably good at this, in one example generating a story about unicorns that maintained a consistent narrative for hundreds of words.

On social media, comments & posts are typically much shorter than this, which means that OpenAI’s model could generate content that isn’t obviously written by a machine. This is a significant and scary advance — it means that anyone with the model and deep enough pockets could flood online communities with tailored propaganda, and you wouldn’t be able to fight back by filtering all this robot content. Even forum admins would have a tough time working out what’s human and what’s not. The end result is an Internet filled with bots, drowning out human voices.

We see clear and present dangers posed by convincing AI/ML bots. They allow narrow but well-funded interests to dominate online debate. This damages our ability to perceive the public mood, affecting news media sentiment and direction. It also affects government policy in democracies. Worse, bots can directly modify public sentiment by creating the appearance of uncertainty where there is none (c.f. Climate change) or by exploiting and amplifying social rifts — excluding out-groups, increasing fear and division, and targeting victims as enemies.

Social media has often been dismissed as irrelevant to the course of nations. But increasingly it appears to be a target for weaponized software that attacks society rather than industrial or military infrastructure.

I want to believe

Recent research has identified YouTube’s content recommendation engine (a typical ML system) as being a key driver of fake news including anti-vaccination scare stories and conspiracy theories about politics & aliens.

But YouTube’s recommendation engine isn’t designed to be evil. Instead, it’s more likely that these psycho-social harms are an indirect consequence of the drive to recommend content that is (a) popular and (b) similar to things already watched. In the same way that the concept of shareholder-value-maximization leads to psychopathic or psychotic corporate behaviour, the successful recommendation algorithm inevitably leads people down dark rabbit-holes into echo-chambers of bizarre, extreme opinions.

That’s not to say that the big tech platforms are entirely innocent. To varying degrees, they are aware or culpable for the consequences of their design choices. As an industry, we must do more to understand and eliminate the negative consequences of improving technology, while maximizing the benefits to society as a whole. We also need to shout out when our work is used for nefarious purposes.

Despite good intentions, AI risk-awareness groups such as MIRI may cause more harm than good by focusing public debate on the more distant existential risks of AI, while distracting from the immediate risks and harm being perpetrated right now using AI & ML. These harms are likely to escalate rapidly in the near future. At least, we should ask whether the fear of strong AI is, today, a greater risk than strong AI.

As I understand it, OpenAI’s strategy is to withhold dangerously effective technology to allow time for mitigating defences to be developed and deployed. This is a delaying tactic, which can buy us time, but not victory. We’re in a new arms race — and so far — the bad guys are ahead.

One defence to keep online communities democratic might be increased requirements to prove one’s humanity. Effective methods are somewhat limited right now, because tools like captchas are intrusive and inconvenient; plus we lack a persistent way to trace identity between sites, to reduce the need for repeated “proof-of-human”. There’s also a huge risk that persistently identifying people introduces new harms, such as limits to freedom of speech or association. User-controlled pseudonymity, as implemented by Bitcoin addresses, might be the ideal balance of social risk versus individual freedom.

Originally published at agi.io on March 19, 2019.