China’s Social Credit System: A Step Towards Dystopia? Part One: Introduction

This year I’ve taken an increasing interest in AI-related news from China. It escalated when my little brother moved to Shenzhen for work a few months ago, and since then, I’ve gone down a fantastic rabbit hole.

Here’s the situation: four hundred billion dollars (!!!) in government AI funding, huge investments in engineering talent, and over a billion people generating data with loose privacy restrictions. Basically, Disneyland for big data.

These circumstances lead to a set of unique questions. The most significant of these are about the “social credit system” (SCS) , a technical and social project taking place on the largest scale in the world…maybe ever.

When people first hear about this concept — every citizen given a government-issued mark which determines their standing and value — it invokes images of the Black Mirror episode Nosedive, where life is a never-ending nightmare of pleasing others to gain status.

A scene from Nosedive where the main character receives a rejection based on her score.

The SCS is a huge piece of the $400 billion China is investing in AI over the next decade. No matter what direction it goes in, it changes life for every person in China. By extension, its gravity is felt worldwide.

As you’ll see, the current state of the SCS, thankfully, is quite far removed from Nosedive. But that doesn’t mean it’s entirely out of the question — the idea of using technology to monitor and punish citizens is certainly appealing given the philosophy of the Chinese Communist Party, which views itself as both the legal and the moral authority over its citizens. (This level of involvement is the status quo, which can come as a surprise to Westerners.)

The heart of Shenzhen

Over the next several blog posts I’m going to walk through all of the important pieces of the SCS, with some thoughts, questions, and commentary interspersed.

Unsurprisingly, most of the existing literature comes from China. This has presented a language barrier, but recently we fell into some luck: the illustrious Rogier Creemers compiled 118 (!) sources into an English paper called China’s Social Credit System: An Evolving Practice of Control. It can be found here.

Let’s get started with the historical context behind the SCS.

Next: Part Two: Historical Context

Part One: Introduction
Part Two: Historical Context
Part Three: 2014 Social Credit Plan
Part Four: Joint Punishment System
Part Five: Barriers
Part Six: Looking Ahead