China 2020: George Orwell’s Unwritten Sequel
George Orwell’s 1984 imagines a London ruled over by a totalitarian regime. One that has countless, sleepless video cameras spying on its citizens. If you fail to adhere to the government’s rules then the thought police will swiftly swoop in and sort you out — and not in a good way. You’re not allowed to have close friends, keep a diary, or fall in love. It’s a dystopian vision that I’m sure none of us would want to live through — except maybe Kim Jong Un. Until now the level of mass-surveillance imagined in the novel has been a technical feat gracefully out of humanity’s reach. But, we’re an innovative bunch of atoms, us humans, and we’re getting worryingly good at making surveillance systems.
SenseTime, a startup based in Beijing, recently raised $600 million at a valuation of $3 billion. They are now the most valuable artificial intelligence (AI) startup in the world, having doubled their valuation in only a few months. SenseTime are amongst a core of artificial intelligence companies that Beijing hopes will drive them to become the world-leading centre for AI by 2030.
You might have guessed what SenseTime do, what their business is. The 1984 intro maybe gave it away. Note to self, be more mysterious in future.
SenseTime develops face recognition technology that can automagically identify people from pictures and video footage. From a technical point of view, this is really awesome work. Harnessing artificial intelligence to accurately identify people from less than crystal clear video is very impressive.
The company’s technology is already being used by Chinese police. A couple of weeks ago a fugitive was caught at a music festival in Jiangxi province when security cameras recognised him, thought to them-silicon-selves, that yes this is that bad man who committed economic crimes, and alerted the police. The man was shocked when he was scooped up amidst a crowd of over 50,000 people. But he should’ve known better, he should’ve read his tech news. Last year 25 wanted people were caught at a beer festival when cameras at its entrances flagged them to security. And then 7 more were caught by police in Zhengzhou testing out sunglasses with built-in facial recognition tech.
Catching criminals with this kind of technology doesn’t seem too sinister - after all criminals are criminals. But what happens if ‘crime’ takes on a broader definition. Can we trust an undemocratic state to act responsibly? Shouldn’t some degree of privacy be an inalienable human right? Indeed, this tech has its critics; civil libertarians are already reporting that facial recognition systems have been used in Xinjiang to track activists and oppress minorities. Oh but don’t worry too much, SenseTime’s co-founder Xu Li, says the tech “will not affect privacy because only authorised persons can access it”. Yeh, but who is doing the authorising Mr. Li, huh? That’s correct, not the people you’re surveilling.
Until now SenseTime’s systems have only been able to work on a relatively small scale. To rectify this the company is developing a new service, Viper, which will be able to interpret 100,000 live camera feeds simultaneously. Ponder that for a second. Or maybe don’t.
Let’s think of something else. It’s not any less scary though. China is developing a social credit system. It’s already being tested across 3 dozen cities, and the government has promised to roll out the scheme nationally by 2020. China says that the system, which mixes traditional western-style credit scores with more expansive and 1984-esque measures, will help to promote “trustworthiness” in its economy and society. Get a speeding ticket and you lose 5 points, help out your family during a period of unusual difficulty and you gain 30 points. On the surface, the scheme seems pretty cool. It effectively formalises the consequences of social niceties and not-niceties that we perform in our day to day lives.
Citizens with high social credit scores enjoy perks such as more favourable bank loans and cheaper heating bills. Citizens with low social credit scores find life quite difficult, not being able to buy high-speed train or plane tickets, amongst other punishments. Rongcheng is the country’s most successful trial city for social credit. Here the citizens have embraced the scheme because they see it as sensible. At the moment the system only deducts points for breaking the law and only rewards people for verifiable and documented acts. The people appreciate its precise punishments and generous rewards.
Indeed, it is complicated to criticize China’s forays into social credit, because in many cases they address legitimate concerns and do so elegantly. Some think that we shouldn’t worry about it, that social credit is not all that novel, that it is just formalizing the way the party already operates: automating existing controls and the monitoring of Chinese citizens. But there’s something wrong here, something that doesn’t feel quite right. Just listen to the scheme’s founding doctrine,
“allow the trustworthy to roam everywhere under heaven while making it hard for the discredited to take a single step.”
In a time when the Chinese Communist Party are becoming more and more present in the lives of their citizens, it is hard not to see social credit as another tightening of freedoms.
China has long experimented with social control, but it has never had the technology to meaningfully support such a system. In times gone by citizens might have been encouraged to spy on their neighbours. An encouragement that never scaled very well. But soon China will have a plentitude of artificially intelligent video cameras able to do the spying for them. And this will scale. It is not hard to imagine that information captured by the country’s surveillance network will feed into people’s social credit scores. In a country that already does its best to control the thoughts and actions of its people, it is not all that exciting to think that their ability to coerce is only set to get stronger.
It is estimated that in less than two years China will have more than 600 million CCTV cameras. That’s more than one for every three citizens. George Orwell’s 1984 was published in 1949, perhaps it is time for its non-fiction sequel.
This piece was transcribed from The Zip Files — an irreverent weekly 20–25 minute podcast that I produce to help the busy millennial catch up with all of the week’s most important tech news. Here’s the episode in which this piece was featured:
On Apple Podcasts:
On Stitcher:
