George Orwell’s 1984 was published exactly 70 years ago today but its horrifying vision of a world in which every citizen is monitored at all times by an all-knowing, all-seeing authoritarian “Big Brother” government seems oddly closer and more threatening than ever. Driven by a combination of surveillance and data processing technologies — artificial Intelligence, deep learning, facial recognition software, Internet of Things, the world is quickly adopted the very technologies that make such a system possible.
What would Orwell think of China, which is already building a vast surveillance network designed to eventually track all 1.4 billion of its citizens through a combination of facial-recognition technology, group-chat monitoring, and smartphone apps created by the state — giving people scores based on their “social credit.” People with good scores will be rewarded, and those with bad ones will be punished. Sixteen areas in China are already using facial-recognition technology. Writes People’s Daily:
What People Don't Get about Privacy? - Data Driven Investor
You know the feeling. Whether it is for access to an app, a subscription, or your favorite pair of sneakers. You press…
The system is able to identify 40 facial features, regardless of angles and lighting, at an accuracy rate of 99.8 percent. It can also scan faces and compare them with its database of criminal suspects at large at a speed of 3 billion times a second, indicating that all Chinese people can be compared in the system within only one second.
The system is proof of outstanding performance in technology based assistance in cracking down various cases, such as drug trafficking, theft, robbery and abduction. In the past two years, more than 2,000 criminals at large were caught by public security organs with the aid of the system.
The system is part of Skynet, a nationwide monitoring program launched in 2005 to increase the use and capabilities of surveillance cameras. The system basically marries AI to CCTV surveillance and uses facial recognition and GPS tracking to spy on everybody. Currently, there are 170 million surveillance cameras in China and, by 2020, the country hopes to have 570 million. That’s roughly one camera for every two citizens.
The most frightening aspect of Skynet is its “social credit system,” also known as Citizen Score, which is linked to 1.3 billion Chinese citizens’ national ID cards, and scores each citizen on their behavior. The ACLU said the system leveraged “all the tools of the information age — electronic purchasing data, social networks, algorithmic sorting — to construct the ultimate tool of social control.”
For example, your Citizen Score, which is tied to credit scores, could go down if a “friend” in social media did or said something the government considered wrong, such as buying video games the state considered inappropriate, posting political comments without permission or posting anything that might annoy or embarrass the state.
In short, if you set out to build the most authoritarian country in the world in which every citizen’s behavior is monitored and directly controlled by the state, this is the type of system you would use. Transgressions covered range from tracking down fugitive officials spending stolen money abroad to the somewhat more mundane. In restrooms at Beijing’s Temple of Heaven Park, patrons have their faces scanned before a machine spits out a 24-inch strip of toilet paper. It will not dispense more to the same person until nine minutes have passed. It brings a whole new meaning to ‘taking a dump.’
Imagine that a western government used an AI-powered system to monitor and control the behavior of its citizens in the manner that China has openly embraced. Imagine it was used to monitor political enemies? What if the system was used to identify only members of a particular ethnic or minority or religious group? Should it ever be used to launch tear gas or smoke bombs? There are so many unanswered questions and the technology is outpacing the moral, ethical and legal choices.
1984 was published in 1949–13 months before Orwell died at the age of 46.
So widely read and influential has 1984 become that nowadays we would describe it with the adjective “Orwellian.” That is surely a tribute to the author’s understanding of the worst impulses of human nature, the cynical and cyclical nature of political power structures, and his pessimism about the ability of those in power to relinquish it and those not in power to change their predicament.