Digital Migrants, Virtual Reality, and Machine Learning

This post is by Louis Hyman, an associate professor of economic history and director of the Institute for Workplace Studies at Cornell University and author of the recent book, Temp: How American Work, American Business, and the American Dream Became Temporary.

Three years ago, in the spring of 2016, I visited a cutting-edge robotics lab at the University of California, Berkeley. On YouTube, I had seen this amazing video of a robot folding towels. Now, a towel-folding robot doesn’t sound that impressive to us, because as humans we have great vision (and lots of experience folding towels). But folding a towel is a very difficult problem for a robot. Simply seeing the edges of the towel is tricky using computer vision. In bad paintings, fabric looks flat, while in the paintings of the best Dutch masters the fabric flows in convoluted colors and shadows. Reality, it turns out, is superior to Rembrandt. Robots have a hard time seeing a towel. Edges, particularly on a clumped towel, are hard to find. Once the robot manages to find the edge, even folding the towel is still hard. The folds have to be done in a particular way that cannot be programmed ahead of time.

This towel folding is a great example of the progress in “narrow AI” and “machine learning” that has happened in the last few years. “Narrow AI” is the more mundane cousin to the “general AI” of self-aware machines, like Skynet or Westworld. For decades, AI researchers thought the secret to artificial intelligence was to create ever more complicated rules. It turns out that the secret is just to show a machine lots of examples. This kind of AI is not deductive, but inductive, and it requires massive amounts of data. A very simple, very generic algorithm then figures out what makes those examples successful. The secret of this lab, then, was to teach robots how to find the edges of a towel by showing examples — lots of examples — of towel folding.

A graduate student puts a towel in front the robot, moves its pincers to grab and fold the towel — over and over again. I was told they had to do it about 100 times for the robot to start to be able to fold the towel. Through this process, the robot slowly builds up a mental map of what it thinks its task is. No one directly tells it how to do it, or even what the goal is. Every time is a little different (just like towels) but the robot adapts. If you want to see this, just go on YouTube and search for towel robot. It is pretty neat. I was excited to see it in action.

For an hour, I sat there as a very patient graduate student put the towel on the table, hit the button, and the robot attempted to fold the towel.

Again. And again. The attempts took forever. The towel, after that hour, remained unfolded. Though it worked sometimes, it didn’t work then. Something was off about the towel that day. Given enough training, though, the robot could have folded the towel every time, if it had seen enough examples (just like humans learn to fold towels).

While the robot was cool, I had no fear of robots replacing towel-folding employees anytime soon.

Another graduate student offered to let me play with the robot using the new virtual reality rig that she had just gotten. It was an HTC Vive and this was weeks before either it or the Oculus was available in stores. I put on the googles, I grabbed the controls, and suddenly I was looking through the eyes of the robot at the table with the towel. Though I had never used a VR control before, I reached down with the robot’s arms, grabbed the towel, and folded it. My speed was only limited by the speed of the robotic arms. What the most advanced robot could not do, a human could do nearly instantly through cheap virtual reality.

This changes everything.

Machine learning works best with lots of examples, and while a grad student can move a robotic arm a few hundred times (or maybe even a few thousand times), they can’t do it a million times. With millions of examples even complex tasks become machine learnable. Today, towels; tomorrow, cars.

In 2016, Tesla, the electric car company, announced that its controversial “autopilot” had driven customers over 200 million miles. That is a truly astounding number for a project that had been launched only two years earlier. Meanwhile, Google’s program, started over a decade ago, had driven paying customers an equally astounding zero miles. Why was Tesla able to advance so quickly while Google, with all of its money, brains, and head start, unable to catch up?

The answer is important not just for self-driving cars but for the future of work in general. This machine learning is how Tesla could scale up its autopilot so quickly. From millions of miles of human driving, the car’s eyes recognized what successful driving looked like. Google tried to train its cars directly. With its own employees. It took years.

Tesla simply sold its cars, and let the drivers train the cars. It took months.

The business of machine learning is getting somebody else to create the data that your business will own. Like the rest of today’s business, machine learning is about subcontracting the work and claiming the rewards.

I realized, in that towel-folding moment, that VR operators could train robots to fold towels.

And that will change everything.

But why would anyone use VR to control a towel-folding robot when low-wage workers (especially migrant workers) are so cheap?

Sometime in the next few years, an entrepreneur will realize that while no robots can do household or workplace tasks, human-operated robot bodies can.

Some entrepreneur will realize that in the U.S., Europe, and the Arab States are vast markets for service labor (the International Labour Organization estimates that 71 percent of all migrant workers are in the service sector).

At the same time, those countries, caught in the midst of a vast political crisis, want fewer immigrants in their countries. A robot who can replace a migrant laborer will have a clear value proposition, both economically and politically.

These workers don’t even need to be in the same time zone. Bangladeshis, Mexicans, and every other migrant population could, though robot bodies, fold towels in Miami hotels or pick onions in Alabama. These robot bodies will be able to do any task a human can do, but unlike a human worker whose body and mind come bundled together, the mind of the robot’s body can be replaced with a click of a button. Workers who don’t perform can be replaced instantly. Workers in these bodies will be able to work 24 hours a day. Owners will not need to worry about theft or unionization or desertion or anything else, since the workers can be replaced instantly.

All those physical jobs that “can’t be moved overseas” will be moved overseas.

The business model is simple: Sell a robot body to consumers, who then pay for cheap overseas operators to run the body. A digital labor platform like fiverr will find them for you. Meanwhile, the company keeps track of how that body is being used, just like Tesla, and uses that data for machine learning. Instead of owners training Teslas, it will be workers training the robot servants. With VR goggles and hand controls, these overseas workers will be a digital migrants, training the robots to replace them.

More importantly, as these overseas workers drive the robots, the machines will learn. With millions of examples these overseas VR operators will just be a transitional labor force.

This feedback loop of VR/machine learning/cheap labor will rapidly bring self-working robots into our lives in a way that, without those millions of hours of training, could never happen.

Instead of 100 towel folds by an expensive PhD student, cheap overseas robotic operators will fold towels millions of times. The first tasks will be repetitive, simple tasks, like towel folding, that require really good visual recognition. As those narrow tasks get mastered, they can be aggregated. Operators will see a towel, and hit a button that says “fold towel.” Then they’ll move on to the socks. And then they’ll make the bed.

The need for actual humans to drive the robots will decline as the robots learn these tasks.

Then the second wave of digital migration will begin.

The second wave of robot operators will be more skilled, and perhaps even slightly better paid. Operators will be able to assign a succession of tasks: towels, socks, bed. Robots will be manually operated only when there will some kind of failure when the robot doesn’t know what to do. This hand-off will define the second-wave of narrow AI/VR work. Fewer humans will manage more robots, taking over the task only when there is a failure. When a towel can’t be folded, a human will intervene. But as the data accumulates, those towels will become fewer and fewer.

Productivity will accelerate. Eventually, robots will be able to do what low-cost migrant service labor does now. What makes this path possible is disposable, transitional workers: through the VR and the internet, our robots will be as smart as ordinary people from the get-go, and only over time will AI take over from humans.

Our debates over migrant labor will cease to matter as Americans and Europeans opt for domestic robot bodies and foreign human minds. Robot bodies will “solve” the migration problem — at least from the perspective of Americans and Europeans. For the migrants abroad, and the service workers here, the solution might be less appealing.

In a broader sense, this hand-off will define 21st century work as the assembly line defined work in the 20th century. Making that hand-off seamless, error-free, and cheap will differentiate the successful firms from the failures.

What makes this hand-off from AI robots to human operators so interesting is that it complements what has already happened in the other direction, from human managers to AI managers. Right now millions of Americans, about 20% of the workforce, work in shifts, some in manufacturing but mostly in service work making burritos and lattes. Their schedules are determined by managerial algorithms that minimize the numbers of workers while, at the same time, guaranteeing enough workers to serve the customers. So far these algorithms have been used in our just-in-time service workforce to prevent W-2 employees from going over the 30 hour threshold between part-time and full-time.

This one-directional management is not the only option. These algorithmic bosses could just as easily begin to work for us, arranging gig jobs according to our preferences. We might work a shift at the coffee shop, but then our AI manager would know to fill out the rest of our time, at the right price, between a few hours of Upwork, Fiverr, and Lyft. Depending on our skill mix, that mix of gig hours would change. What would not change is that this algorithmic boss would not, in truth, be a boss, but a manager. The AI would tell us when to work, but it would be helping the worker maximize portfolio of jobs. Just like Lyft lines up the next passenger for the driver, so would this AI line up the next gig — but based on the worker’s preferences. Managers would, for the first time, work for the employees.

Humans will hand off problems to AI that they cannot solve, and AI will hand-off problems to humans that it cannot solve. This layer cake of human to AI to human to AI will define labor relations in the 21st century, just as unions and bosses did in the 20th. In the coming century, this layer cake will be just as political.

The hand-off and the layer cake are inevitable — as is a tendency to monopoly. Whoever controls the training data will always control the narrow AI. The best algorithms without training data are useless. In the absence of data, the car-driving, the towel-folding, the fruit-picking, they will never, ever succeed. Who owns the robots will not matter. The company that owns this data will own the future. Other companies will supply generic, commodified robots, but whoever owns the data will own the brains in those robot bodies. This is what is at stake.

On the one hand, we will suddenly have commodity robots that can do nearly every narrow AI task. What will we do with them? Will they make us all richer, enabling better lives? Or will those who own the data become a new entrenched global ruling class? Sensible people are torn between optimism, “no human should do the work of a robot,” and pessimism, “what will all those people do?”

Perhaps a better historical analogy to AI is not the assembly line but the mechanical thresher. Most humans, until the advent of the thresher, worked in agriculture, stooped for the harvest. Starting in the 1830s, farmers found that they no longer needed hands to bring the crop, and it inaugurated decades of agricultural mechanization — and displacement. The majority of Americans worked in agriculture in the mid-19th century, and by the turn of the twentieth, only a third still did. Today less than 2% of our workforce in agriculture, and yet we still eat.. The mechanical thresher displaced more workers than Uber ever will replace taxi drivers. Moreover, that mechanization freed nearly all of us from back-breaking work and made modern industrialization possible. It fed us, but it also liberated us to become more human, less like a machine.

Those workers found their way into the factories, where their backs might not be broken, but, in the monotony of the assembly line, their souls were. It was better than the fields, but it was not yet fully human. To be freed of monotony, and be paid enough to live, that is the promise of this new age. But it is a promise that must be seized.

In the industrial age of the 19th century, semi-skilled workers faced immiseration in factories and mills, as owners reaped their reward, producing decades of violence. Workers in the 20th century, through unions and laws, gained both a voice and a share. In that postwar industrial democracy, everybody, including the wealthy, became better off. This postwar economy is the moment for which all of us are nostalgic. The voice and the share will be as necessary to digital democracy in the 21st century, if we want democracy to survive.

When I visited that towel folding robot lab two years ago, I had these thoughts but also believed that we had lots of time. I was wrong. Last summer, I visited a lab at MIT in which the technology of VR-control of robots, with very gentle hands, gentle enough to pick fruit, already existed. In just that one year, this possibility was already becoming a reality.

We have less time to grapple with these issues than we might think. Once the feedback loop of VR-AI-Robot comes, it will build on itself rapidly. And while some parts will look like the past, there will be key differences. This is not just a productivity tool that will go off patent in a few years. It will not be a national technology but a global one.

This time around, I hope that we find a way to avoid those decades of conflict and use the rising productivity to solve our shared, global challenges. Instead of decades, we have years. But we also know that it is coming. We know what the challenges will be. We have models from the past to show us how to harness technology for greater productivity and greater democracy. We know that these changes could liberate us all to live more fully human lives, as caring, curious and creative people. There are always better uses for people than to fold towels, just as there were better uses for people than to thresh wheat.

What is not inevitable, is what we chose to do with all those people who had formerly been consigned to tasks that a machine could do, whether they will be liberated or discarded. The technology may be inevitable, but the consequences of that technology are not.