The Automation Fallacy

In today’s world, a superficial reading of tech newsletters and startup press releases leads one to believe we are on the cusp of a machine-driven society. Indeed, “Artificial Intelligence” and “Machine Learning” have become table-stakes marketing terms, with companies like Tesla (autopilot) and Google (duplex) leading the charge toward fully automated solutions to everyday problems. Cloud providers have made AI and advanced machine learning technology are more accessible than ever, almost eliminating the barrier to entry for smaller companies that has existed for most of the past two decades.

But: we need a reality check. Fully automating something is hard. Doing it accurately is next to impossible. And, most importantly, often times it is the wrong approach.

We need to redefine our expectations around automation and instead focus on improving the efficiency of a task; by doing so, we reduce the burden on automation technology while simultaneously opening up other means of improvement, like process and tooling.

How to Automate

In order to automate something, we need to tell the computer how a specific task should be performed. This is a universal truth that is easily forgotten when technology is complex. But of course, it is not as simple as saying “hey computer, do this!” At the bare minimum, we need to be able to tell a computer what the expected input and desired output is for a particular task.

How we accomplish this depends on two questions:

1. How predictable is the input?
2. Can we define the task as a binary tree structure? (if this, then that)

The Deterministic Approach

If the answers to the above are “100% predictable and validate-able”, and “yes”, then a task can be easily automated with code. This is the basis of software development. We as the engineer define the precise steps a computer must take to turn the expected input in the desired output (with, of course, validation, authentication, and other business logic along the way). Workflow automation tasks like these are common, and require no training data to implement.

As the task becomes more complex, however, we have another option:

The Non-Deterministic Approach

There are two reasons to bring in non-deterministic (statistics or heuristics) decision-making to an automation problem.

1. The input is not 100% predictable
2. The steps to reach the desired output cannot be represented in a binary tree structure.

In both of these cases, we need to rely on algorithms that “do their best” to turn the input into the desired output.

In heuristics-based automation, we subjectively take the “common case(s)”, and as many edge cases as can be identified and accounted for, and effectively turn them into a deterministic decision: we write code to identify the case deterministically (for example, by looking for key words on a page) and return the “most likely to be accurate” output according to that case. Of course, with this approach, we must acknowledge and be okay with inaccurate output being a common part of the workflow.

In statistics-based automation (e.g. machine learning), we gather a large set of training data, and using that data to “teach” an algorithm that given a certain input, it should return a certain output. With enough training data, these algorithms can get exceptionally close to 100% accuracy: we help them extrapolate and interpret enough relevant data points from the input such that the output can be almost guaranteed to be what you want.

Yet, regardless of the approach, 100% accuracy can never be guaranteed. This is important. Let’s explore:

Accuracy is a function of predictability

A common naive requirement for automation is as follows:

Given <X>, automatically do <Y>, with 100% accuracy.

Reality: unless <X> is a machine-readable and machine-validate-able data structure, the above is mathematically impossible.

An automation algorithm’s ability to be accurate is directly dependent on how predictable its inputs are. If the input is 100% predictable, 100% accuracy is relatively trivial. That’s how APIs work. But if it is at all unpredictable, 100% accuracy becomes impossible to achieve, and generally the less predictable the inputs are, the harder it is to generate highly accurate results. Or, in other words, if an input is not fully predictable, then it is open to interpretation, and we will be relying on a machine to make that interpretation. Teaching a machine to accurately interpret ambiguity is incredibly difficult.

If an input is not fully predictable, then it is open to interpretation, and we will be relying on a machine to make that interpretation. Teaching a machine to accurately interpret ambiguity is incredibly difficult.

So we have three ways to improve the (perceived) performance of an algorithm. (1) Increase the predictability of its inputs, (2) make the algorithm better through training and tweaking, or (3) increase our tolerance for inaccurate results. (This decision must be solely based on your product requirements.)

Let’s take financial analysis for example. Our technology at Fincura identifies tabular financial data in unpredictable PDFs and standardizes that data to a lender-defined template, allowing all borrower data to be analyzed in the same way regardless of its original format. The users of our technology are credit analysts — they are paid to scrutinize every last detail of a financial statement because one wrong number could mean a bad loan. The stakes are high. In our case, we can’t improve the predictability of the inputs (it’s whatever the borrower is already sending the lender). We could certainly improve our technology over time. But can we increase our tolerance for inaccuracies?

Let’s say in this case, instead of targeting 100% accuracy, we targeted 99% accuracy. Still pretty high, right? But what that means is 1 out of every 100 executions will be flat-out wrong. This is typically unacceptable in use cases that demand a highly accurate output, including ours (in our case, if an analyst finds one wrong number, they’ll assume, rightfully or not, that there could be other wrong numbers, and any efficiency gains from the automation are effectively erased as they now feel the need to comb through each line).

“Black box” automation solutions inevitably resemble this cologne from “Anchorman”

What makes this more complex is the algorithm will not know which one of a set of 100 executions is the wrong one! All it can guarantee is that on average, 99 out of 100 will be correct. (And this assumes your training data is representative of your production data. Further reading).

We thus find ourselves at the base of a mountain: in order to meet a product requirement, we need an immense amount of training data, and even if we had that training data, we still wouldn’t be able to guarantee we’d get close enough to 100% accuracy. But: what if the task were not to fully automate the process, but instead to make the process as efficient as possible within the product constraints? Then we could achieve our goal of near-100% accuracy.

Expanding our Mindset

By rule, machine learning algorithms have diminishing returns. Thus, even with continuous advancements on our automation algorithm, we can continue to inch ever closer to 100% accuracy, never quite achieving it, and slow down progress significantly as we go. This is universally true. And it means that sometimes throwing more machine learning technology at a problem is not necessarily the best solution.

It is important to understand that automation is but one of four levers we have at our disposal to improve the efficiency of a task. Given the diminishing returns of focusing squarely on automation technology, it is oftentimes much more valuable to redefine the problem as “make this task more efficient” and bring in other components to solve it.

People, process, and tooling can tighten the curve of the diminishing returns graph.

Embrace the People

Especially at technology-first companies, it is easy to forget that people are the natural solution to fill these gaps. Indeed, it has become almost taboo to admit the imperfection of a technology and propose addressing it using the very thing that technology is supposed to unseat. But the reality is: using technology to reduce the starting cost of a task, and designing process and interfaces with the intention of relying on people, is in many cases much faster and easier than solving the problem purely with technology.

Of course, bringing people into the fold comes with its own set of new problems: if we don’t want to slow down, we now need to intentionally design for people to be involved. Specifically, that means:

1. Tooling: we need the proper interface between the person and the technology.
2. Process: we need a workflow that maximizes the efficiency of the analog steps, and augments the person with the technology via the tooling wherever possible.
3. Errors: people make mistakes, but they make different mistakes than the ones that machines make. We need to make it easy to catch and correct those mistakes.

In many ways this should be obvious. When Henry Ford starting building Model-T’s he didn’t try to invent machinery to do it all automatically — that would have been preposterous! He instead found the most efficient combination of machinery, people, and process (the assembly line) to achieve his goal: quickly, cheaply and reliably produce Model-T’s.

Similarly, most software tools are not built to fully perform a task end-to-end. They assist with a task, making the user a more efficient producer. Nobody on the Microsoft Word development team has sought to automatically write essays and resumes, but they’ve certainly created tools to help get the job done more quickly.

A tool.

So how do we get caught in this trap? We see what industry leaders like Google and Tesla are doing, and we have easy access to powerful machine learning technology in the cloud, and we want to automate things. “That should be automated” has been a common phrase for decades, but we are now seeing more and more complex tasks having the potential to be automated with existing technology. But as we can see, just because it’s now possible doesn’t mean it should be pursued.

So, I suggest that if “automation” is the goal, it needs to be rethought. “Automation” is one possible way to achieve a goal, and oftentimes is better when paired with other solutions as well.