Letting Go of the Wheel

10 Steps to Trustworthy Automation

We all look forward to the day we can step into our cars and get to our destinations without touching the steering wheel. While there is no question that automation is the way of the future, there are plenty of obstacles to overcome before getting fully automated vehicles onto the road. A major obstacle is trust: how do you convince users to trust their lives with automation you’ve built?

Although not quite as perilous as highway driving, Quantcast has high stakes in the billion dollar programmatic advertising industry. Our clients trust us to make decisions on their behalf, with the expectation that their brand will be protected and their budget will be put to effective use. In building automated systems for our clients, we have found that we encounter a set of problems common to our fellow pioneers in this space.

The Important of Trust

Automation is a control system that tries to reach an optimal or ideal state by making automatic, data-driven decisions. However, conflicting objectives make it difficult to define, let alone reach, an optimal state. Every day, drivers on the road are making a series of complex decisions that involve conflicting objectives without even realizing it. As you step on the gas pedal, the amount of pressure you apply depends on how fast and safely you want to get to your destination. When developing an automated vehicle, optimizing purely for speed would be disastrous for safety and optimizing purely for safety would result in a really annoying travel experience.

Humans are (often subconsciously) aware of the dangers of these conflicting objectives and consequently are skeptical of automated systems. For example, if you’ve been driving since you were a teenager, it may be hard to trust a machine to understand every subtle technique you’ve picked up over time. Building a system that balances trust and transparency alongside accuracy is difficult; however it is absolutely critical to user adoption, and widespread adoption is the key to making an automated system realize all its intended benefits and maximize its impact.

The 10 Steps to Trustworthy Automation

1. Prepare to balance conflicting objectives
   Frame automation as building a system to solve an optimization problem. Assign each objective a cost function based on what is believed to be its value to the business. Build a system to minimize overall cost. Note, some things are trickier to quantify than others so you should…
2. Create heuristics from talking to customers
   Customers don’t always quantify their needs but are willing to have a conversation about what is important to them. It’s your job to understand and calibrate how each of their objectives weighs against each other. At Quantcast, when a customer tells us that delivering the entire budget of their advertising campaign is their primary objective, we need to dig deeper to understand what that means. If we can’t deliver the budget exactly, is over-delivery or under-delivery more favorable? How much are they willing to spend, and are they okay with seeing drastic day-to-day changes in delivery levels to meet this goal?
3. Model costs from available past data
   You will need to leverage historical data to understand how a given action will impact objectives, especially cost. At Quantcast, many of our campaigns have multiple objectives, and we rely on inferences from historical data to better understand how pulling a given lever will affect various objectives and predict the costs associated with these actions.
4. Create and track the right metrics
   Track metrics for business impact early to understand the intended and actual benefits of the system and find out what went wrong if it deviates. Dashboards of these metrics can provide evidence of why customers should adopt.
5. Align incentives
   Besides a general distrust of the system, a reason that customers may be reluctant to adopt might be that they have different goals. For example, the system may benefit the company as a whole but not the individual customer. Take a step back to understand why that could be- the problem could very likely lie somewhere outside of your system.
6. Provide transparency into the system
   The greatest fear of many of our customers was that the system would get some unexpected input and jump off the rails. Implement alerts for these cases that provide clear instructions on what to fix and an explanation of the behavior that the system will fall back on.
7. Phase roll out
   Release the beta to a small group of trusted, high-performing individuals who take on the responsibility of using the product and providing regular feedback as part of their role. Beta rollouts will help address the biggest issues first and make wider rollout much smoother.
8. Communicate through a clear channel
   Provide a public forum (e-mail aliases, tickets, meetings) where people can go to with questions and feedback. This can help to build a higher level of understanding and nip any misinformation in the bud.
9. Require feedback to opt out
   There are many reasons a customer may want to opt out of a system, but not all of them are valid. Set up a process for opting out that holds them accountable; only legitimate cases should be allowed. This will also help to track and understand current issues with the system.
10. Know that you’re not done
    Allocate resources to development even after roll out so that unexpected issues can immediately be addressed. Customers need to see that problems will be fixed in a timely fashion. Losing steam on a new system could quickly lead to its downfall.

Hidden Lessons in Building Automation

Our journey to full automation has taught us how to build systems that automate many existing processes, but more importantly, it has helped us to take a step back and understand the ultimate problem we are trying to solve. Misaligned incentives, procedural bottlenecks, and misinformation are all issues that have bubbled up and been addressed during the development and rollout of our systems. On the surface, automation simply replaces a process, but it forces us to look at the bigger picture and solve a higher-level problem.

If you’re interested in helping us on our journey, we’re always looking for people with an analytical mind, an ability to anticipate customer needs, and solid engineering skills. Check out open opportunities here.