Human in Control or Automate Everything?

Vladimir Shapiro
Jan 29, 2018 · 4 min read

This blog belongs to the SAP Design series about intelligent system design. You might also be interested in the first blog, Principles of Intelligent System Design.

Copyright: Vladimir Shapiro, SAP SE

It is a common misconception that artificial intelligence inevitably means 100% automation. Movie producers would have us believe that AI is going to control absolutely everything — a Skynet scenario à la Terminator. So, if you design and implement an AI system, should you fear this or just ignore it? Is there a roadmap for intelligent automation of a specific system?

Meet Paul, The Manufacturing Engineer

Let’s take Paul, for example. Paul is a manufacturing engineer of aircraft turbines. Each production order may take up to several weeks to fulfill, and many changes are made on the way. There is a product engineer, who constantly tunes the design for quality and cost reasons, there are dozens of workers required to be routed correctly through the production hall, and, of course, there is a Big Demanding Customer setting deadlines, threatening penalties, and changing requirements.

During his day, Paul makes hundreds of micro-management decisions concerning changes to the production order: apply them or ignore them? If he applies them, should they be downstream, upstream, or in both directions? Everything seems to be under Paul’s control, but, at the end of the day, Paul is very tired. Is there a way to help him?

The Skynet shadow

It is early morning. Paul checks the system to get the status of his production orders.

No-go: A manufacturing engineer is notified of changes to a production order. The system is “smart” and takes over.

Wow. This is probably the typical scenario that every customer we talked to is afraid of. The system delays the production of a complex aircraft engine and rolls back all operations, due to the changes received overnight from the production team.

The system is 99.8% confident in its decision and redirects the workers to other orders. The lights are off. This evening is Paul’s regular sync with the Big Demanding Customer… Good luck, Paul!

If a customer called you the next morning in this situation, what would you tell him? We delayed your production order because of… 99.8%?

We can do it better

Better: The engineer is supported in his decision. The trust relationship starts building. Later the system starts to take over in trivial cases.

What we see here is the same algorithm with a different UX. Paul is notified about the change. He sees the operations affected and understands the reasoning behind the change. He also has access to similar cases and decisions made by him or his peers in the past. We can proudly say that Paul has a complete overview of the situation and can make faster and better decisions.

The system can also learn from Paul explicitly or implicitly by recording new cases and decisions. As time goes by, one morning, Paul may find himself in front of the following screen.

Now, Paul is confident enough to delegate trivial decisions to the system. He still wants to approve them. But, as his trust in the system grows, he will probably only need to review them from time to time, in the future.

Levels of automation

Of course, in other scenarios, there could be more levels. The amount and the final level to be reached is always defined by the company.

In the end, it comes down to the right balance between human control and the autonomy of an intelligent system. The correct mix fosters efficient collaboration and builds trust. Where full automation is not feasible, we should aim for greater efficiency. By combining automation with better use of existing information, transparency, and learning effects, we can help users to obtain the same result with fewer steps.

But, we’re just scratching the surface. Stay tuned for more interesting topics and add your thoughts in the comments section below!

Several illustrations in this article were created with Scenes™ by SAP AppHaus

Originally published at on January 29, 2018.

SAP Design

We make work delightful through a human-centered design

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store