Human in Control or Automate Everything?
This blog belongs to the SAP Design series about intelligent system design. You might also be interested in the first blog, Principles of Intelligent System Design.
It is a common misconception that artificial intelligence inevitably means 100% automation. Movie producers would have us believe that AI is going to control absolutely everything — a Skynet scenario à la Terminator. So, if you design and implement an AI system, should you fear this or just ignore it? Is there a roadmap for intelligent automation of a specific system?
Let’s take Paul, for example. Paul is a manufacturing engineer of aircraft turbines. Each production order may take up to several weeks to fulfill, and many changes are made on the way. There is a product engineer, who constantly tunes the design for quality and cost reasons, there are dozens of workers required to be routed correctly through the production hall, and, of course, there is a Big Demanding Customer setting deadlines, threatening penalties, and changing requirements.
During his day, Paul makes hundreds of micro-management decisions concerning changes to the production order: apply them or ignore them? If he applies them, should they be downstream, upstream, or in both directions? Everything seems to be under Paul’s control, but, at the end of the day, Paul is very tired. Is there a way to help him?
The Skynet shadow
Imagine what would happen with full automation. Paul’s company just implemented a new software system that can “learn” from historical and environmental data, and recommend a decision with a certain level of confidence.
It is early morning. Paul checks the system to get the status of his production orders.
Wow. This is probably the typical scenario that every customer we talked to is afraid of. The system delays the production of a complex aircraft engine and rolls back all operations, due to the changes received overnight from the production team.
The system is 99.8% confident in its decision and redirects the workers to other orders. The lights are off. This evening is Paul’s regular sync with the Big Demanding Customer… Good luck, Paul!
If a customer called you the next morning in this situation, what would you tell him? We delayed your production order because of… 99.8%?
We can do it better
Imagine the same morning in a parallel universe. Paul checks his system for changes.
What we see here is the same algorithm with a different UX. Paul is notified about the change. He sees the operations affected and understands the reasoning behind the change. He also has access to similar cases and decisions made by him or his peers in the past. We can proudly say that Paul has a complete overview of the situation and can make faster and better decisions.
The system can also learn from Paul explicitly or implicitly by recording new cases and decisions. As time goes by, one morning, Paul may find himself in front of the following screen.
Now, Paul is confident enough to delegate trivial decisions to the system. He still wants to approve them. But, as his trust in the system grows, he will probably only need to review them from time to time, in the future.
Levels of automation
What we have done here? We have introduced two different levels of automation, which were applied depending on the use case and the user’s trust in the system. They helped us introduce and grow the AI complexity sequentially, without excluding the user and losing his trust.
Of course, in other scenarios, there could be more levels. The amount and the final level to be reached is always defined by the company.
In the end, it comes down to the right balance between human control and the autonomy of an intelligent system. The correct mix fosters efficient collaboration and builds trust. Where full automation is not feasible, we should aim for greater efficiency. By combining automation with better use of existing information, transparency, and learning effects, we can help users to obtain the same result with fewer steps.
But, we’re just scratching the surface. Stay tuned for more interesting topics and add your thoughts in the comments section below!
Several illustrations in this article were created with Scenes™ by SAP AppHaus
Originally published at experience.sap.com on January 29, 2018.
