Control, trust and accountability in a world of machines…

Part 1 — Who is control?

By Bertrand Maltaverne

Over on Buyers Meeting Point, Kelly Barner, launched, together with Rosslyn Analytics, an exciting initiative, whose purpose is to create “an active discussion around procurement’s relationship with automation: both what it is and what it ought to be.” She will share content and commentary on Buyers Meeting Point and Twitter, using the hashtag #AutoProcure.

As a coincidence, on the day she launched #AutoProcure, I published a post here on Medium that fits with the theme of #Autoprocure. My post looked at some* of the challenges and opportunities that new technologies like Artificial Intelligence (AI) and Machine Learning (ML) represent for the future of Procurement,Cognitive Procurement.

Because of this coincidence and this common interest, my next posts will be focused on my contribution to #Autoprocure and identified as such.

*: I also had in mind to write more posts to explore various aspects of Cognitive Procurement; these posts, together with some older ones, are / will be available under the dedicated section of my Medium publication.

In her post, Kelly raises good questions. These questions go beyond the field of Procurement as they reveal broader fears, challenges, and opportunities that our society faces with regards to new technologies linked to “automation.”

I will address some of these points in a series of posts. I will look at those with a broader perspective than just Procurement and in domains with more direct and immediate impacts than Procurement decisions.

Questions raised by Kelly, and more generally, by the role of advanced technologies in Procurement:

  • Q: As part of raising to a strategic position, and to deliver more value to organizations, Procurement wants has to gain control over more spend. It is the spend under management challenge. At the same time, Procurement must accept to lose (some) control on (some of) its activities as they will be (fully) performed by technology. It is the automation challenge. Both challenges are interconnected. One lever to manage more spend is to reduce the time spent on less important tasks by automating them. Efficiency fuels effectiveness!
  • Q: “Users” of Procurement put their needs in the hands of Procurement. It creates a relation based on trust. Addressing those needs will be more and more fulfilled by technology, not by Procurement people. So, if tasks are fully performed by a machine, who is accountable for the outcome? The “user,” the Procurement organization, the provider of the solution?

These are important questions about control, trust, and accountability! I will cover these aspects in four posts:

  1. Who is control?
  2. Who is accountable?
  3. Who to trust: the programmer or the teacher?
  4. What do we want?

Who is in control?

Whenever computers replace humans, it raises concerns. I remember the launch of the Airbus A320 in 1987. The A320 was one of the first commercial fly-by-wire planes. It introduced more technology, thus reducing the number of crew in the cockpit from 3 to 2. More importantly, it also gave the computer much more control. More than the pilot.

“Novices erroneously think of the autopilot, which can be switched on and off,” explains Gerhard Hüttig of the Flight Simulation Center in Berlin. The autopilot is also a factor, but it is only one of many, and it guides the airplane along a course pre-programmed by the pilot. But Airbus’s flight computers do a lot more. They are automatically activated if the aircraft enters a dangerous angle, loses too much speed or threatens to complete a violent rolling motion. The Airbus engineers christened the software, which is designed to keep the aircraft within a green zone at all times, “Flight Envelope Protection.” “The computers intervene,” explains Hüttig, “no matter how hard the pilot pulls on the controls.” The Computer vs. the Captain, Will Increasing Automation Make Jets Less Safe? — Part 1

Protests from pilots worried about their future. Worries for safety… These were the topics in the news when the A320 was launched.

Since then, Boeing also introduced similar technologies.

This is because advances in many other technologies made planes safer, and more complex to pilot. These presents pilots with challenges that“have reached the limits of what the human sensory perception system can handle” and that they can only manage with the help of a computer.

Interestingly enough, how Boeing and Airbus approached the same question in different ways.

There is a difference between the American and European concepts. In the case of Airbus, the pilot is essentially prevented from disabling the flight computers. Unlike the autopilot, the flight computer can only disable itself, and only if its systems become so confused that it would otherwise malfunction.The Computer vs. the Captain, Will Increasing Automation Make Jets Less Safe? — Part 3

Both approaches have their pros and cons. There are numerous stories illustrating situations where on-board computers have prevented disasters or contributed to creating dangerous situations. What is sure is that safety has never been at a higher level and that Airbus and Boeing planes with such computers are on par with regards to safety record. Even pilots do not have a clear preference between the two:

“Boeing or Airbus? Among pilots, this has become almost a question of faith. “That’s just as hard to decide as the question of whether Mercedes or BMW is better,” says pilot representative Braun.” The Computer vs. the Captain, Will Increasing Automation Make Jets Less Safe? — Part 3

Although Procurement decisions have fewer life-and-death implications (at least in terms of immediacy; there are, sadly, examples of bad Procurement decisions that had dramatic consequences…), the questions of who is in control and who has authority are relevant as illustrated in this post by the Sourcing Doctor.

Pilots had to accept a loss of control and had to learn to trust the machines and to cooperate with them. Their role became more focused on strategic activities: managing difficult and exceptional situations. Be it the most dangerous parts of a flight, a flock of birds leading to a water landing (in an A320!) or… a computer failure.

Just like pilots, Procurement faces new challenges:

And, some thirty years after pilots did, Procurement people also have to embrace technology:

But, there are still many other important questions to be answered… This is what the next parts of this series will cover!

If you enjoyed this, please scroll down and click the “recommend” or “share button”.

If you have your own “perspectives”, just use the “response” feature.