The 6 Principles of Trustworthy AI

A brief overview of the AI regulation landscape

How can we establish trust for autonomous systems? — Photo by Dezeen

AI is seeing an unprecedented adoption globally. However, many incidents are around the corner which may bring entire fields to shut down.

First the good news: AI is influencing healthcare, automotive, finance, policing and consumer goods like never before [1]. In healthcare, drugs are developed at ten times the speed [2], countries such as Singapore opened 40% of streets for testing autonomous systems [3] and financial sector profits from e.g. enhanced risk assessment models [4].

Now, the bad news: When unregulated autonomous systems fail it can bring an entire field to halt. Taking the fatal uber accident as an example in 2018 [5] which caused an immediate governmental reaction that shut down the entire program. Hence, quality assurance and governance is the highest priority for safety-critical applications to continue growth and adoption.

AI in Healthcare — Photo by New York Times

When autonomous systems are governed and follow a human-centric approach, trust is established and adoption stimulated. This can be achieved following the principles of trustworthy AI.

The 6 Principles of Trust

For Trustworthy AI six guiding principles can be defined, which together enable trust, ethical behavior, the ability to govern and even to insure, which has the highest potential for mass adoption.

The Six Principles of Trustworthy AI

Taken lung cancer as an example, AI systems have shown better accuracy in predicting lung cancer than professional doctors [6]. However, this success is only useful when patients trust the diagnosis. Eventually, doctors will use autonomous systems as guidance to formulate their diagnosis. So, ultimately, doctors need to trust such applications, in the first place.

To establish trust for autonomous systems the six principles are presented following the lung cancer diagnosis example:

1. Starting with transparency doctors need to understand the main factors for the prediction. This enables them to verify the decision and provide the patient with a more conclusive picture of the result.
2. Responsibility is another important factor, which comes into play when the hospital is sued for a wrong diagnosis. Then, a consensus needs to be in place which determines, who is responsible for the diagnosis. Responsible entities may be the doctor, the software developer or the hospital. New emerging models illustrate responsibility insurances which are mandatory and become more expensive with higher error rates. Hence, markets may regulate themself if enough competition exists.
3. Robustness is one of the most important principles, ensuring that best practices are used from development, deployment, maintenance and system evolution. Next to general accuracy, robustness towards difficult situations needs to be assured. These can be rare lung cancer types or for autonomous driving situations that do not exist in high quantity during training such as car crashes.
4. Another important principle is privacy, which ensures that any patient diagnosis data can’t be retrieved by third-parties and anonymizes all data used for creating the system in the first place. It would be severe to identify that the cancer diagnosis of a current president may be positive. Since many data pools are merged for training highly accurate systems, this assumption may be considered realistic.
5. Security is the penultimate principle of trustworthy AI, which ensures overall system integrity and defense mechanisms against adversarial attacks. Attacks can be from physical and digital nature. A common physical attack example in autonomous driving is applying a sticker on a stop sign, which may lead the car to misclassify the sign as 100 mph tempo sign. Similarly, lung cancer system sensors can be manipulated. Hence, overall system validation is necessary to make a confident decision.
6. The final principle and most covered in media is fairness, which urges for equal treatment of patients regardless of their skin color, gender or political/religious beliefs. Many efforts have been taken place to ensure fair systems. However, achieving perfectly fair AI remains a challenging task [7], as the original data which initially stems from humans, may be inheriting human bias in the first place.

Global Efforts to Govern AI

Several global committees and national initiatives have been formed to further define what guidelines need to be in place for AI applications to strive for mass adoption. This is especially true for safety-critical sectors such as autonomous driving or most recently debated facial recognition.

In future, articles on current global and national developments will occur soon. For now, the following list provides a first overview of individual approaches of nations and committees:

• Global standards led by ISO: here
• Policies around AI imposed by European Union: here
• The Chinese approach to artificial intelligence analysed from political, ethical and regulatory aspects: here
• The American AI Initiate: here
• Singapore’s AI governance framework which was recently presented at the World Economic Forum: here

Conclusion

Not all AI applications require a complete trustworthy approach, as their use is limited to recommending the next movie on Netflix or providing a shopping recommendation on Alexa. Nevertheless, Alexa should be secured against transferring money via credit card and recommendation systems should treat all subjects fair without taking sensitive attributes into consideration.

Therefore, when approaching autonomous systems it is crucial to identify how safety-critical an application is and then take the necessary precautions along mentioned principles.

For questions on AI feel free to reach out anytime: david.berend@mine-it.eu or visit safe-intelligence for more.

Related Article:

Worried about biased AI? Worry about human bias first.

References

[1] McKinsey & Company — Global AI Survey: AI proves its worth, but few scale impact
[2] Nature — How artificial intelligence is changing drug discovery
[3] Land Transport Authority — Autonomous Vehicle Testbed to be Expanded to Western Singapore
[4] Oliver Wyman — ARTIFICIAL INTELLIGENCE APPLICATIONS IN FINANCIAL SERVICES
[5] New York Times — Self-Driving Uber Car Kills Pedestrian in Arizona, Where Robots Roam
[6] Google — A promising step forward for predicting lung cancer
[7] Safe Intelligence — Worried about biased AI? Worry about human bias first.