Identify Unknowns, Weaknesses, and Risks in AI

There’s a lot that can go wrong with AI. Let’s talk about how to avoid it.

Previously, I covered what AI prototyping projects are, how to launch AI prototyping projects, and running AI prototyping projects. In this article we’ll discuss managing risks in AI.

While communicating the vision of an AI project through an interactive demo is generally understood to be the main goal of an AI prototyping project, one of the key pieces of value is the ability of these projects to uncover and resolve risks.

With AI projects your organization may not know the things they don’t know. You don’t know what aspects of your data or prompts will prove insufficient to get the results you’re looking for. You don’t know how people will interact with your application or the specific areas in which your application may prove inaccurate. If you’re working with new models or new technologies, the performance, reliability, formats, and specific behaviors of these systems may not be known to you in advance.

For example, I was part of an AI prototyping project team at Leading EDJE where we fed pictures of users to a model hosted on Azure for Azure OpenAI GPT-4 to remark on the user’s attire. While this project was successful, one of the things we discovered in testing was that our model frequently remarked on the user’s face being blurry or mysterious.