AI Governance in the Foundation Model Era
AI Governance has been playing a very important role in the responsible adoption of traditional AI models. The advent of Generative AI has further accentuated its need as is evident from several incidents that have happened in the recent past. One such example is from an airline company which was found liable by a court to honor a discount which was incorrectly promised by its chat bot to a client. The chatbot in this case was hallucinating and promised a discount which did not exist.
Such problems can be handled by the proper adoption of AI Governance principles. So what exactly is AI Governance in the Generative AI era? At a high level, AI Governance is about three things:
- Lifecycle Governance
- Risk Management
- Regulatory Compliance
Lifecycle Governance
Many-a-times AI Governance is equated with AI Observability and that to, only for models or prompts deployed in production. However, AI Observability is a much bigger concept in the context of AI Governance — it needs to provide observability for the entire lifecycle of the AI model or Prompt. This implies that we need to have observability which helps provide answers to the following questions for each and every model or prompt built in the organization:
- What is the business problem for which the prompt / AI model was built?
- Who asked for it to be built?
- Who built it? What data was used for testing it?
- What kind of testing was done for the prompt? What was the quality of the prompt? Was it checked for its propensity to generate biased content?
- Who validated the prompt? What was the justification given while approving the prompt for production deployment?
- Can you prove that the prompt did not act in a biased manner two months ago at a specific point in time? This requires continuous observability of the prompt when it is running in production.
Lifecycle governance in watsonx.governance helps your enterprise answer all of the above questions. It provides technology which will help capture “facts” that provide an up-to-date, 360 degrees view of all AI models and prompts being used in the enterprise. This is true for all models and prompts built and deployed using any technology such as watsonx.ai, AWS, Azure, OpenAI, etc. Thus Lifecycle governance extends the traditional definition of AI Observability and applies it to every stage of the model/prompt lifecycle.
Risk Management
Consider a scenario where a data science team has been asked to build a prompt (or an LLM Application or an AI model) to solve a business problem. They realize that they have already built a similar prompt in the past which is running in production. This prompt has undergone proper due diligence by the Model risk management team and has been approved for deployment to production. Not just that, this has been performing really well in production. So the data science team decides to share the existing prompt with the new team. This seems the right thing to do, correct? Wrong!
The problem with the above scenario is that the new requirement could be from a team which plans to use the prompt in a different manner than the first team. E.g., the second team might be planning to send the output of the prompt to external users / clients whereas the first prompt was for internal usage only. The kinds of tests that are to be done on a prompt will depend on how the prompt is planned to be used. Hence the simple oversite mentioned above can expose a company to regulatory fines. Unfortunately such situations are far too common in today’s world and such reuse is very easy especially with prompts and LLM Applications.
In order to avoid such problems, there is a need for a technology for process governance which can help define, automate and govern the process that needs to be followed as a prompt (or AI model) moves from development to validation to production. In the above example, a technology like watsonx.governance (which supports process governance) would have helped identify the set of tests that need to be done before the prompt is deployed to production and automate the end to end flow so that teams do not inadvertently end up deploying prompts to production bypassing one or more of the identified tests. This helps with Risk Management and ensures that organizations are not exposed to undue risks as the right processes are properly enforced across the organization.
Regulatory Compliance
There are several AI regulations that are coming up all over the world. One of the most stringent ones is the EU AI Ethics Act which stipulates a minimum fine of 35 Million Euros or 6% of the annual revenue of the company — whichever is higher; for non-compliance of the regulation. In order to comply with the regulations, enterprises need to do a bunch of things including evaluating the prompts at different stages of the lifecycle.
Hence the new worry that is plaguing every CDO is: Should their data scientists focus their energies on understanding the regulations or should they focus on building the best of breed AI models and prompts? CDOs obviously want the latter, but they do not want to take the risk of ending up being non-compliant with the regulation. Hence there is need for a technology which can help them translate the regulation into set of actional steps that their teams need to take for ensuring compliance with the regulation. This will let their data science teams focus on doing what they do best and at the same time ensure that they are complying with the AI Regulations. This is the third and very critical aspect of AI Governance — Regulation Compliance.
Conclusion
In this blog, we have provided an overview of the three aspects of AI Governance: Lifecycle Governance, Risk Management and Regulatory Compliance. watsonx.governance provides top notch capabilities across all these aspects. Consequently, it has been named as the leader in the IDC MarketScape: Worldwide AI Governance Platforms for 2023. Check out the IDC report here: https://ibm.co/4b7RsQ2
Want to try out watsonx.governance for free? Just head to: https://www.ibm.com/products/watsonx-governance
