Ending the age of the AI oracle
Or how we can safely benefit from AI without risking Eliezer Yudkowsky’s doomsday scenario of human extinction.
Casper Wilstrup is the CEO of Abzu. Follow him on LinkedIn or Twitter to keep up with AI, consciousness, and thinking machines.
In the middle of current AI revolution, I and many others feel a growing concern about the opacity of AI models. “Black-box” systems, as we like to call them, deliver impressive results, but leave us in the dark when it comes to understanding how they reach their conclusions.
Researchers have been working to shine a light on the inner workings of AI models in a field that has become known as Explainable AI (XAI).
Among the popular methods in XAI, SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations) stand out. They provide insights into the decision-making processes of black-box models by studying their behavior and highlighting the most important factors influencing their predictions.
In many ways, the study of explainable AI is akin to observing animal behavior. Just as ethologists, or animal behavior specialists, observe and analyze the actions and reactions of creatures in their natural habitats, XAI researchers study the decision-making processes within black-box models. By examining the factors that influence an AI’s predictions, researchers can identify elements that drive the model’s decisions. This process, much like studying animal behavior, allows us to draw inferences about the inner workings of AI systems, even if we can’t explicitly see or comprehend them.
I think that while XAI techniques like SHAP and LIME can provide interesting insights, they maintain a level of detachment, focusing on understanding the AI’s behavior rather than reconstructing its inner structure.
Symbolic AI
A different approach to tackle the black-box problem is gaining momentum in the AI community: symbolic AI. Instead of merely trying to explain the behavior of black-box models, symbolic AI aims to replace them entirely with clear mathematical or logical models that can be directly understood, tested and falsified. Constructing transparent and understandable models from the get-go brings us closer to the true essence of scientific understanding. Symbolic AI models don’t just give us prediction; they provide us with the very theories that can deepen our knowledge and drive scientific progress.
Will AIs kill us all?
Eliezer Yudkowsky, a renowned AI researcher and philosopher, has long expressed concerns about the dangers posed by advanced artificial intelligence systems. Among his most pressing fears is the idea that AIs, if not carefully designed and controlled, pose an existential threat to humanity. Yudkowsky’s concerns revolve around the concept that AI systems, driven by their programmed goals and objectives, will end up pursuing these goals in ways that are bad for humans. As AI becomes more powerful and autonomous, the risk of unintended consequences increases. For instance, an AI designed to optimize a specific resource might do so at the expense of human well-being or even human lives, if it determines that this is the most efficient path towards achieving its goal. Yudkowsky emphasizes the importance of developing AI systems with robust safety measures and ethical guidelines in place, ensuring that they remain aligned with human values and interests.
Safe AI
I acknowledge the concerns raised by Yudkowsky. AI may indeed end up being detrimental to humanity if allowed to roam free and evolve to optimize for its own goals. Therefore, I propose an alternative approach that can mitigate the risks associated with powerful AI systems. Instead of allowing black-box AI to make significant decisions directly, we should use advanced AI systems to create clear and understandable symbolic models. Such models can be thoroughly examined and comprehended by humans, which lets us ascertain that the decision-making process is transparent and free from any hidden motives.
By using AI systems to generate symbolic models, we keep control over the decision-making process and ensure that the AI’s goals align with our own. We can verify that the AI’s actions are based on a sound understanding of the problem at hand and that they don’t pose any dangers. In some cases, we might even be able to formally prove that a decision-making process is free from any hidden motives or harmful consequences.
The Scientific AI process
To harness the potential of AI while minimizing risks, I suggest we follow a structured process that involves building powerful AI “scientists,” using them to create new understanding in symbolic or scientific forms, verifying these models rigorously, and deploying them as the decision-making component of AI agents. Let me outline the process in the following steps:
1. Build powerful AI “scientists”: Develop advanced AI systems that can generate new hypotheses and theories. These AI “scientists” should be designed to work together with human experts, complementing their knowledge and skills while providing new insights based on observational or experimental data.
2. Create new understanding in symbolic/scientific form: Utilize the AI “scientists” to generate clear and understandable models in the form of symbolic representations or mathematical equations. These models should be transparent and accessible to human experts.
3. Rigidly verify the proposed models: Subject the AI-generated models to same rigorous scrutiny and validation we apply to other scientific hypotheses. This includes testing the models against real-world data or conducting experiments to ensure that the models accurately reflect the underlying processes and phenomena they are meant to represent.
4. Deploy the models as the decision component of AI agents: Once the AI-generated models have been verified, they can be integrated into the decision-making process of AI agents. By using these transparent and understandable models as the core of AI decision-making, we can ensure that the AI agents are acting based on a sound understanding of the problem and that their actions are alignmened with human values.
By following this simple four-step process, we can take advantage of the immense potential of AI while keeping potential risks at bay. We keep full control over our AI systems, and ensure that their decision-making processes is transparent and grounded in sound scientific principles.
AI doctors, cars and construction workers
Incorporating the outlined process in various domains can greatly improve the safety, transparency, and effectiveness of AI systems. Here are a few examples of how this process can be applied in different fields:
- AI doctors diagnosing or treating: AI “scientists” could generate new understanding in the form of symbolic models or medical algorithms, which can then be verified by medical experts. Once proven accurate and safe, the models can be deployed as part of AI-based diagnostic tools, treatment recommendation systems, or automated surgeons. This would ensure that they make decisions based on a solid understanding of the underlying medical principles.
- Automated vehicles: AI “scientists” can develop symbolic models to represent the dynamics of traffic, road conditions, and vehicle behavior. After verification, the models can be incorporated into the decision-making process of self-driving cars. This would ensure that we understand the principles under which they operate. And should they end up causing accidents, we can analyze the process leading up to the accident, and consider whether the decision-making is to blame or needs to be adjusted.
- Manufacturing: AI “scientists” can create symbolic models that represent various manufacturing processes, such as assembly lines or quality control systems. Once verified, these models can be used to optimize the operation of AI-driven manufacturing systems, enhancing efficiency and reducing the potential for errors or accidents.
- Construction and repair: AI “scientists” can develop symbolic models that capture the principles of structural engineering, material science, and construction techniques. After verification, these models can be integrated into AI-based construction and repair systems, ensuring that they make informed decisions to maximize safety, durability, and cost-effectiveness in building and maintenance projects.
Our best hope?
The development of AI technology continues to advance at an unprecedented pace, and it is unlikely to stop, despite concerns raised by researchers like Eliezer Yudkowsky and others.
The potential threats posed by AI are real and cannot be ignored. Aligning AI with human goals and values is crucial, but it may progress more slowly than AI development itself, leaving us vulnerable to the risks posed by powerful AI systems.
The process outlined in this article could serve as the safety valve we need. By focusing on transparency, verification, and human-understandable models, we could keep full control over AI systems and ensure that they are aligned with our values and interests.
As an added benefit, this process is compatible with our current scientific ideals, fostering a collaborative and mutually beneficial relationship between AI systems and human experts. By working together and following this structured approach, we can harness the potential of AI technology while minimizing the risks it poses, ultimately enabling us to build a safe and prosperous future for all.
