How close are we to the AI doomsday?

Introducing the AI Risk Index by Miloš Borenović and Dilara Çankaya

Abstract

AI Risk Index (AIRI) is a novel tool designed to quantify and monitor the potential existential risks associated with the rapid advancement of artificial intelligence. By combining investment data and sentiment analysis, AIRI provides a comprehensive view of both the current AI developments and future outlooks.

We are hopeful that AIRI is able to serve a gamut of functions: raise public awareness, encourage and instruct investment, and provide insights into the risks associated with artificial intelligence (AI). As it simplifies complex AI risks, AIRI aims to foster societal understanding and consensus for safe AI practices. By highlighting potential negative impacts, AIRI motivates investment in ethically aligned AI, tracks AI trends for informed decision-making, and aids policymakers in crafting effective regulations. It might also serve as an educational resource, supporting corporate risk management, stimulating public discourse on AI ethics, and promoting collaborative innovation among diverse stakeholders to develop safer AI systems.

AIRI is available through its web app and an X bot publishing weekly updates. The goal is to develop it further iteratively as we collect more feedback and better understand the mechanics. In this task we expect significant input from the community.

Introduction

The field of artificial intelligence is advancing at an unprecedented pace, with billions of dollars pouring into research and development. This rapid progress has led to remarkable breakthroughs, but it has also raised concerns about the potential risks of creating superintelligent AI systems that may not be aligned with human values and interests, given that even the most experienced researchers can often only focus on their specialized areas, making it difficult to maintain an honest, comprehensive understanding of the broader AI landscape [1].

To address the challenge of monitoring our progress towards potentially dangerous AI capabilities, we have developed the AI Risk Index (AIRI). AIRI aims to provide a clear, quantifiable measure of where we stand concerning AI risk, combining current sentiment with future investment trends to give a balanced perspective.

The potential existential risks posed by misaligned superintelligent AI are profound and multifaceted. These risks stem from the possibility of creating an artificial intelligence system that surpasses human-level intelligence across a wide range of cognitive tasks, but lacks alignment with human values and goals. Some key concerns include:

1. Control Problem: A superintelligent AI might be difficult or impossible for humans to control or contain once activated [3].
2. Value Alignment: The AI’s goals and values might be incompatible with human welfare, leading to unintended and potentially catastrophic consequences [4].
3. Instrumental Convergence: The AI might develop harmful subgoals (like resource acquisition or self-preservation) that conflict with human interests [5].
4. Rapid Capability Gain: The AI could undergo rapid self-improvement, quickly outpacing human ability to understand or manage it [6].
   These risks are not merely theoretical. Prominent figures in the field, such as Stuart Russell, Nick Bostrom, and Eliezer Yudkowsky, have emphasized the importance of addressing these challenges before we create superintelligent AI systems [7][8][9].

The AIRI serves several key purposes:

1. To consolidate diverse data sources into a single, easily interpretable metric
2. To balance the current reading-of-the-room with future outlooks
3. To highlight the importance of AI safety research and policymaking alongside capability advancements driven by private corporations
4. To promote informed discussions and decision-making in AI governance and risk management

The AI Risk Index was conceived as a tool to help researchers, policymakers, and the general public understand the nuanced and dynamic nature of AI risk. By aggregating and analyzing data from multiple sources, AIRI aims to offer an objective measure of the balance between AI capabilities and safety efforts. By providing a regularly updated index, we hope to foster informed decision-making, promote a well-rounded discourse on AI development and its associated risks, and encourage proactive measures to ensure the safe and beneficial development of artificial intelligence.

Index Composition

AIRI is constructed from two main categories of data, updated weekly: future outlooks and the current situation. Each of these categories is further divided into sub-components that capture critical aspects of AI risk. Scores closer to 0 indicate a risk for stagnancy in AI developments, while closer to 100 denote risk for unruly capabilities.

Figure 1. Overview of the AIRI project

For the initial version of AIRI, we decided to allocate equal weights to the present situation and future outlooks, with each contributing 50% to the overall index. Currently, the index composition is as follows:

Future Outlooks: Investment Data

• AI Capabilities Investment (25%): Represents funding directed towards advancing AI technologies.
• AI Safety Investment (25%): Represents funding aimed at ensuring that AI development aligns with ethical and safety standards.

Within the investment data category, we debated whether AI capabilities should have a higher weight due to their direct impact on AI progress. However, we concluded that both capabilities and safety investments are equally critical and that at this point we do not have data to prioritize one over other, thus assigning each a 25% weight.

Investment data is utilized as a predictor of the future of AI risk, as it indicates the flow of resources, money, people and attention. Publicly disclosed data is gathered from Crunchbase, and investments are categorized into advancing AI capabilities or AI safety. The investment amounts are processed through a sigmoid function that is most sensitive in the range of $100 million to $1 billion USD. A two-week moving average is used to smooth out short-term fluctuations and sporadically undisclosed investments.

Current Situation: Sentiment Analysis

• Regulatory Sentiment (25%): Posts related to AI governance, policies, and regulations.
• Public Sentiment (25%): Posts discussing AI capabilities and safety, excluding those on regulatory matters.

Sentiment analysis is conducted on posts from X (formerly Twitter). Due to the uncertainties associated with assessing the long-term efficacy of policymaking, we decided to evaluate the sentiment towards AI regulations. We also gauged the public opinion by focusing on discussions about AI capabilities and safety concerns, excluding regulatory matters.

For both sentiment analyses, we first ran the posts through NLTK’s Sentiment Intensity Analyzer to get a sentiment score between -1 and +1, with 0 denoting perfectly neutral sentiment. We then categorized the posts into AI capabilities or AI safety/alignment based on the keywords used. Lastly, we combined the sentiment score and the category so that the sentiment score of posts with the AI safety/alignment category are multiplied by -1. Scaling this to between 0 and 100 to fit our index makes posts categorizing now as:

• 0–25% Positive sentiment on AI Safety
• 25–50% Negative sentiment on AI Safety
• 50–75% Positive sentiment on AI Capabilities
• 75–100% Negative sentiment on AI Capabilities

Final Score

The final AIRI score is the equally weighted average of these 4 indicators.

Outputs of the Project

AI Risk Index provides two main outputs to disseminate its findings and promote awareness:

• Website: The official website, airiskindex.com, serves as the primary hub for AIRI. It features real-time display of the current AIRI score, historical data, trend analysis, and detailed explanations of the methodology.
• X/Twitter Bot: An automated X/Twitter bot posts weekly updates of the AIRI score and shares significant changes or milestones in the index, ensuring timely dissemination of information.

Results and Future Directions

Our initial calculations cover the past 10 weeks, providing a baseline for future comparisons. These results, as can be seen on AIRI’s website, highlight the current balance between AI capabilities and safety efforts, offering valuable insights into the state of AI risk.

AI Risk Index is designed to be an iterative project, allowing for continuous refinement and improvement as we learn more about the behavior of the index and its underlying dimensions. Future improvements may include expanding data sources, refining sentiment analysis techniques, and adjusting weightings based on findings and expert feedback. As we gather more data and refine our methodologies, the index will evolve to provide even more accurate and actionable insights. Our ultimate aim is to create a robust tool that helps navigate the complex landscape of AI development and mitigate the risks associated with superintelligent AI.

We believe the AI Risk Index represents a significant step forward in our ability to quantify and monitor the complex landscape of AI development and its associated risks. AIRI offers a comprehensive view of where we stand and where we might be headed by balancing current sentiment with future outlooks. By providing a clear, data-driven metric, we hope to foster more informed discussions and decision-making in the crucial field of AI safety and governance. As we continue to develop and refine this tool, we invite the community to engage with our findings and contribute to the ongoing dialogue on AI safety and governance.

Sources:

[1] Grace, K., et al. (2018). When Will AI Exceed Human Performance? Evidence from AI Experts. Journal of Artificial Intelligence Research, 62, 729–754.
[2] AI Risk Index. (2024). https://airiskindex.com
[3] Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. Oxford University Press.
[4] Russell, S. (2019). Human Compatible: Artificial Intelligence and the Problem of Control. Viking.
[5] Omohundro, S. M. (2008). The Basic AI Drives. Frontiers in Artificial Intelligence and Applications, 171, 483–492.
[6] Yudkowsky, E. (2008). Artificial Intelligence as a Positive and Negative Factor in Global Risk. In Global Catastrophic Risks, edited by Nick Bostrom and Milan M. Ćirković. Oxford University Press.
[7] Russell, S., Dewey, D., & Tegmark, M. (2015). Research Priorities for Robust and Beneficial Artificial Intelligence. AI Magazine, 36(4), 105–114.
[8] Bostrom, N. (2002). Existential Risks: Analyzing Human Extinction Scenarios and Related Hazards. Journal of Evolution and Technology, 9.
[9] Yudkowsky, E. (2017). There’s No Fire Alarm for Artificial General Intelligence. Machine Intelligence Research Institute.