Highlights of AI Index Report 2023

Published in

𝐀𝐈 𝐦𝐨𝐧𝐤𝐬.𝐢𝐨

14 min readMay 7, 2023

Standford University Human-centered artificial intelligence (HAI) published the AI Index Report 2023 (you can find the complete report here).

There are 7 chapters which are included in that report. I have just summarized and pointed out the highlights of that report to make it easy to understand.

Small introduction

This report provides an overview of the state of artificial intelligence (AI) research and collaboration from 2010 to 2021. The report highlights various trends in AI research and development in 2022, including the saturation of traditional benchmarks, the emergence of generative AI and flexible AI systems, and the environmental impact of AI. Additionally, the report discusses recent trends in the field of AI and computer science, including a growing focus on AI specialization and a shift towards industry positions for AI PhDs. The role of policymakers and government in shaping the landscape of AI is growing in importance. Finally, the report touches on the diversity in computer science education in North America, noting improvements in ethnic diversity but a gender imbalance.

Chapter 1: Research and Development

The United States and China had the greatest number of cross-country collaborations in AI publications from 2010 to 2021, although the pace of collaboration has slowed.

AI research collaborations between the US and China increased by about 4 times since 2010
The number of collaborations was 2.5 times greater than that of the next nearest country pair (UK and China)
The total number of US-China collaborations increased by only 2.1% from 2020 to 2021. This is the smallest year-over-year growth rate since 2010.

AI research is on the rise, across the board

The total number of AI publications has more than doubled since 2010.
The dominant AI research topics include pattern recognition, machine learning, and computer vision.

China continues to lead in total AI journal, conference, and repository publications.

The United States leads in AI conferences and repository citations, but the leads are slowly decreasing.
The total number of AI publications has more than doubled since 2010.
The dominant AI topics in research are pattern recognition, machine learning, and computer vision.
American institutions produce the majority of the world’s large language and multimodal models (54% in 2022).

Industry races ahead of academia

Before 2014, the most significant machine learning models were released by academia.
Since 2014, the industry has taken over and produced more significant machine learning models.
In 2022, there were 32 significant industry-produced machine learning models compared to just three produced by academia.
Industry actors have more resources such as large amounts of data, computer power, and money than nonprofits and academia.
Building state-of-the-art AI systems requires more resources which industry actors inherently possess in greater amounts than non-profits and academia.

Large language models are getting bigger and more expensive

GPT-2 was released in 2019 and had 1.5 billion parameters and it cost an estimated $50,000 USD to train.
PaLM, launched in 2022, had 540 billion parameters and it cost an estimated $8 million USD to train.
PaLM was around 360 times larger than GPT-2 and cost 160 times more.
Large language and multimodal models are becoming larger and pricier across the board.

Chapter 2: Technical Performance

Performance saturation on traditional benchmarks

AI is still achieving state-of-the-art results
Year-over-year improvement on many benchmarks is marginal
Benchmark saturation is being reached at an increasing speed
New comprehensive benchmarking suites such as BIG-bench and HELM are being released

Generative AI breaks into the public consciousness

In 2022, new text-to-image models like DALL-E 2 and Stable Diffusion, text-to-video systems like Make-A-Video, and chatbots like ChatGPT were released.
These systems can generate outputs that are incoherent or untrue, making it difficult to rely on them for critical applications.

AI systems become more flexible

BEiT-3, PaLI, and Gato are single AI systems.
These systems can navigate multiple tasks such as vision and language.
They challenge the traditional trend where AI systems performed well on narrow tasks but struggled across broader tasks.

Capable language models still struggle with reasoning

Language models improved their generative capabilities.
However, they still struggle with complex planning tasks.

AI is both helping and harming the environment

AI systems can have significant environmental impacts, with some training runs emitting large amounts of carbon.
New research suggests that AI models can be used to optimize energy usage, such as through reinforcement learning.
BLOOM’s training run emitted 25 times more carbon than a single air traveler on a one-way trip from New York to San Francisco.
BCOOLER is an example of a reinforcement learning model that can optimize energy usage.

The world’s best new scientist … AI?

AI models used in 2022 to aid hydrogen fusion, improve the efficiency of matrix manipulation, and generate new antibodies.
AI is accelerating scientific progress.

AI starts to build better AI

Nvidia used an AI reinforcement learning agent to enhance chip design for AI systems.
Google utilized its language model, PaLM, to suggest improvements to the model itself.
Self-improving AI learning is expected to boost AI progress.

Chapter 3: Technical AI Ethics

The effects of the model scale on bias and toxicity are confounded by training data and mitigation methods

Several institutions have developed their own large models using proprietary data.
Large models are still prone to bias and toxicity issues.
New evidence suggests that these issues can be somewhat reduced by training larger models with instruction-tuning.

Generative models have arrived and so have their ethical problems

Generative models were popular in 2022
These models have ethical challenges
Text-to-image generators can be biased along gender dimensions
Chatbots like ChatGPT can be tricked into serving nefarious aims

The number of incidents concerning the misuse of AI is rapidly rising

The AIAAIC database tracks incidents related to the ethical misuse of AI
The number of AI incidents and controversies increased 26 times since 2012
Notable incidents in 2022 include deepfake video of the Ukrainian President and US prisons using call-monitoring technology on inmates
The increase in incidents is due to the greater use of AI technologies and awareness of misuse possibilities.

Fairer models may not be less biased

Extensive analysis of language models shows a correlation between performance and fairness.
However, fairness and bias can be in conflict with each other.
Language models that perform better on fairness benchmarks tend to have a worse gender bias.

Interest in AI ethics continues to skyrocket

The number of accepted submissions to the FAccT conference has doubled since 2021 and increased by a factor of 10 since 2018.
More submissions were received in 2022 from industry actors.

Automated fact-checking with natural language processing isn’t so straightforward after all

11 out of 16 automated fact-checking datasets rely on leaked evidence from fact-checking reports that did not exist at the time the claim was made.
This suggests that current datasets are insufficient for training models to perform automated fact-checking in real time.
Further research is needed to create more robust datasets for automated fact-checking.

Chapter 4: The Economy

The demand for AI-related professional skills is increasing across virtually every American industrial sector

The number of AI-related job postings has increased in every sector in the US except for agriculture, forestry, fishing, and hunting.
The increase in job postings from 2021 to 2022 is on average 0.2%.
Employers in the US are seeking workers with AI-related skills.

For the first time in the last decade, year-over-year private investment in AI decreased

Global AI private investment in 2022 was $91.9 billion, a 26.7% decrease from 2021.
The total number of AI-related funding events and newly funded AI companies also decreased.
However, AI investment has significantly increased over the last decade, with the 2022 amount being 18 times greater than it was in 2013.

Once again, the United States leads in investment in AI

The U.S. received the highest amount of AI private investment in 2022, with $47.4 billion invested, which is approximately 3.5 times more than China, the next highest country at $13.4 billion.
The U.S. also had the highest number of newly funded AI companies, with 1.9 times more than the European Union and the United Kingdom combined, and 3.4 times more than China.

In 2022, the AI focus area with the most investment was medical and healthcare ($6.1 billion); followed by data management, processing, and cloud ($5.9 billion); and Fintech ($5.5 billion).

AI private investment decreased in most AI focus areas in 2022 compared to 2021.
The three largest AI private investment events in 2022 were:

A $2.5 billion funding event for GAC Aion New Energy Automobile, a Chinese manufacturer of electric vehicles.
A $1.5 billion Series E funding round for Anduril Industries, a U.S. defense products company that builds technology for military agencies and border surveillance.
A $1.2 billion investment in Celonis, a business-data consulting company based in Germany.

While the proportion of companies adopting AI has plateaued, the companies that have adopted AI continue to pull ahead

The proportion of companies adopting AI has doubled since 2017.
The adoption rate has plateaued between 50% and 60% in recent years.
Organizations that have adopted AI report realizing significant cost decreases and revenue increases.
This information is based on McKinsey’s annual research survey.

AI is being deployed by businesses in multifaceted ways

The most commonly embedded AI capabilities in businesses are robotic process automation, computer vision, natural language text understanding, and virtual agents.
The most common AI use cases adopted by companies in 2022 were service operations optimization, creation of new AI-based products, customer segmentation, customer service analytics, and new AI-based enhancement of products.
AI adoption has more than doubled since 2017, with 50–60% of organizations having adopted AI in recent years.
Companies that have adopted AI have reported meaningful cost decreases and revenue increases.

AI tools like Copilot are tangibly helping workers

88% of surveyed respondents in a GitHub survey on the use of Copilot feel more productive when using the system.
74% feel they are able to focus on more satisfying work.
88% feel they are able to complete tasks more quickly.

China dominates industrial robot installations

China became the leading nation in installing industrial robots in 2013, surpassing Japan.
The gap between China and the second-leading nation in terms of industrial robot installation has widened since 2013.
In 2021, China installed more industrial robots than the rest of the world combined.

Chapter 5: Education

More and more AI specialization

The proportion of new computer science Ph.D. graduates from U.S. universities specializing in AI:

19.1% in 2021
14.9% in 2020
10.2% in 2010.

New AI PhDs increasingly head to the industry

In 2011, about the same proportion of new AI Ph.D. graduates took jobs in industry (40.9%) as in academia (41.6%).
However, since then, the majority of AI PhDs have headed to industry.
In 2021, 65.4% of AI PhDs took jobs in industry, more than double the 28.2% who took jobs in academia.

New North American CS, CE, and information faculty hires stayed flat

The total number of new North American CS, CE, and information faculty hires has decreased in the last decade.
There were 710 total hires in 2021 compared to 733 in 2012.
The total number of tenure-track hires peaked in 2019 at 422 and then dropped to 324 in 2021.

The gap in external research funding for private versus public American CS departments continues to widen

In 2011, the median amount of total expenditure from external sources for computing research was roughly the same for private and public CS departments in the US.
The gap between private and public universities’ total expenditure from external sources for computing research has widened since then.
In 2021, the median expenditure for private universities was $9.7 million, compared to $5.7 million for public universities.

Interest in K–12 AI and computer science education grows in both the United States and the rest of the world

In 2021, 181,040 AP computer science exams were taken by American students, a 1.0% increase from the previous year.
Since 2007, the number of AP computer science exams has increased ninefold.
As of 2021, 11 countries, including Belgium, China, and South Korea, have officially endorsed and implemented a K–12 AI curriculum.

Chapter 6: Policy and Governance

Policymaker interest in AI is on the rise

The number of bills containing “artificial intelligence” that was passed into law grew from 1 in 2016 to 37 in 2022, according to an AI Index analysis of legislative records of 127 countries.
Mentions of AI in global legislative proceedings have increased nearly 6.5 times since 2016, according to an analysis of parliamentary records on AI in 81 countries.

From talk to enactment — the U.S. passed more AI bills than ever before

In 2021, only 2% of federal AI bills in the United States were passed into law.
In 2022, the percentage of passed federal AI bills increased to 10%.
In 2021, 35% of state-level AI bills were passed into law.

When it comes to AI, policymakers have a lot of thoughts

Policymakers in different nations think about AI from a range of perspectives.
In the United Kingdom, policymakers discussed the risks of AI-led automation.
In Japan, policymakers considered the necessity of safeguarding human rights in the face of AI.
In Zambia, policymakers looked at the possibility of using AI for weather forecasting.

The U.S. government continues to increase spending on AI

U.S. government AI-related contract spending has increased about 2.5 times since 2017.

The legal world is waking up to AI

In 2022, there were 110 AI-related legal cases in US state and federal courts.
This is roughly seven times more than in 2016.
The majority of these cases originated in California, New York, and Illinois.
The cases concerned issues relating to civil, intellectual property, and contract law.

Chapter 7: Diversity

North American bachelor’s, master’s, and PhD-level computer science students are becoming more ethnically diverse

White students are still the most represented ethnicity among new resident bachelor’s, master’s, and PhD-level computer science graduates.
However, students from other ethnic backgrounds (such as Asian, Hispanic, and Black or African American) are becoming more represented.
In 2011, 71.9% of new resident CS bachelor’s graduates were white.
In 2021, that number dropped to 46.7%.

New AI PhDs are still overwhelmingly male. In 2021, 78.7% of new AI PhDs were male

In higher-level AI education, there continues to be a gender imbalance.
In 2021, only 21.3% of new resident bachelor’s, master’s, and PhD-level computer science graduates were female.
This is a 3.2 percentage point increase from 2011, but there is still a long way to go in terms of gender diversity in AI education.

Women make up an increasingly greater share of CS, CE, and information faculty hires

The proportion of new female CS, CE, and information faculty hires has increased from 24.9% to 30.2% since 2017.
However, most CS, CE, and information faculty in North American universities are still male (75.9%).
Only 0.1% of CS, CE, and information faculty identify as nonbinary as of 2021.

American K–12 computer science education has become more diverse, in terms of both gender and ethnicity

The share of AP computer science exams taken by female students increased from 16.8% in 2007 to 30.6% in 2021.
There has been an increase in the share of Asian, Hispanic/Latino/Latina, and Black/African American students taking AP computer science exams over the years.

Chapter 8: Public Opinion

Chinese citizens are among those who feel the most positively about AI products and services. Americans … not have so much

78% of Chinese respondents, 76% of Saudi Arabian respondents, and 71% of Indian respondents agreed that products and services using AI have more benefits than drawbacks.
Only 35% of sampled Americans agreed that products and services using AI had more benefits than drawbacks.

Men tend to feel more positively about AI products and services than women. Men are also more likely than women to believe that AI will mostly help rather than harm

Men are more likely than women to report that AI products and services make their lives easier.
Men are more likely to trust companies that use AI.
Men are more likely to feel that AI products and services have more benefits than drawbacks.
Men are more likely than women to agree with the statement that AI will mostly help rather than harm their country in the next 20 years.

People across the world and especially in America remain unconvinced by self-driving cars

Global survey: 27% of respondents reported feeling safe in a self-driving car.
Pew Research suggests that only 26% of Americans feel that driverless passenger vehicles are a good idea for society.

Different causes for excitement and concern

Among surveyed Americans, those who feel excited about AI are most excited about making life and society better (31%) and saving time, and making things more efficient (13%).
Among surveyed Americans, those who feel concerned about AI worry about the loss of human jobs (19%), surveillance, hacking, and digital privacy (16%), and the lack of human connection (12%).

NLP researchers … have some strong opinions as well

77% of NLP researchers agreed or weakly agreed that private AI firms have too much influence.
41% of NLP researchers said that NLP should be regulated.
73% of NLP researchers felt that AI could soon lead to revolutionary societal change.

Conclusion:

In conclusion, the report provides a comprehensive overview of the state of artificial intelligence research and development from 2010 to 2022. AI research is on the rise, with China leading in the total number of AI publications while the United States leads in AI conferences and repository citations. The landscape of AI research and development is constantly evolving, with the emergence of generative and flexible AI systems that can navigate multiple tasks. However, capable language models still struggle with complex planning tasks. While AI has shown great potential, ethical concerns have surfaced regarding the ethical misuse of AI. Furthermore, policymakers and governments play a growing role in shaping the landscape of AI. There is a widening gap in external research funding between private and public American CS departments, and there are still issues related to diversity in AI education. Despite these challenges, AI is still seen as a promising field, and the demand for AI-related skills is increasing across virtually every industrial sector in the US.

My Opinions on the basis of these highlights:

Advantages:

AI research and development is on the rise, with the total number of publications more than doubling since 2010.
Large language and multimodal models are being produced, which are useful in various industries.
AI is accelerating scientific progress and is starting to build better AI.
AI tools like Copilot are helping workers, with users reporting increased productivity and completion of tasks more quickly.
Interest in K-12 AI and computer science education is growing in many countries.

Disadvantages:

Large language models are getting bigger and more expensive, making it difficult for smaller organizations to keep up.
AI systems can still generate outputs that are incoherent or untrue, making it difficult to rely on them for critical applications.
AI incidents related to ethical misuse are increasing, and interest in AI ethics is growing.
Bias and toxicity issues are still present in AI models.

How AI can be harmful in upcoming days:

Here are some of the ways in which AI can be harmful based on the report:

AI incidents related to ethical misuse have rapidly increased since 2012, as more AI technologies are used and awareness of their misuse possibilities increases. Ethical challenges have surfaced with generative models, such as text-to-image generators and chatbots, being tricked for nefarious purposes.
Large models continue to be developed by various institutions, but they are still prone to bias and toxicity issues. AI models can amplify existing biases in data, leading to discrimination against certain groups of people or perpetuating societal inequities. Toxicity issues refer to the tendency of some AI models to generate offensive or harmful content, such as hate speech or violent imagery.
AI systems can generate outputs that are incoherent or untrue, making it difficult to rely on them for critical applications. Although AI is improving, capable language models still struggle with complex planning tasks and reasoning tasks.
The environmental impact of AI is a concern, as the development and use of AI systems require significant amounts of energy and computing power. However, the report also suggests that AI models can be used to optimize energy usage and reduce the environmental impact of other industries.
AI-related legal cases have been on the rise, with issues ranging from civil to intellectual property and contract law. This highlights the need for regulations and laws to govern the use of AI technologies.

Overall, while AI has the potential to bring many benefits, it is important to be aware of the potential harms and challenges associated with its development and use. It is essential to ensure that AI technologies are developed and used responsibly, with careful consideration of ethical and societal implications.

Thank you for taking the time to read this article. I hope you found it informative and thought-provoking