The 2021 AI Index: Major growth despite the pandemic

The last decade was a pivotal one for the AI industry, and 2020 saw AI substantially increase its impact on the world despite the chaos brought about by the COVID pandemic: Technologists made significant strides in massive language and generative models; the United States witnessed its first drop in AI hiring ever—pointing to a maturation of the industry—while hiring around the world increased; more dollars flowed to government use of AI than ever before, while colleges and universities offered students double the AI courses from a few years ago.

These are just some of the findings from the 2021 AI Index, an annual study of AI impact and progress developed by an interdisciplinary team at the Stanford Institute for Human-Centered Artificial Intelligence (HAI) in partnership with organizations from industry, academia, and government.

"The impact of AI this past year was both societal and economic, driven by the increasingly rapid progress of the technology itself," said AI Index co-chair Jack Clark. "With the AI Index, we can actually measure and evaluate the changes, enabling leaders and decision makers to take meaningful action to advance AI responsibly and ethically with humans in mind."

The 2021 AI Index is one of the most comprehensive reports about AI to date, analyzing and distilling patterns about AI's impact on everything from national economies to job growth to the analysis of technical progress within AI research itself, and analysis of the diversity (or lack of) among the people who create AI systems.

The new index shows that, in 2020:

• Private investment in AI substantially increased—despite the COVID crisis negatively impacting the economy in other ways.
• China surpassed the U.S. in significant scholarly work. Chinese-affiliated scholars were cited in more peer-reviewed journals than any other country's scholars, indicating China's AI research has increased in quality and quantity. However, the United States has consistently (and significantly) more cited AI conference papers than China over the last decade.
• Synthetic media, colloquially known as deepfakes, are on the rise, with breakthroughs in the generation of synthetic text, imagery, and video demonstrating the progress of AI but also highlighting the potential for unethical or dangerous use.
• Ethical challenges of AI applications have become a bigger focus for the AI community, with a significant increase in papers mentioning ethics and related keywords between 2015 and 2020.
• Diversity in AI is low—in 2019, 45% of new AI Ph.D. graduates who stayed in the United States were white, while 2.5% were African American and 3.2% were Hispanic. AI researchers are forming more affinity groups to try to improve diversity in the field, and these groups are seeing significant growth in their membership and impact: Black in AI members had twice the number of papers accepted at NeurIPS in 2019 compared to 2017, and participation at workshops held by the Women in Machine Learning Group has grown from under 200 participants in 2014 to more than 900 in 2020.
• Since Canada published a national AI strategy in 2017, other nations have followed, with more than 30 countries committing to national AI strategies by 2020.
• More AI Ph.D.s took jobs in private industry rather than academia, and professors continued to leave higher education for roles in corporations.
• Corporations have come to dominate the tools that AI researchers use, with corporate-backed software libraries (Google's TensorFlow and Keras, and Facebook's PyTorch) becoming the most popular frameworks on GitHub.
• Government interest in AI continues to be significant, with the U.S. government spending billions of dollars across civil and non-civil uses of AI. AI has been mentioned three times more in this Congress than in the previous one.

Along with telling what happened in 2020, the index also highlights what researchers don't yet know about AI development and deployment. While assembling this report, the authors found a few areas where data sources are lacking or aren't—yet—comprehensive. For instance: Most government spending data is inconsistently structured and frequently untrustworthy; AI ethics as a field has made various qualitative advances but it's challenging to quantify these or model them over time; and data relating to diversity in AI—especially about the makeup of people at universities and companies—is relatively scarce.

"It's clear from the data that, in 2020, AI started to have a more significant impact on the world, while the technology continued to evolve very rapidly," Clark said. "Making this report has also highlighted for us the need for governments to invest more in gathering data about AI, and for researchers to develop new, harder tests for contemporary AI systems. Our bibliometric analysis shows that the United States and China have become peer nations with one another on AI development, when analyzed through a variety of dimensions."

AI Index steering committee member Barbara Grosz added, "It's encouraging to see in this year's AI Index report an increased commitment to examining matters of diversity in the field. The report makes evident, however, the lack of data available from academic institutions, scientific societies, and industry about the diversity of the AI research and industry communities—of students, scientists, and AI professionals. Diversifying the field is essential if it is to yield systems that work for all of society, and success requires that the research community and industry collect and analyze data that tracks progress."