Dive is now part of ISPD Group.
Read the announcement to learn more.

Artificial Intelligence Index Report. Stanford University Human-Centered Artificial Intelligence

Stanford University Human-Centered Artificial Intelligence

The Artificial Intelligence Index Report 2025 offers a comprehensive overview of the artificial intelligence landscape, encompassing everything from technical advancements and research and development, to the ethical, economic, political, and educational implications at a global level. The report highlights the continued leadership of industry in the development of notable models and of academia in the most highly cited research, along with the significant growth in AI patenting. The report also analyzes the evolution of responsible AI, trends in public and private investment, and the increasing integration of AI in science and medicine, all while exploring public opinion and the emerging regulatory challenges in this dynamic field.



1. Research and Development (Chapter 1)

· Patents: Globally, there has been a steady and significant increase in the number of AI patents granted between 2010 and 2023, rising from 3,833 to 122,511. Notably, in the last reported year, this number grew by 29.6%. This highlights the increasing innovation and intellectual property activity in the field of artificial intelligence.

Over the past dozen years, the number of AI patents has grown steadily and significantly, increasing from 3,833 in 2010 to 122,511 in 2023. In the last year, the number of AI patents has risen 29.6%.


· Geographic Distribution of Patents: China is the dominant leader in the total percentage of AI patents granted worldwide, accounting for 69.70% in 2023. The United States (14.16%) and Europe (13.00%) follow at a considerable distance. This indicates a significant concentration of AI patenting activity in China.

Granted AI patents (% of world total) by select geographic areas, 2010–23 Source: AI Index, 2025 | Chart: 2025 AI Index report 69.70%, China”.


· Patents per Capita: When considering patents relative to population size, Luxembourg and South Korea show the highest number of AI patents granted per 100,000 inhabitants in 2023, with 17.27 and 15.31 respectively. In terms of growth in this metric between 2013 and 2023, Denmark and Australia experienced the largest percentage change, with increases of 8,216% and 6,317% respectively. This provides a different perspective on AI innovation, highlighting countries with high patenting activity relative to their population.

Granted AI patents per 100,000 inhabitants by country, 2023 Source: AI Index, 2025 | Chart: 2025 AI Index report Luxembourg 17.27, South Korea 15.31″.

Percentage change of granted AI patents per 100,000 inhabitants by country, 2013 vs. 2023 Source: AI Index, 2025 | Chart: 2025 AI Index report Denmark 8,216%, Australia 6,317%”.


· Notable AI Models: The computational power used to train notable AI models has grown exponentially since 2012, with the industry being the primary driver of this increase. There is also an observed trend towards the development of multimodal models, capable of processing and generating different types of data, such as text and images. This underscores the increasing scale and complexity of AI model development.


· Data Limits: The report addresses the important question of whether AI models will run out of data for training, presenting projections regarding the utilization of the stock of public data. This explores potential limitations to future AI development due to data scarcity.


· Hardware and Energy Efficiency: A significant improvement in the energy efficiency of AI hardware is highlighted, with examples like the Nvidia B100 being 33.8 times more efficient than the Nvidia P1001. This trend towards more energy-efficient hardware is crucial for the sustainability of AI development, given its increasing computational demands.

This means the B100 is 33.8 times more energy efficient than the P100.


· Environmental Impact: The report estimates the carbon footprint associated with training select AI models, comparing it to the emissions from everyday activities. This analysis sheds light on the environmental costs of AI development and raises awareness about its impact.


· Open-Source AI Software: The United States and Europe lead in the number of accumulated GitHub stars for projects located in their regions. However, India and China show strong year-over-year growth in this area, indicating a growing contribution to open-source AI by these regions.




2. Technical Performance (Chapter 2)

• Language Models (LLMs): The performance of various LLMs is evaluated using benchmarks such as MMLU (Massive Multitask Language Understanding), which tests a model’s ability to understand and reason across a wide range of topics. The report highlights top-scoring models like o1-mini (92.0), o1-preview (90.4), and Claude-3.5-Sonnet (85.2) as of November 2024 on the Arena-Hard-Auto leaderboard. This reflects the continuous advancements in the capabilities of large language models.

As of November 2024, the top-scoring models on the Arena-Hard-Auto leaderboard were o1-mini (92.0), o1-preview (90.4), and Claude-3.5-Sonnet (85.2).


• Long Context: The report emphasizes the importance of evaluating information retrieval capabilities in long contexts, which is crucial for handling complex tasks and large documents. Google’s Gemini model set a new benchmark with its 1 million token context window, significantly exceeding GPT-4 Turbo’s 128K token limit. This advancement allows models to process and understand much larger amounts of information.

Google’s Gemini model set a new benchmark with its 1M token context window, far exceeding GPT-4 Turbo’s 128K token limit.


• Image and Video Generation: Notable progress in the quality of video generation is evident, with comparisons between examples from 2023 and 2024. Models such as DeepMind’s Veo and Pika show significant improvements in generating realistic and coherent videos from text prompts. This indicates rapid advancements in AI’s ability to create visual content.


• Coding: Benchmarks like SWE-bench and BigCodeBench are used to evaluate the coding capabilities of AI models. The report presents the performance of models such as GPT-4o and Qwen 2.5-Coder on these benchmarks, highlighting the increasing proficiency of AI in software development tasks.


• Mathematics and Reasoning: The report highlights advancements in solving complex mathematical problems (IMO-AG-30) and performance on general reasoning benchmarks like MMMU and GPQ. This demonstrates the growing ability of AI to tackle sophisticated cognitive tasks.


• Robotics: The report illustrates the advancements in the ability of AI-trained robots to perform complex tasks, as demonstrated by the ALOHA robot. This showcases the increasing integration of AI into physical systems and its potential for automation.


• Benchmarking and Data Contamination: The report mentions emerging benchmarks for evaluating various model capabilities and cautions about the problem of data contamination in evaluation datasets3 . This raises important considerations for the reliability and validity of AI performance evaluations.




3. Responsible AI (Chapter 3)

• Factuality: The factuality of AI models is assessed using benchmarks like FACTS, which measure the accuracy and truthfulness of the information generated by models. The report shows the scores of models such as Gemini 1.5 Pro and GPT-4o on this benchmark. This is crucial for understanding the reliability of AI-generated content.


• Biases: The report examines implicit biases in language models related to race, gender, religion, and health. This analysis highlights the potential for AI systems to perpetuate and amplify societal biases, necessitating efforts to mitigate these issues.


• Security and Safety: The security of AI models is evaluated using the HELM Safety benchmark, which measures a model’s susceptibility to generating harmful or inappropriate content. The report shows the scores of various models over time, reflecting the ongoing efforts to improve the safety alignment of AI systems.




4. Economy (Chapter 4)

• Employment: The report analyzes the impact of AI on the labor market, showing the year-over-year percentage change in the number of online AI job postings across select sectors and the number of AI job postings in the United States by state in 2024. This provides insights into the evolving demand for AI-related skills across different industries and regions.


• AI Talent: The report analyzes the concentration of AI talent by gender and geographic location, showing the penetration of AI skills among men and women in different countries. It also analyzes the net AI talent migration, indicating the flow of AI professionals across different geographic areas.


• Investment: Global private investment in generative AI has increased significantly, reaching $33.94 billion in 2024. The number of newly funded AI companies and the average size of investments have also grown. This highlights the strong financial backing and growing interest in generative AI technologies.


• Corporate Activity: The report analyzes the adoption of AI in different business functions, showing the expected change in workforce size by function due to AI adoption. This provides a perspective on how AI is expected to reshape the workforce within organizations.


• Robot Deployment: The report analyzes the annual growth rate of industrial robots installed in different countries, showing significant variations. This reflects the varying levels of automation and adoption of robotics across different economies.




5. Science and Medicine (Chapter 5)

• Notable Discoveries: The report highlights important milestones in the application of AI in medicine and biology, such as AlphaFold 3 for protein-ligand docking, which shows improved performance compared to other docking tools. This underscores AI’s growing role in scientific breakthroughs.


• The Central Dogma: The report shows the growth of public protein science databases and the increase in AI-driven scientific publications in biology. It also analyzes the number of foundation models per microscopy techniques. This reflects the increasing use of AI in advancing biological research and understanding.


• Clinical Care and Imaging: The report compares the training dataset token volumes for medical versus nonmedical language and imaging models. It also mentions notable models and modeling approaches in medical imaging. This demonstrates AI’s increasing capabilities in analyzing medical images and supporting clinical decision-making.


• Drug Discovery: The report highlights the use of generative AI for in silico optimization of drug formulations, mentioning a recent study in Nature on the optimization of formulations and particle engineering using AI. This showcases AI’s potential to accelerate and improve the drug discovery process.


• Ethics in Medical AI: The report analyzes the top ethical concerns discussed in medical AI ethics publications and shows the NIH funding for medical AI ethics. This reflects the growing awareness and research focused on the ethical implications of AI in healthcare.




6. Public Policy and Governance (Chapter 6)

• Public Procurement: The report analyzes the median value of public AI-related contracts in select countries. This provides insights into government investment in AI through public contracts.


• U.S. Federal Spending: The report examines the distribution of U.S. federal spending on AI-related contracts by department. This shows which government agencies are investing the most in AI.


• Regulation of Deepfakes: The report presents the state of laws at the state level in the U.S. regulating AI-generated deepfakes in intimate imagery. This highlights the emerging legal landscape surrounding AI-generated content and its potential harms.




7. Education (Chapter 7)

• K-12 CS and AI Education: The report analyzes access to computer science (CS) and AI education in U.S. K-12 schools, showing disparities based on socioeconomic and geographic factors. This underscores the need to address inequities in access to foundational AI education.


• AP Computer Science Exams: The report shows the increase in the number of AP computer science exams taken, as well as the participation by race/ethnicity in relation to national demographics. This provides insights into the growing interest in computer science education and the representation of different demographic groups.


• Postsecondary CS and AI Education: The report analyzes the number of international CS PhD students in U.S. universities and the number of new ICT PhD graduates by country. It also presents the distribution of ICT graduates by level of education in different countries. This reflects the global landscape of advanced education in computer science and related fields.




8. Public Opinion (Chapter 8)

• Support for AI Policy: The report analyzes the likelihood of local U.S. officials making AI policy decisions by political party and year. This provides insights into the political dynamics influencing AI policy at the local level.




In summary, this report offers a detailed and multifaceted view of the current state and trends of artificial intelligence, encompassing research and development to the economic, social, ethical, and political impact at a global level. It emphasizes the rapid advancement of the technology, the increasing investment, the concentration of talent in certain regions, and the need to address the ethical and safety implications of AI. The report also highlights existing disparities in access to CS and AI education, as well as the growing attention of policymakers towards the regulation of this technology.


Linkedin
X (Twitter)
Facebook
Related Posts