Name: AI By The Numbers—And For The People
Uploaded: 2025-07-22
Duration: 10 min 53 s

What can a global, multi-source dataset tell us about where AI is headed, and who’s steering the ship? Russell Wald, Executive Director at Stanford HAI, breaks down key insights from the AI Index, the world’s most comprehensive report on AI’s progress and impact. In this fast-paced session, he highlights surprising shifts in talent, investment, and regulation—and makes the case for keeping AI innovation rooted in human values and societal good.

AI By The Numbers—And For The People

Conferences

So our next speaker takes us inside the AI index. This is one of the most comprehensive data sets tracking the field's rapid evolution. The AAI index reports on global trends, technical progress, investments, policies, and impacts. To tell us more about where AI is headed, please welcome Stanford Institute for Human Centered Artificial Intelligence executive Director Russell Walt. Russell. Good afternoon and thank you everyone. I'm Russell Wald, the executive director of the Stanford Institute for Human Centered Artificial Intelligence, or what we also call as HAI. We are Stanford University's globally recognized interdisciplinary Research Institute at the forefront of shaping AI development for the public good. HAI was established in 2019 with the goal of advancing AI research, education, policy and practice. And through our role. through our convening role and rigorous study of AI, we have become the trusted partner on AI governance for decision makers in industry, government, and civil society. And today I'm going to talk about what we produce at HAI, which is the AI index, and then which is an annual data-driven analysis of trends in AI and tracks research, development, deployment, and the socioeconomic impact of AI across academia, government, and industry. The index has gained worldwide recognition as the authoritative report on the state of AI. And for the next few minutes, I will highlight a few standout segments of the 2025 report, and I say it's a few segments of the report because it's over 500 pages, and these are just some quick hot takes from it. First up, we see AI performance consistently seems to improve year over year. An example of this is in this slide where we use mid journey, a text image generator with a query of a hyperrealistic image of Harry Potter. And from February of 2022 to July of 2024, we see rapidly increasing quality in these generated images. The progression demonstrates the significant improvement in Midd Journey's ability to generate hyperrealistic images over a two year period. In 2022, the model produced cartoonish, inaccurate renderings of Harry Potter, but by 2024, it could create starting, startlingly realistic depictions. And you'll note we have essentially gone from what mirrors a Picasso painting to an uncanny rendering of Daniel Radcliffe, the actor who played Harry Potter in the movies. And because of this consistent performance growth, we are increasingly challenged when it comes to benchmarking these models. As of 2024, there are very few task categories where human ability surpasses AI. And even in these areas, the performance gap between AI and humans is shrinking rapidly. Essentially, from image recognition to competition level mathematics to PhD level science questions, AI is exceeding human capabilities and it's becoming increasingly harder for us to benchmark that. AI is also coming into our everyday lives, and from healthcare to transportation, AI is rapidly moving from the lab into our daily life. In 2023, the US FDA approved 223 AI-enabled medical devices, and that's up from just 6 in 2015. On the roads, self-driving cars are no longer experimental. For example, Waymo, which I regularly take, living in San Francisco, is one of the largest US operators and provides over 150,000 autonomous rides each week. While Baidu's affordable Apollo Go Robo Taxi has a fleet now that serves numerous cities across China. Business use of AI increased significantly after stagnating from 2017 to 2023. The latest McKinsey report reveals that 78% of surveyed respondents say their organizations have begun to use AI in at least one business function, marking a significant increase from 55% in 2023. Driven by increasingly capable small models, the inference cost for systems for a system performing at the level of excuse me, the inference cost for a system performing at the level of GT 3.5 dropped over 280-fold between November of 2022 to October of 2024. Hardware costs have declined 30% annually, while energy efficiency has improved by 40% each year. I want to note, this is in reference to inference costs, not training costs, which I'll refer to in just a moment. And I'll also note here too from this, what we see is open weight models are also closing the gap with closed models, reducing the performance from 8% to just 1.7% on some benchmarks in a single year. Together, these trends. are rapidly lowering the barriers, uh, lowering the barriers to advanced AI. However, Even with infra and hardware costs going down, training costs remain out of reach for academia and most small players. Nearly 90% of notable AI models in 2024 came uh came from industry, which is up from 60% in 2023. And while academia remains a top source of highly cited research, it does struggle now at this point to stay as advanced in the game at the frontier level. Model scale continues to grow rapidly. Training compute doubles every 5 months, data sets every 8, and power use annually. Yet performance gaps are shrinking. The score difference between the top and 10th ranked models fell from 11.9% to 5.4% in a year, and the two top, the top two models are now separated by just 0.7%. Pardon me, that was the slide that was in reference to that. So give you all just a moment to see that I I didn't advance it. But the main point that I'm trying to make here is that the frontier is increasingly competitive and increasingly crowded. And In recent years, AI model performance at the frontier has converged with multiple providers now offering highly capable models. This marks a shift from late 2022 when Chat GBT's launch, widely seen as AI's breakthrough into the public consciousness, coincided with the landscape dominated by just two players, and that was OpenAI and Google. While the prior slide noted performance by model, this slide focuses on training cost by model. And what I want you to take away from this slide is one of the most important things is to note that the Transformer model costs $930 for Google to train in 2017, and that is the T and GPT, the baseline level of architecture, and now today we're at $200 million to train Gemini Ultra. Last year's AI index was among the first publications to highlight the lack of standard benchmarks for AI safety and responsibility evaluations, and while companies measure math, coding and language skills, there's no standardized benchmark for any of this. The index has also been analyzing global public opinion, and one point to note here is if you are from a non-Western industrialized nation, you are more likely to view AI positively than not. China has an 83% positive view. Indonesia 80%, and Thailand 77%, whereas Canada is at 40%, the US 39%, and the Netherlands 36%. And I'll close with the geopolitical situation in AI. The US still maintains a lead in AI followed closely by China. However, this gap is tightening. My intention with this slide is not to exacerbate the idea of an AI arms race between China and the US, but instead to highlight the different approaches between the most advanced frontier AI model developers. Over the last several years, the US has. On a few proprietary model providers. Meanwhile, China has deeply invested in its talent base and more importantly in open source environment which has allowed for a thriving base. If I were to as this trend continued and I appeared here next year, at this rate, China would surpass the US in terms of model performance. You can access this year's AI index by following the uh the QR code here. Thank you.