AI Report Highlights Smaller, Better, Cheaper Models

Where AI Is Now: Smaller, Better, Cheaper Models

By Nicola Jones & Nature magazine

The artificial intelligence (AI) race is heating up: the number and quality of high-performing Chinese AI models is rising to challenge the US lead, and the performance edge between top models is shrinking, according to an annual state of the industry report.

The report highlights that as AI continues to improve quickly, no one firm is pulling ahead. On the Chatbot Arena Leaderboard, which asks users to vote on the performance of various bots, the top-ranked model scored about 12% higher than the tenth-ranked model in early 2024, but only 5% higher in early 2025 (see ‘All together now’). “The frontier is increasingly competitive — and increasingly crowded,” the report says.

The Artificial Intelligence Index Report 2025 was released today by the Institute for Human Centered AI at Stanford University in California.

On supporting science journalism

If you’re enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.

The index shows that notable generative AI models are, on average, still getting bigger, by using more decision-making variables, more computing power and bigger training data sets. But developers are also proving that smaller, sleeker models are capable of great things. Thanks to better algorithms, a modern model can now match the performance that could be achieved by a model 100 times larger two years ago. “2024 was a breakthrough year for smaller AI models,” the index says.

Bart Selman, a computer scientist at Cornell University in Ithaca, New York, who was not involved in writing the Index report, says it’s good to see relatively small, cheap efforts such as China’s DeepSeek proving they can be competitive. “I’m predicting we’ll see some individual teams with five people, two people, that come up with some new algorithmic ideas that will shake things up,” he says. “Which is all good. We don’t want the world just to be run by some big companies.”

Neck and neck

The report shows that the vast majority of notable AI models are now developed by industry rather than academia: a reversal of the situation in the early 2000s, when neural nets and generative AI had not yet taken off. Industry produced fewer than 20% of notable AI models before 2006, but 60% of them in 2023 and nearly 90% in 2024, the report says.

The United States continues to be the top producer of notable models, releasing 40 in 2024, compared with China’s 15 and Europe’s 3. But plenty of other regions are joining the race, including the Middle East, Latin America and southeast Asia.

And the previous US lead in terms of model quality has disappeared, the report adds. China, which produces the most AI publications and patents, is now developing models that match their US competition in performance. In 2023, the leading Chinese models lagged behind the top US model by nearly 20 percentage points on the Massive Multitask Language Understanding test (MMLU), a common benchmark for large language models. However, as of the end of 2024, the US lead had shrunk to 0.3 percentage points.

“Around 2015, China put itself on the path to be a top player in AI, and they did it through investments in education,” says Selman. “We’re seeing that’s starting to pay off.”

The field has also seen a surprising surge in the number and performance of ‘open weight’ models such as DeepSeek and Facebook’s LLaMa. Users can freely view the parameters that these models learn during training and use to make predictions, although other details, such as the training code, might remain secret. Originally, closed systems, in which none of these factors are disclosed, were markedly superior, but the performance gap between top contenders in these categories narrowed to 8% in early 2024, and to just 1.7% in early 2025.

“It’s certainly good for anyone who can’t afford to build a model from scratch, which is a lot of little companies and academics,” says Ray Perrault, a computer scientist at SRI, a non-profit research institute in Menlo Park, California, and co-director of the report. OpenAI in San Francisco, California, which developed the chatbot ChatGPT, plans to release an open-weight model in the next few months.

Better, smaller, cheaper

After the public launch of ChatGPT in 2022, developers put most of their energy into making systems better by making them bigger. That trend continues, the index reports: the energy used to train a typical leading AI model is currently doubling annually; the amount of computing resources used per model is doubling every five months; and the training data sets are doubling in size every eight months.

Yet companies are also releasing very capable small models. The smallest model registering a score higher than 60% on the MMLU in 2022, for example, used 540 billion parameters; by 2024, a model achieved the same score with just 3.8 billion parameters. Smaller models train faster, give faster answers and use less energy than larger ones. “It helps everything,” says Perrault.

Some smaller models can emulate the behaviour of larger models, says Selman, or take advantage of better algorithms and hardware than those in older systems. The index reports that the average energy efficiency of hardware used by AI systems improves by about 40% annually. As a result of such advances, the cost of scoring just over 60% on the MMLU has plummeted, from about US$20 per million tokens (bits of words produced by language models) in November 2022 to 7 cents per million tokens in October 2024.

Despite striking improvements on several common benchmark tests, the index highlights that generative AI still suffers from issues such as implicit bias and a tendency to ‘hallucinate’, or spit out false information. “They impress me in many ways, but horrify me in others,” says Selman. “They surprise me in terms of making very basic errors.”

This article is reproduced with permission and was first published on April 7, 2025.