AI won’t fix a broken system

A study published earlier this year in Nature analysing 41 million research papers across the natural sciences found something that should unsettle anyone who believes that AI will revolutionise scientific discovery[1]Hao, Q. et al. Artificial intelligence tools expand scientists’ impact but contract science’s focus. Nature 649, 1237–1243 (2026)..

Yes, scientists who adopt AI tools publish three times more papers and receive nearly five times more citations. Their careers accelerate. However, the collective range of scientific topics under investigation shrinks by nearly 5% and researchers’ engagement with one another’s work drops by 22%. The tools that turbocharge individual scientists may, in fact, be narrowing science as a whole.

Another study, published last December in Science, examined more than two million preprints and found that large language model (LLM) use is associated with posting up to 60% more manuscripts[2]Kusumegi, K. et al. Scientific production in the era of large language models. Science 390(6779), 1240–1243 (2025).. But for LLM-assisted papers, writing complexity is correlated with lower publication probability, which is the opposite of what has historically been true. One interpretation is that researchers are churning out work of questionable depth dressed up in polished prose.

Major discoveries are often linked with new views on nature, not with optimised analysis of standing data

Meanwhile, studies of AI-assisted writing have documented a ‘homogenising’ effect[3]Moon, K. et al. Homogenising effect of large language models on creative diversity. Comput. Hum. Behav. Artif. Hum. 6(100207) (2025)..[4]Sourati, Z. et al. The homogenising effect of large language models on human expression and thought. Trends Cogn. Sci. (2026).. In short, people use AI to produce work that may be individually refined, but is more similar to the work of other authors. One study, of over 2,000 college admissions essays, found that each additional human-written essay contributed more new ideas to the collective pool than each additional AI-generated essay, and this gap widened as more essays were analysed.

Some scholars have warned of ‘scientific monocultures’, where reliance on the same AI tools, trained on the same data, causes scientists’ questions and methods to converge[5]Messeri, L. & Crockett, M.J. Artificial intelligence and illusions of understanding in scientific research. Nature 627, 49–58 (2024).. When AI is positioned as an objective collaborator that can overcome human bias, researchers may grant it unwarranted trust, believing they understand more than they do because the tools produce confident-seeming outputs from data with which scientists never fully engaged themselves.

AI tools need large data sets to function, but the questions that lack abundant data – which may very well be some of the most important ones – are at risk of being left behind. AI leaders continue to promise that their tools will cure cancer, double the human lifespan and compress a century of biology into a decade, justifying massive investment and positioning AI as the saviour of science itself. However, the reality is turning out to be more complicated. AI may be accelerating the markers of scientific production while potentially degrading both the quality and diversity of what gets produced. And it isn’t accelerating science so much as optimising scientists to thrive in an already broken reward system.

Academic churn

Researchers are responsible for raising their own funds through grants, with success rates of around 19% across UKRI grants[6]‘The grant lottery’. Nature Career News, September 2025.. That creates enormous pressure to keep producing – to always have the next application in the pipeline, the next paper ready to publish. Institutions reinforce this by judging researchers on what is easy to measure: publication counts, grant funding, citation metrics. These are markers of production, not progress.

The optimised response is risk aversion. If you want to keep your position, you have one shot to prove your research programme is worthwhile before the tenure review arrives. You can’t afford to spend years chasing uncertain ideas. Under these pressures scientists inevitably pursue safe, incremental projects that will reliably yield papers, even if they never significantly advance understanding. In 2015 a survey by Pew found that 69% of scientists from the American Association for the Advancement of Science said that a “focus on projects expected to yield quick results has too much influence on the direction of research[7]Pew Research Center. Chapter 4: AAAS Scientists’ Views on the Scientific Enterprise (2015).”.

Now add AI to the mix. These tools excel at processing data and finding patterns in data sets. They are exceptionally good at doing more of what is already being done, faster. But science doesn’t progress principally through optimised efficiency. As the Nature authors note, major discoveries are often linked with new views on nature, not with optimised analysis of standing data.

None of this means that AI tools are useless for science. They are clearly useful to individual scientists. But, zooming out, the bottleneck to scientific progress isn’t primarily technological. According to an interview with ex-NIH official Mike Lauer, in the US scientists spend roughly 45% of their time on admin rather than doing science, grant applications have ballooned from four pages in the 1950s to over 100 and the average age at which a scientist receives their first major independent grant is now 45. Think about that: someone can be trusted to perform brain surgery a decade before they are considered ready to run their own research programme.

These are unsexy problems of bureaucratic dysfunction, misaligned incentives and institutional inertia. They will not be solved by faster tools.

More of the same

This is not an argument that AI can’t advance science. It already has in fields from protein biology to nuclear fusion. But those breakthroughs involved systems designed to solve specific scientific problems. The broad adoption of AI suggests it could lead to more of what we are already doing, but faster – or, worse, simply enable the rapid production of mediocre manuscripts. Science advances not only by solving well-defined problems but by generating new ones, and the institutional problems shaping how most scientists actually use AI are upstream of any technology.

AI companies, science funders and policymakers seem to be treating AI as a magic accelerant – something to sprinkle onto the scientific enterprise to make it go faster. But, as one recent analysis put it, this is like “adding lanes to a highway when the slowdown is actually caused by a tollbooth”[8]Could AI slow science? . The question is not how to build more lanes, it is why the tollbooth is there in the first place.

Tim Requarth is a neuroscientist and journalist. He is research assistant professor at the NYU Grossman School of Medicine, New York, US, and author of the newsletter The Third Hemisphere.

References