AI-generated research is a threat, both in terms of society’s knowledge and public trust in science. This was the conclusion made by the researchers from the Swedish School of Library and Information Science, University of Borås, Sweden, that recently identified over a hundred suspected AI-generated articles in the Google Scholar search engine.
A new study has discovered that fake AI-generated scientific articles have flooded the search engine Google Scholar. The study's findings of AI-fabricated junk science mean that fake science has been made available and can be spread widely and at a much lower cost for malicious actors. This poses a danger to both society and the research community, according to Jutta Haider and Björn Ekström from the Swedish School of Library and Information Science, who are behind the study together with Kristofer Söderström at Lund University and Malte Rödl at the Swedish University of Agricultural Sciences.
Increased risk of evidence hacking
One of the major concerns with AI-generated research is the increased risk of evidence hacking – that fake research can be used for strategic manipulation.
“The risk of what we call ‘evidence hacking’ increases significantly when AI-generated research is spread in search engines. This can have tangible consequences as incorrect results can seep further into society and possibly also into more and more domains,” said Björn Ekström, who holds a doctorate in Library and Information Science.
The researchers behind the study have already seen that these problematic articles have spread to other parts of the research infrastructure on the web, in various archives, social media and the like. The spread is fast and Google Scholar makes the problematic articles visible. Even if the articles are withdrawn, there is a risk that they have already had time to spread and continue to do so.
In addition, AI-generated research causes problems for the already hard-pressed peer review system.
Places higher demands on information literacy
The fact that AI-generated research is spreading in search engine databases places higher demands on those who take part in research online.
“If we cannot trust that the research we read is genuine, we risk making decisions based on incorrect information. But as much as this is a question of scientific misconduct, it is a question of media and information literacy,” said Jutta Haider, Professor of Library and Information Science.
She points out that Google Scholar is not an academic database. The search engine is easy to use and fast yet lacks quality assurance procedures. That's already a problem with regular Google results, but is even more problematic when it comes to making science accessible.
“People's ability to decide which journals and publishers – for the most part – publish quality-reviewed research is important for finding and determining what constitutes reliable research and is of great importance for decision-making and opinion formation,” concluded Jutta Haider.
The study was carried out within the framework of the Mistra Environmental Communication research programme.