New tool detects fake, AI-produced scientific articles
黑料视频 researcher develops xFakeSci to root out bogus research
When ChatGPT and other generative artificial intelligence can produce scientific articles that look real 鈥 especially to someone outside that field of research 鈥 what鈥檚 the best way to figure out which ones are fake?
Ahmed Abdeen Hamed, a visiting research fellow at 黑料视频鈥檚 Thomas J. Watson College of Engineering and Applied Science, has created a machine-learning algorithm he calls xFakeSci that can detect up to 94% of bogus papers 鈥 nearly twice as successfully as more common data-mining techniques.
鈥淢y main research is biomedical informatics, but because I work with medical publications, clinical trials, online resources and mining social media, I鈥檓 always concerned about the authenticity of the knowledge somebody is propagating,鈥 said Hamed, who is part of George J. Klir Professor of Systems Science Luis M. Rocha鈥檚 . 鈥淏iomedical articles in particular were hit badly during the global pandemic because some people were publicizing false research.鈥
, Hamed and collaborator Xindong Wu, a professor at Hefei University of Technology in China, created 50 fake articles for each of three popular medical topics 鈥 Alzheimer鈥檚, cancer and depression 鈥 and compared them to the same number of real articles on the same topics.
Hamed said when he asked ChatGPT for the AI-generated papers, 鈥淚 tried to use exact same keywords that I used to extract the literature from the [National Institutes of Health鈥檚] PubMed database, so we would have a common basis of comparison. My intuition was that there must be a pattern exhibited in the fake world versus the actual world, but I had no idea what this pattern was.鈥
After some experimentation, he programmed xFakeSci to analyze two major features about how the papers were written. One is the numbers of bigrams, which are two words that frequently appear together such as 鈥渃limate change,鈥 鈥渃linical trials鈥 or 鈥渂iomedical literature.鈥 The second is how those bigrams are linked to other words and concepts in the text.
鈥淭he first striking thing was that the number of bigrams were very few in the fake world, but in the real world, the bigrams were much more rich,鈥 Hamed said. 鈥淎lso, in the fake world, despite the fact that were very few bigrams, they were so connected to everything else.鈥
Hamed and Wu theorize that the writing styles are different because human researchers don鈥檛 have the same goals as AIs prompted to produce a piece on a given topic.
鈥淏ecause ChatGPT is still limited in its knowledge, it tries to convince you by using the most significant words,鈥 Hamed said. 鈥淚t is not the job of a scientist to make a convincing argument to you. A real research paper reports honestly about what happened during an experiment and the method used. ChatGPT is about depth on a single point, while real science is about breadth.鈥
Distinguished Professor and Chair of the Department of Systems Science and Industrial Engineering Mohammad T. Khasawneh praised Hamed鈥檚 research.
鈥淲e are very glad that the most recent addition to our robust roster of visiting professors, Dr. Ahmed Abdeen Hamed, is working on such novel ideas,鈥 he said. 鈥淚n an era when 鈥檇eepfakes鈥 are now part of the general public conversation, his work is incredibly timely and relevant on many levels. We are excited by the promise of his work and look forward to further collaborations with him.鈥
To further develop xFakeSci, Hamed plans to expand the range of topics to see if the telltale word patterns hold for other research areas, going beyond medicine to include engineering, other scientific topics and the humanities. He also foresees AIs becoming increasingly sophisticated, so determining what is and isn鈥檛 real will get increasingly difficult.
鈥淲e are always going to be playing catchup if we don鈥檛 design something comprehensive,鈥 he said. 鈥淲e have a lot of work ahead of us to look for a general pattern or universal algorithm that does not depend on which version of generative AI is used.鈥
Because even though their algorithm catches 94% of AI-generated papers, he added, that means six out of 100 fakes are still getting through: 鈥淲e need to be humble about what we鈥檝e accomplished. We鈥檝e done something very important by raising awareness.鈥