What researchers need to know about AI in scientific publishing

Scientists face constant pressures to maintain a strong publication record. AI tools can make writing easier, but their limitations pose risks for scientific publishing and for authors.

The American Society for Biochemistry and Molecular Biology has adopted clear guidelines governing AI-assisted writing in its journals. Because publishers cannot reliably detect AI-generated text, authors must understand the technology’s limitations and use it responsibly.

ASBMB editorial policies: How AI can and cannot be used

Can AI be an author?

To put it simply: No.

ASBMB Editorial Policies require authors to be “accountable for all aspects of the work” and to resolve questions about accuracy or integrity. An AI tool cannot accept responsibility for inaccuracies because it simply generates output based on its programming. Human authors remain responsible for the accuracy of their manuscripts.

However, ASBMB editorial policies do allow authors to use AI to improve the grammar or style of existing text. This recognizes that AI can help authors communicate more clearly, particularly those writing in a second language. AI may polish text but may not write any part of a manuscript.

Does AI use need to be declared?

Yes, and in a specific way. Rather than listing an AI tool as an author or in the acknowledgments section, authors must disclose AI use in a separate section, consistent with Elsevier publications. The section is entitled “Declaration of generative AI and AI-assisted technologies in the writing process,” and along with the name of the tool used by the authors, it must state that the authors have reviewed the output and take responsibility for its content.

Can AI be used to produce figures for a manuscript?

In general, AI tools may not be used to create, manipulate or refine images. The exception is when AI-generated images are produced as part of the experimental design. For example, if a study uses deep learning to improve medical imaging, the resulting AI-generated images may be included. Authors must describe how the images were generated in sufficient detail for others to reproduce the work, just as they would any other experimental method.

How AI introduces errors into scientific writing

Why does AI introduce errors?

AI language models generate text by predicting the most likely next word. Each word is selected based on patterns learned from large training datasets. This works well when the training data is high quality, and the request aligns with patterns in that data. If you ask a model trained only on commentaries about Moby-Dick to write a biology paper about whales, the results probably won’t be useful. Even so, the model will still generate an answer without indicating that it lacked the information needed to respond accurately. It will fill in the gaps. This is referred to as “hallucination,” and it’s a feature of AI that is unlikely to disappear.

Hallucinated citations

Hallucinations are especially concerning when they appear in a manuscript’s reference list. Unfortunately, fabricated references are becoming increasingly common. In biomedical literature, an estimated one in 277 papers now contains fabricated references, sometimes called “aren’ticles.” These fabricated citations often appear convincing enough to escape peer review, even including inaccurate or nonexistent DOIs. Authors should carefully verify every AI-generated citation. Because references may function as research data, fabricating them could constitute research misconduct.

Tortured phrases

AI-assisted writing can also produce so-called “tortured phrases,” in which familiar scientific terms are distorted. These phrases often arise when AI-generated text is processed by plagiarism-avoidance software that replaces words with synonyms. Because technical terms cannot simply be replaced with synonyms, the result can be absurd phrases such as “bosom peril” instead of “breast cancer” or “polymerase chain response” instead of “polymerase chain reaction.”

Approaching a powerful tool with caution

AI will undoubtedly continue to shape scientific writing, although its long-term impact on science remains uncertain. When AI contributes to scientific writing, careful human oversight remains essential. Left to its own devices, AI can confidently generate falsehoods that mislead readers and damage authors’ credibility. Regardless of how technology evolves, clear guidelines and responsible use will be essential for protecting the integrity of the scientific record.