"Well-Formulated Crap": The Real Problem with AI in Scientific Writing

Nature recently published a survey on AI in research. Asked about the impact of large language models on scientific publishing, one respondent offered a memorable phrase:

"Well-formulated crap."

That, the respondent said, is what AI produces when applied to scientific writing without adequate human oversight. Text that is grammatically flawless, stylistically appropriate, and scientifically unreliable.

This captures something that the current debate about AI in science consistently misses. The conversation has been framed as a language problem: can the AI produce good English? In fact it is a logic problem. Can AI produce good science?

At this stage, the answer is no.

We have been editing manuscripts for Japanese biomedical researchers since 1995. For most of that time, the challenge was clear: our clients had excellent science and imperfect English. The editing profession existed to bridge that gap. Fix the grammar, refine the phrasing, and ensure the journal's style requirements were met. The science was the author's job.

AI has inverted this. Any researcher can now produce grammatically correct English in seconds. The language barrier that sustained an entire global editing industry, with an estimated global value of $4–5 billion, has effectively disappeared. What remains is something far more difficult to fix.

When a large language model "helps" write a Discussion section, it does not understand the data. It generates plausible-sounding interpretations based on patterns in its training corpus. It connects findings to literature it has not actually read. It draws conclusions that sound authoritative but may not follow from the results. It strips away the hedging and qualification that distinguish careful science from reckless speculation.

The output reads beautifully. But it may be wrong.

This is the trap. A non-native English speaker who previously submitted a paper with imperfect grammar but sound logic can now submit a paper with perfect grammar and compromised logic. The reviewer, who would once have said "the language needs to be improved" — a frustrating but ultimately fixable comment — now says "the claims exceed the data" or "the interpretation is not supported by the results.” This is a far more serious problem that cannot be fixed by running the text through another round of AI.

The editing industry needs to reckon with this shift. Grammar correction is no longer a scarce service. What is scarce, and what researchers increasingly need, is a human eye to verify that the science in the manuscript holds together: claims match data; citations are real; and the logical architecture of the argument is sound.

We are adjusting our own practice accordingly. But the broader point stands: the next time someone tells you that AI has "solved" scientific writing, ask them whether it has solved the science. Perfect English and flawed reasoning is a more dangerous combination than imperfect English and solid data. Or in words, well-formulated crap.