AI Slop and the Battle to Protect Scientific Integrity

Freelance writer and doctor of applied mathematics Benjamin Skuse tested IEEE’s publishing defenses by submitting AI-generated research papers, demonstrating how easily plausible slop can penetrate peer-reviewed scholarly channels. A recent scroll through social media gotJorge Cuadal Calle thinking about trust, as about half of social media feeds are filled with AI slop that astute users call out as machine-generated, sometimes to the embarrassment of the rube who unknowingly spreads it or to the delight of the poster who shares it deliberately to amuse or sow confusion. Remembering the era of GPT-3 when chatbots quickly spat back plausible but false answers, that period marked when the word “hallucination” as applied to AI answers entered the lexicon. Six years and several versions of a few different chatbots later, users still cannot trust the answers they get, even though systems like Claude or Gemini often do useful work, leaving current AI requiring users to trust but verify.

The Tech TL;DR:

• Benjamin Skuse generated plausible scientific papers using free tools and launched them straight into IEEE’s publishing workflow.
• Advanced Large Language Models now avoid elementary errors, escalating an arms race between AI-generated slop and research-integrity detection systems.
• Volunteer initiatives like WikiProject AI Cleanup face parallel battles maintaining factual integrity on platforms like Wikipedia.

Faking Scientific Papers Through IEEE Defenses

As features and AI beat editor Eliza Strickland reported for IEEE Spectrum, Benjamin Skuse wanted to test the resilience of academic gatekeepers against automated text generation. Skuse approached the experiment by executing basic automated runs and generating full research papers without manual intervention or substantive revision. According to Strickland, Skuse was surprised at just how effortless it was to generate a paper that was more or less plausible and more or less science. For one of his experiments, as Strickland noted, Skuse basically hit run experiment and write paper, and then he didn’t do a damn thing to it before launching it into the maw of the publishing engine.

Skuse discovered that IEEE’s human-review processes and automated research-integrity checker programs are currently holding their ground. However, this dynamic functions as an ongoing cat-and-mouse game against users deploying free tools. Because advanced LLMs have stopped making the obvious logical missteps characteristic of earlier iterations, publishers find themselves locked in an escalating arms race against increasingly sophisticated synthetic submissions.

Volunteers Battle AI Slop on Wikipedia

Commercial scholarly publishers are not alone in attempting to filter out machine-generated material. Wikipedia faces the identical operational pressure on its crowdsourced platform. For a recent Five Questions feature reported by Strickland, the publication interviewed Ellia Lebleu, a founding member of WikiProject AI Cleanup. Lebleu leads a volunteer effort dedicated to detecting and purging AI-generated material from Wikipedia.

Lebleu’s team confronts the exact same structural phenomenon as commercial publishers, but operates with significantly fewer personnel and tooling resources. As Strickland observed regarding how this affects the platform, Wikipedia is known as a repository of human-generated information, and they want to hold on to that, though it is unclear how long volunteers can sustain that defense.

The Erosion of Institutional Trust

Ultimately, however, pervasive uncertainty undermines public trust in the institutions built to curate human knowledge and promote technological advancement for humanity.

Strickland also raised a stark long-term scenario regarding autonomous systems: in a future where artificial intelligence conducts the underlying science and writes the resulting papers, human validation of outputs may become obsolete. As Strickland framed the trajectory, it feels striking to contemplate a future where AI writes exclusively for other AI systems, leaving human researchers entirely out of the loop.

# Basic validation check for synthetic text probability
import openai

def evaluate_submission(abstract_text):
    response = openai.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": f"Analyze for synthetic patterns: {abstract_text}"}]
    )
    return response.choices[0].message.content