AIGenerated
EN · IT · ES

New method makes AI text detectors 93% accurate, catching disguised AI writing

A training trick lets detection tools adapt in real time so they stay accurate as AI and human writing styles keep changing.

Researchers from MIT and Cornell developed a way for AI detectors to keep adapting as writing styles change.
Picture: news.mit.edu
How it works

A training trick that lets AI detectors improve themselves after release, catching AI writing that humans try to disguise.

  1. New text enters the detector as it works
  2. It spots patterns that separate human from AI writing
  3. It quietly adjusts its own settings to get sharper

What you could do with it

  • A teacher could use an AI detector equipped with test-time adaptation to confirm whether a student's term paper was likely authored by a large language model rather than by the student, helping them catch disguised AI writing.1
  • A cybersecurity expert could deploy the adapted detector to spot malicious emails or SEO fraud where attackers disguise AI-generated content as human-written to evade standard detection tools.1
  • A hiring manager could rely on the resilient detector to evaluate application materials and flag submissions that were artificially generated and altered to appear human, improving screening reliability.1

From the announcement: A robotic hand holding a pen writing on a yellow document with the word 'HUMAN' visible among stylized text lines and sparkle decorations

Picture: news.mit.edu

The problem with today's detectors

AI detectors are themselves AI models that compare a piece of writing against patterns learned from known human and AI text 1. When the text they are asked to review suddenly looks different from what they were taught, their accuracy drops — a problem researchers call distribution shift .

This happens for two reasons. First, human writing styles naturally evolve: people write about different topics, adopt new analogies, and change how they structure sentences over the years 1. A detector trained on 2010-era text may flag natural 2030-style writing as AI-generated . Second, once a detector goes public, developers of "humanizer" tools race to craft text that slips past it, creating a constant cycle of adaptation and counter-adaptation .

In tests designed to mimic this cat-and-mouse game, a commercial detector caught only about 27 percent of AI-generated text that had been deliberately disguised 1.

At a glance

How accurate are AI detectors before and after the fix?

Traditional (detector)Adapted (detective)
Gradual shift in human writing73%85%
Deliberately disguised AI text27%93%

MIT and Cornell researchers testing detection tools against changing human styles and adversarial 'humanizer' tools.

How test-time adaptation works

Instead of locking the detector after release, researchers let it quietly adjust itself as new text flows through it 1. This incoming text is unlabeled — the system does not know which pieces are human or AI — but it still represents current writing habits from both groups . Over time, the detector picks up on statistical patterns that separate the two, refining its own internal settings without needing a full retraining from scratch .

To study whether this actually works, the researchers created datasets mixing human-written and AI-written text, then ran some of the AI examples through humanizer tools specifically designed to defeat existing detectors 1. They then applied test-time adaptation and evaluated performance on labeled ground-truth data .

The results

When mimicking the gradual shift in human language, the adapted detector reached 85 percent accuracy compared with 73 percent using traditional methods 1. On disguised AI text, it achieved 93 percent accuracy versus just 27 percent for the commercial baseline .

Adapting to slower, subtler changes in human writing remains harder than catching deliberately deceptive AI text, the researchers noted 1.

The team will present the work at the Conference on Neural Information Processing Systems (NeurIPS) 1. Study co-author Manish Raghavan, who holds positions at MIT Sloan School of Management, MIT Schwarzman College of Computing and MIT's Laboratory for Information and Decision Systems, said: "We've shown that test-time adaptation allows your AI detector to be much more robust, even with continual adversarial behavior by AI users" .

Lead author Kevin Ren, a graduate student at Cornell Tech, added that test-time adaptation could help detectors "proactively address this iterative gaming behavior by adversarial AI users" 1. Senior author Nikhil Garg is an assistant professor at Cornell Tech .

Key facts

  • A commercial detector catches only 27% of deliberately disguised AI text; a test-time adapted detector reaches 93%
  • Test-time adaptation lets the detector self-adjust after release instead of needing full retraining
  • On shifting human-language benchmarks, adapted detectors hit 85% accuracy vs. 73% for traditional methods
  • Researchers come from MIT and Cornell Tech and will present at NeurIPS
  • Technique could help educators, publishers, hiring managers and cybersecurity teams flag disguised AI writing

Sources

  1. Helping AI text detectors keep up with changing conditions

If this story concerns you or your company, or uses something that belongs to you, ask for a correction or for it to be taken down. How requests work

Keep reading