Guide

How AI detectors work, and why they get it wrong

By Aditya Sikarwar and Sanidhya Dhangar · ResearchGuru.AI ·

An AI text detector does not see how a text was written. It reads the finished words and estimates how closely they resemble the output of a language model. The methods differ, but each turns statistical patterns into a score, and each has weaknesses that matter when that score is used to judge a person's work.

What a detector actually measures

A language model writes by repeatedly choosing a likely next word, or token. Its output therefore tends to follow paths that the model itself finds probable. Detectors look for traces of that process: wording a model finds unsurprising, a rhythm that varies little, or patterns learned from labelled examples of human and machine text.

The result is a probability-like estimate for a text, sometimes with highlighted sentences. It is not a count of how many words a model produced. It cannot tell a model's draft that a person rewrote from a human draft that a grammar tool polished, and it knows nothing about the drafts, notes and sources behind the text.

Signal 1: how predictable the words are

Perplexity measures how surprised a language model is by a text. The model scores each token by the probability it would have given it; text made of high-probability choices has low perplexity. Generated text often scores low because sampling favours likely words.

Burstiness describes how much that predictability varies from sentence to sentence. Human writing tends to mix plain sentences with unusual ones, while generated text can be evenly smooth. Several commercial detectors describe their scores in these terms.

The weakness is that careful academic prose is predictable too. Standard method descriptions, defined terms and cautious phrasing lower perplexity in text that no model wrote.

Signal 2: classifiers trained on examples

Many current detectors are supervised classifiers: a transformer model, often from the BERT family, fine-tuned on large sets of human-written and machine-generated passages until it can separate them. The classifier learns whatever differs between the two sets, from vocabulary and phrasing to punctuation habits.

A classifier is only as good as its training data. Text from a newer model, another language, an unusual discipline or an unusual length can fall outside what it has seen, and its accuracy on such text is unknown until someone tests it. A published accuracy figure describes the test set on which it was measured.

Signal 3: zero-shot statistical tests

Zero-shot methods need no training on labelled examples. DetectGPT (Mitchell et al., 2023) makes small rewrites of a passage and compares a model's probability for the original with its probability for the rewrites. Generated text tends to sit near a local peak of the model's probability, so the rewrites score clearly lower; human text shows a smaller drop.

Binoculars (Hans et al., 2024) contrasts how surprising a text is to one language model with how surprising that model finds a second, closely related model's predictions for the same text. The ratio separates generated from human text without task-specific training. Both methods need suitable language models and work best when those models resemble the one that produced the text.

Signal 4: watermarks

A watermark is added while text is generated rather than inferred afterwards. In the scheme proposed by Kirchenbauer et al. (2023), each preceding token determines a green list covering part of the vocabulary, and generation slightly favours green tokens. A detector that knows the rule counts green tokens and tests whether there are more than chance would produce.

Google DeepMind described its SynthID Text watermark in Nature in 2024 and reported its use in Gemini (Dathathri et al., 2024). A watermark identifies only text from a generator that applied it, and heavy rewriting weakens it, so the absence of a watermark says nothing about whether another tool was used.

Why detectors misjudge real writing

  • Non-native English writing: Liang et al. (2023) found that several detectors classified more than half of a set of TOEFL essays by non-native speakers as AI-generated, while essays by US school students were classified almost entirely correctly.
  • Short passages: a paragraph offers little statistical evidence, and a few edits can move it from one class to the other.
  • Paraphrasing and editing: Krishna et al. (2023) showed that passing generated text through a paraphrasing model sharply reduced detection rates. Sadasivan et al. (2023) argued that detection grows harder in principle as model output comes closer to human writing.
  • Mixed authorship: a human draft polished by a tool, or a generated draft rewritten by a person, falls between the classes a detector was trained to separate.
  • Newer models: a detector tuned on older model output can miss text from newer systems or misread their style.

OpenAI withdrew its own AI text classifier in July 2023 because of its low rate of accuracy. Its earlier evaluation had found that the classifier labelled 26% of AI-written text as likely AI-written, while it mislabelled 9% of human-written text as AI-written.

Reading a detector result responsibly

  1. Treat the score as one signal about a text, not a finding about a person.
  2. Look at which sentences drove the score. Formulaic method or definition text often explains a high reading.
  3. Prefer whole sections to single paragraphs, and compare like with like, such as the same writer and the same kind of assignment.
  4. Give drafts, notes, version history and a conversation about the work more weight than any score.
  5. Follow the applicable institutional or journal policy, which may permit disclosed AI assistance.

ResearchGuru's AI content detector reports a model-based score with a per-sentence view of what drove it. Its method and limitations set out what the score can and cannot show, and are AI detectors accurate? covers false positives and what to do if your own work is flagged.

Sources

  1. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature — Mitchell et al., ICML 2023 (arXiv:2301.11305)
  2. Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text — Hans et al., 2024 (arXiv:2401.12070)
  3. A Watermark for Large Language Models — Kirchenbauer et al., ICML 2023 (arXiv:2301.10226)
  4. Scalable watermarking for identifying large language model outputs — Dathathri et al., Nature, 2024
  5. GPT detectors are biased against non-native English writers — Liang et al., Patterns, 2023 (arXiv:2304.02819)
  6. Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense — Krishna et al., NeurIPS 2023 (arXiv:2303.13408)
  7. Can AI-Generated Text be Reliably Detected? — Sadasivan et al., 2023 (arXiv:2303.11156)
  8. New AI classifier for indicating AI-written text (withdrawn in July 2023 for low accuracy) — OpenAI
Questions

Frequently asked questions

How do AI detectors detect AI writing?

They estimate how closely a text matches patterns typical of language-model output, using predictability scores such as perplexity, classifiers trained on human and AI examples, zero-shot statistical tests, or, for some generators, a watermark.

Can AI detectors be wrong?

Yes. They can flag human writing, especially formal, short or non-native English text, and miss generated text that has been paraphrased or edited. A score needs human review and other evidence before any decision.

What is perplexity in AI detection?

Perplexity measures how surprised a language model is by a text. Low perplexity means the words were predictable to the model, which is common in generated text but also in formulaic human writing.

Do AI detectors work on paraphrased text?

Often poorly. Research has shown that paraphrasing generated text can sharply lower detection rates, and edited or mixed text is among the hardest cases for any detector.

Understand the next step

Get conference deadlines and research tips by email

Once a month: upcoming paper deadlines, new guides and one practical tip. Confirm by email; unsubscribe any time.

We only email you after you click the confirmation link. Privacy policy

Tools for this

Do it faster