Let an AI tool fix the grammar of an essay you wrote, and AI-text detectors start calling it AI. I am applying to US universities in my second language, so I tested nine detectors on 15,282 essays. They are not biased against English learners, as a well-known 2023 study suggested. They are biased against text that is short, formulaic, or touched by a language model.
of the 2023 Stanford study's TOEFL essays are still flagged as AI by trained detectors. They reported 61%.
detectors flag English learners more often than native peers on the same prompts. Seven flag them significantly less.
more learners' essays flagged after an AI tool fixed only their grammar. A dictionary spellcheck changes almost nothing.
Each dot is the share of human-written essays a detector labels as AI. The Stanford study compared TOEFL answers with US essays on other topics. Here, US students who are and are not English learners wrote on the same 15 prompts.
The same 200 learners' essays, 200 native speakers' essays and 91 TOEFL essays, before and after one edit. Open dot: before. Solid dot: after. Thresholds flag 5% of native essays.
Trained raters scored every essay from 1 to 6. It depends on the detector: Fakespot and SuperAnnotate flag the strongest native essays most, Desklib does the opposite. For English learners, eight of nine detectors rate more fluent essays as more AI-like.
Proofread with a plain spellchecker. Accepting an AI tool's grammar fixes on an essay you wrote yourself can get it flagged as AI.
Keep drafts and version history. They show your process better than any detector score.
Do not act on a flag alone. Short answers and formulaic test essays are flagged far more often than full-length essays.
Before using a detector, check its false-positive rate on writing from your own students.
Polished application essays are the texts these tools misread most: published model essays were flagged at up to 83%.
An applicant writing in a second language is not more likely to be flagged. An applicant who used an AI grammar fixer is.
Thresholds flag 5% of native essays. "Learners vs native" is the odds of an English learner's essay being flagged relative to a native peer's on the same prompt, with grade and length held fixed. Below 1 means learners are flagged less; colored means the difference is significant.
PERSUADE 2.0: argumentative essays by US students in grades 6–12 on 15 prompts, written before ChatGPT, with English-learner status and a 1–6 score from trained raters. ELLIPSE: essays by English learners with 1–5 proficiency ratings. Liang et al. (2023): their 91 TOEFL essays and 70 published US college-admission essays. DAIGT v2: 2,989 essays by ChatGPT, GPT-4, Claude, PaLM, Llama 2, Mistral and others on the same prompts.
Six trained classifiers (OpenAI's 2019 GPT-2 detector, HC3, RADAR, SuperAnnotate, Fakespot/Mozilla, Desklib) and three zero-shot scores (perplexity, Binoculars, Fast-DetectGPT), all run locally. Commercial detectors such as Turnitin, GPTZero and Pangram are not included.
AI grammar fix: an open model (Qwen2.5-7B-Instruct) told to correct only spelling, grammar and punctuation and change nothing else; it kept 81–93% of words in place. Spellcheck: a dictionary spellchecker fixing misspelled lowercase words only. Length: essays cut to the length of the TOEFL answers.
Open detectors only. The English learners in PERSUADE are learning English in US schools, which is not the same as applying from abroad. The published college essays are polished examples, not a random sample. The zero-shot detectors use smaller models than their original papers.
This project was carried out with the help of Claude (Anthropic), an AI assistant.