This post is going out early as a thank-you to paid subscribers. The full piece opens up to free subscribers next Tuesday. I’m grateful you’re here, and grateful for your commitment to rethinking what education can be. Your support is what keeps this work going.
Every company selling AI detection to educational institutions leads with a number. Turnitin, GPTZero, Copyleaks, and Pangram all publish one, usually an accuracy rate or a false-positive rate, often reported to two decimal places. Pangram, for example, self-reports a false-positive rate of 0.19 percent and a false-negative rate of 1.4 percent on standard datasets. A false-positive rate of 0.19 percent means that out of 1,000 human-written essays, the detector falsely flags only about two texts as AI-generated.
Sounds great, doesn’t it? But unfortunately, none of those numbers can really tell you what will happen in your classroom. Because outside of a controlled lab environment, they have very little meaning.
I have made this point on The Augmented Educator Substack for over two years now, particularly in The Paradox of AI Detection. But so far I have mostly asserted rather than explained what might be the most complex part of this argument: that the trade-off at the center of these tools is subject to a hard mathematical limit. Better engineering can shift that trade-off. But nothing can eliminate it.
Absolutely nothing. The math is baked in.
So in this post, I want to complete my core argument against AI detection. And I have decided to do it in plain language and with no complex math notation. I wanted to keep this approachable to a lay audience. If you are interested in the finer details of this argument, you might want to check out the audio deep dive podcast episode linked at the end of this post.
How a score becomes a verdict
Let’s start with what a detector actually returns in practice. Underneath the percentages and the categories, it is fundamentally a score. The detector assigns a text passage a number between zero and one, say 0.83 or 0.11, showing how strongly the system associates it with machine-written text.
Early detectors often used that score to return a simple yes/no verdict, which ended up being highly problematic because it outsourced the integrity decision from the instructor to the detector. Most of the products available today therefore stop short of that. They present a percentage or a category and leave the final judgment of what that means to whoever reads the report. This separates the detector’s result from any misconduct accusation. It is now the teacher who makes that call and not the AI detector.
Turnitin, for example, reports what share of a document it estimates was AI-written. It simultaneously cautions that the figure cannot be used as proof of misconduct. GPTZero splits a submission into percentages of AI, mixed, and human. And Pangram sorts text into human, AI-edited, and fully AI-generated, and its EditLens model estimates how much AI editing a passage received.
None of that is an accusation. Every one of those reports leaves somebody else to decide what happens next, and that decision is a cutoff.
Somewhere between an innocent detection report reading some percentage and an email accusing a student of misconduct, a line gets drawn. Statisticians call that line a threshold. Think of it as a dial which the teacher sets, and think of the number it points at as a bar the score has to clear before anyone acts on it. Turn the dial up, and fewer texts clear the bar. The same detector, on the same essay, leads to entirely different outcomes depending on what number that dial points at.
When a company says its tool is 99.85 percent accurate, it is describing the result of applying a chosen threshold to a chosen test set. Move the dial or change the test set, and the advertised accuracy and error rates move with it.
Two piles that overlap
Let’s assume we want to use a detector to score an extensive set of known examples. Take thousands of texts you know a human wrote, for example, and score them all. Then take thousands you know came from a language model and score those too. You get two piles of scores.
If those piles sat in separate ranges, say, with every human text below 0.4 and every machine text above 0.6, detection would be a solved problem. You would just have to put the dial in the empty gap between 0.4 and 0.6 and the system would never make a mistake in either direction.
