This post is going out early as a thank-you to paid subscribers. The full piece opens up to free subscribers next Tuesday. I’m grateful you’re here, and grateful for your commitment to rethinking what education can be. Your support is what keeps this work going.
A few days ago, an academic paper made the rounds that I found deeply disturbing. Not necessarily the paper itself, but rather the reactions to it. They reminded me how hard it is, even for those of us who consider ourselves critical thinkers, to recognize the unintended consequences of a technology that seems helpful at first but in reality causes more harm than good.
And I am not talking about artificial intelligence itself, although one could argue that it falls into the same category. I am talking about AI detection, and the inexplicable urge some feel to put the ability to detect AI above everything else, including humanity itself. You can be as skeptical of AI as you like, but two wrongs still don’t make a right.
I have written about AI detection many times before, and making the same scholarly argument again felt repetitive. So this week I wrote a fictional short story instead. It follows a teacher and her daughter across fifteen years, told in reverse, and it is about how any detection technology, once it is sophisticated enough, will ultimately be used for surveillance.
I should warn you that calling myself a fiction writer would be like calling myself an airline pilot because I have flown on planes. The prose is not the point. The story is. And the nods to 1984 and The Minority Report are deliberate.
The paper that set all this off is linked at the end, along with a NotebookLM audio deep dive into the research behind the story. Yes, you could scroll down right now. But that would be cheating, wouldn’t it?
October 2041
“June Ellery?”
The man at her door is about her mother’s age, in a gray jacket, a tablet in one hand. A younger woman waits a step behind him. “Sorry about the hour,” he says. “We try to catch people before they leave for the day. We’re with the Office of Cognitive Safety.”
The other student teachers call them the idea police. June has never seen one before. She lets them in because that seems like what you’re supposed to do.
“This isn’t an arrest,” the man says, reading from the tablet in an even voice. “You’re being placed on a preventive assessment hold under the early-intervention standard. Assessment usually takes two to three days. Your student-teaching placement is suspended pending review. Hillcrest Middle School has been notified.”
June looks down at the lesson plan in her hand. She was supposed to teach it at nine.
“Notified of what?”
“The suspension. Not the reason.”
“What’s the reason?”
He scrolls. “Your outline. ‘Failure Modes in Automated Integrity Systems,’ submitted September twenty-ninth.”
It takes her a moment to remember it. She wrote it two weeks ago for her seminar on assessment ethics, in Verified, the university’s writing environment. Nothing written anywhere else counts toward a grade. She hasn’t written a word of the paper yet, only the outline.
“That’s a seminar assignment,” she says. “We were asked to write about the limits of the systems we’ll be using.”
“I understand.” He turns the tablet toward her. “This is what I’m able to show you.”
DETERMINATION: Concurrence 0.97, two of three models. Basis: submitted outline; longitudinal record, 2029 to present.
PROVENANCE: Human (legacy field).
MINORITY FINDING: Academic critique; no operational intent. Recorded. Not adopted.
She points to the second line. “It says human.”
“That field is legacy. Provenance isn’t under review.”
“This one says I had no intent.”
“That’s the minority finding. It’s recorded, but the determination goes with the majority.”
“Has anyone actually read it?”
“I don’t make determinations, Ms. Ellery.” He says it gently, like someone who has said that sentence many times before.
She reads the first line again and stops at 2029. That year she was nine, in fourth grade, and she can’t remember a single thing she wrote.
She looks up at him. “What am I supposed to have done?”
“Nobody’s saying you did anything.” He lowers the tablet. “The review is about where your thinking was headed.”
“You’ll want to bring a few things,” the younger woman says. “Clothes for a couple of days, and anything you take regularly.”
June fills a tote bag and puts on the cardigan she was planning to wear today.
On the way out she passes the bookshelf by the door. Between a statistics textbook and a stack of graded paragraphs is a paperback she took from a box of her mother’s things when she was fifteen. Its spine is so creased that the title is hard to read: 1984.
“You may notify one approved contact,” the man says. “We have Dana Ellery on file.”
June doesn’t remember putting her mother on file, but there is no one else she would call anyway. Her mother picks up on the first ring.
“Mom, it’s me. I’m okay. There are two people here. They say I...” She stops and starts over, this time with the seminar assignment.
When she stops talking, June stands by the door, the tote bag at her feet, and listens to her mother say nothing for a very long time.
