How a Code Review Got Claude Fable 5 Banned
The security flaw that pulled the world's most capable AI model offline is a reminder of why we still need to teach students how to code.
On the evening of June 12, US authorities gave Anthropic ninety minutes to pull its most capable product, which it had released just three days earlier, off the internet. The company complied. By the end of the night, Claude Fable 5 had gone dark, along with its more powerful sibling, Mythos 5.
What makes this event so remarkable is the reasoning behind the deactivation. The government did not shut the model down because it wrote malware, or because someone tricked it into creating the blueprint for a weapon. It was shut down because it was too good at a task we very much want AI to do: reviewing code.
The thing it was punished for was the very thing it was built for.
The exact account is technically complex, but I think it is important for any educator to understand what happened. I therefore want to spend this essay providing a lay explanation that is accessible to non-technical readers. Because once you see the mechanism clearly, a lot of assumptions about AI safety start falling apart.
Two faces of one model
To understand the shutdown, you have to understand what these two models were. Anthropic had trained a single system, the most capable it had ever made, and then released it wearing two different faces.
Mythos 5 was the raw version of the model. It held nothing back, and because of that, Anthropic handed it to only a small, vetted group in what they call Project Glasswing. This involved approximately 150 organizations focused on tasks such as protecting vital infrastructure. The reasoning was simple. A model that can explain, in working detail, how to write attack software is extremely dangerous if it falls into the wrong hands.
Fable 5 was the public version of Mythos 5. It had the same underlying brain, but wrapped in an automatic safety layer. Simply put, it had a bouncer posted at its door.
The bouncer’s job was to read every request coming in and every answer going out, watching for three kinds of dangerous content: instructions for offensive hacking, instructions for building biological weapons, and attempts to expose the model’s hidden reasoning. When it caught one, it quietly handed the conversation off to Claude Opus 4.8, an older and tamer model, which would finish the job. This happened seamlessly. Most users would never notice the swap.
In principle, this was clever engineering. The safety layer let the company sell access to a genuinely powerful model while, in theory, keeping the most dangerous knowledge locked behind a door that only a vetted group of users could open. But the whole arrangement rested on a single assumption: that the bouncer could reliably tell a dangerous request from a harmless one.
And Fable 5 was powerful, in my own assessment too. I’ll spare you the benchmark tables, but one figure shows its capabilities. During testing, the payments company Stripe pointed Fable 5 at a fifty-million-line codebase and asked it to migrate the whole thing to a new framework. It finished in a day. Anthropic’s own estimate was that the same job would have taken a team of human engineers two months.
It is worth holding on to that number, because the ability that let Fable 5 rewrite fifty million lines of code in a single day is also the ability that got it banned.
The most ordinary request in the world
Here is where the trouble started. One of the most legitimate, everyday things you can ask a coding model to do is look over your code for mistakes. “Review this for security issues and edge cases” is a sentence typed thousands of times a day by developers. It is the bread and butter of the job.
So when a team of security researchers at Amazon uploaded large piles of software to Fable 5 and asked exactly that — review the code for security problems — the bouncer waved them through. Of course it did. Nothing about the request looked suspicious. It looked like a developer doing their due diligence, which was precisely what the model was supposed to be good at.
Then Fable 5 did the work, and it did it far too well.
How a list of small problems becomes a break-in
To explain what happened next, let me switch to an analogy.
Suppose you hire someone to walk through your home and tell you how secure it is. A good inspector notices small things. The back window has a latch that doesn’t quite catch. The motion-sensor light over the side gate has a blind spot. When a door or window opens, the alarm waits forty seconds before it sounds, so the family has time to punch in their code. And the code itself is on a sticky note stuck to the fridge.
Each of these is relatively minor. None of them, by itself, gets a burglar into your house and back out again.
Now imagine the inspector doesn’t simply list those four things but connects them: “Come up to the side gate through the blind spot, where the light never catches you. The loose latch on the back window opens in a few seconds. The alarm begins its forty-second countdown — and there’s the code, in the family’s own handwriting, stuck to the fridge a few steps away. Punch it in and the house goes quiet.”
Four trivial observations have just become a single working plan for a robbery. Nothing new was discovered. The flaws were already there. What changed is that someone strung them into a sequence.
Criminals call this casing a house. In software it has a different name: vulnerability chaining.
That is exactly what Fable 5 did with the code Amazon gave it. It found small, individually harmless weaknesses — an old flaw buried in a borrowed software library here, a sloppy permission setting there — and it worked out how to link them into a chain that ended in what’s called remote code execution. This is the digital equivalent of having the keys to the building.
Earlier AI models couldn’t really do this. Holding tens of thousands of lines of code in mind at once, tracking how a weakness in one corner connects to a weakness in a distant corner, and planning several moves ahead requires sustained, wide-angle attention that older models simply lacked.
Fable 5 had that capability because Anthropic had built the model precisely to keep enormous codebases coherent in its memory. That was the entire selling point. It was the very thing that let it rework a massive codebase in a day. Pointed at security, that same capability let it case a building the size of a city.
We have to take note of what Fable’s safety bouncer was up against. There is no clean line between checking the house for weaknesses and planning to burgle it, because they are the same activity carried out with two different intentions. The inspector’s report and the burglar’s plan are identical. The only difference is who’s holding the paper.
When Amazon asked Fable 5 to review code for security problems, the honest version of that task and the malicious version produced the very same document. The bouncer could not block the dangerous one without blocking the useful one, because, on the page, they looked the same.
Why you can’t safeguard your way out
You might think the fix is obvious: train the model to refuse anything that looks like a security analysis. But this turns out to make matters worse.
If you forbid a coding model from understanding vulnerabilities at all, it goes blind to them. It will cheerfully write, approve, and ship code riddled with security holes, because you’ve trained it not to see the very thing you need it to catch. You haven’t removed the danger. You’ve just made sure the model ignores it.
The other option is to let the model find the flaws but forbid it from explaining them. But this creates a different problem. Unable to talk about the weakness, the model may simply patch the code on its own and say nothing. That sounds fine until you remember that every change to software is recorded, permanently, in its version history.
A silent patch is a flashing arrow. Any competent attacker can read that history, see exactly what the model quietly fixed, and now knows precisely where the weakness was — including in every copy of the software that hasn’t been updated yet. The effort to hide the problem hands out a map to it.
So the model that’s allowed to reason about security becomes a weapon, and the model that isn’t becomes a liability. There is no setting on the dial that makes the trouble disappear.
This is the uncomfortable thing the Fable 5 incident exposed, and it’s why we should not interpret the episode as a story about one company’s blunder. Instead, it revealed a fundamental problem that is inherent to these tools.
How a code review became an export-control case
What escalated this from a noteworthy research result to a national emergency was who made the discovery. The researchers were at Amazon, Anthropic’s largest investor and the operator of the servers Fable 5 ran on. And rather than reporting it quietly, Amazon’s chief executive reportedly carried the finding straight to senior officials in Washington.
In response, the government turned to a tool that had never been used in this manner before: export controls, the set of laws regulating the transfer of sensitive technology across borders. The argument was that letting foreign nationals anywhere in the world send prompts to Fable 5 amounted to exporting a cyber-weapon.
Within ninety minutes of receiving the order, Anthropic had to shut the model down for everyone because there was no way to verify the citizenship of hundreds of millions of users in real time. A blunt order met a system with no fine-grained off switch, so the only move left was to pull the plug entirely.
Anthropic has since pushed back, and as I write this, the last word has not been spoken. The company argues that the weaknesses Amazon chained together were already known and fairly minor, and that no model on the market can be made perfectly resistant to this kind of manipulation.
I think they have a real point, but I also think that point doesn’t make the underlying problem go away.
Why this is good news for anyone who can read code
I want to end somewhere more optimistic, because I think this episode clarifies something very hopeful for those of us who teach.
For a couple of years now, some of the loudest industry voices have suggested that learning to code by hand is becoming an outdated exercise, like doing arithmetic without a calculator. Why teach students to write and read code carefully when a model can produce fifty million lines in an afternoon?
The Fable 5 incident offers a persuasive answer.
The danger was not contained by a cleverer safety filter; the filter failed. What caught the problem was people. Amazon’s human researchers understood code well enough to recognize that the model’s tidy security review was, read another way, an attack plan.
And the question Anthropic and the government spent the following days arguing about — were these flaws serious or trivial, a real weapon or a lab curiosity — is one that only people who deeply understand software can ever evaluate.
This is the part the “AI will handle the coding for us” story keeps missing. A tool that can find every weakness in a system is only safe in the hands of someone who could have found those weaknesses themselves. It needs someone who can read the model’s output and tell a fix from a trap.
Take that person away and you don’t have automation. You have a machine writing code no one in the room can check.
So I’ll keep teaching the fundamentals, and I’d urge every educator at every level to do the same. The machines are extraordinary; that was never the issue. But we need to keep teaching people how to read the blueprint because the day no one can read it is the day we’ve handed the keys to whoever asks.
The images in this article were generated with Nano Banana 2.
P.S. I believe transparency builds the trust that AI detection systems fail to enforce. That’s why I’ve published an ethics and AI disclosure statement, which outlines how I integrate AI tools into my intellectual work.