from the seems-bad dept
Late Friday, Anthropic shut down access to its just-released Fable 5 and Mythos 5 models after the Trump administration slapped export controls on them — treating cutting-edge AI, in other words, like weapons. The trigger, it turns out, was a jailbreak. And the entity that tipped off the government? Amazon — one of Anthropic’s biggest investors.
Considering how much Trump-supporting VC bros in Silicon Valley insisted that the Biden admin wanted to shut down powerful AI models during the last administration, it’s quite something to see them cheering on the Trump admin actually doing exactly that.
As you’ll recall, a couple months ago, Anthropic talked about its “Mythos-class” LLM models with (depending on your perspective) the greatest marketing hype ever or an appropriate level of caution for the risks with the model (more likely: somewhere in between). When they first talked about it, they said that it was quite good at finding cybersecurity vulnerabilities, and so initially it was only available to a set group of organizations that might find it useful to patch certain holes. From what I’ve heard from people in the industry, the tool is good and useful, but it’s not magical.
Then, a little over a week ago, they rolled out the latest version of Mythos, which was still limited to pre-vetted companies, but then they offered up “Fable 5” as a tool for anyone else. This was described as “Mythos-class” but with extra guardrails, including that if it thought you might do something bad with Fable, it would drop you down to its previous best-in-class Opus 4.8 model. Fable was also twice as expensive on a per-token basis, but apparently much more efficient, so the actual pricing difference was likely less big. And some of the early tests with Fable 5 showed it to be way more impressive at certain coding tasks. There were also some oddities, like Fable only being available in the commercial subscription plans for a couple weeks before switching over to only (way more expensive) API usage.
Still, there were some concerns about the guardrails, and how frequently they were kicking people out to Opus on perfectly normal queries. There were other concerns about its changed data retention policies for large enterprises. Previously, companies could negotiate a zero retention policy with Anthropic and guarantee that no data was being held by the company. But with the latest models, they required you to let them hold onto any data shared with the models for 30 days. Anthropic insisted this was solely for safety reviews, in case something went wrong, they could track down the reasons why, but it scared away some large enterprises that could risk their own data or source code being retained anywhere else.
Either way, all that went silent late on Friday (amusingly, in the middle of me messing around with Fable) when Anthropic announced that the US government had made them shut down access to the models with zero due process. Technically, the US government claimed that for “national security” reasons, no foreign national could be allowed to have access to the models (including Anthropic’s own foreign national employees), and since Anthropic doesn’t know which of its customers are foreign nationals, they had to shut down all access.
There are a number of different threads to pull on from previous events that are all worth mentioning here as useful background:
- The US government’s plan to ban TikTok by just screaming “national security.” Many of us had called out how problematic that was, but the Supreme Court basically told the US government “all you have to do is say ‘national security’ and you can ban any tech you want” so here we are. What the Supreme Court gifted the US government, the Trump administration has no problem abusing.
- Remember, many of the most powerful people in Silicon Valley had lined up behind Donald Trump, in part because of this very mild executive order on AI technology from the Biden admin that never, ever got remotely close to the level of banning an entire model by screaming national security. Some are vocally defending Trump for doing the very thing they screamed would destroy American innovation if Biden did it (even though he showed no sign that he would). Others are conspicuously quiet. AI’s got your tongue?
- Just a few weeks ago, the Trump administration released its own AI executive order that was effectively the same plan Biden had released that drove Silicon Valley VCs crazy, except this plan was less well-thought out and more confusing. But, still, even that plan didn’t include “banning models for national security.”
- Of course, there is also the ongoing battle between Anthropic and the Trump administration, all because Anthropic wanted to keep some specific terms of use in their contract with the Department of Defense to try to limit a few egregious use cases. The entire Trump admin lost their minds over this, because Pete Hegseth can’t take someone saying no to him.
- And then there’s also Anthropic’s tightrope walking of asking the US government to build them a regulatory moat. Just days before this came down, Dario Amodei had penned a blog post (or was it Claude) laying out a roadmap for how he wanted Trump to regulate Claude. Be careful what you ask for, Dario.
So all of those things came together to lead to this effective ban.
Soon after it was announced, it was revealed that Amazon (one of Anthropic’s biggest investors) had actually alerted the US government to the supposed “bug” that gave the administration the ammo it needed to shut down the model.
Anthropic said it thinks the government became aware of a method of so-called jailbreaking before Friday’s action. “We reviewed a demonstration of this specific technique being used to identify a small number of previously known, minor vulnerabilities. These vulnerabilities all appear relatively simple, and we have found that other publicly available models are able to discover them as well without requiring a bypass,” the company said.
The jailbreak research in question was done by researchers at Amazon, who used a series of prompts to get Anthropic’s model to provide them with information about a handful of security vulnerabilities, said Katie Moussouris, chief executive with the cybersecurity firm Luta Security. Anthropic shared a copy of the report with her, she said.
Now, if you’re thinking “a jailbreak sounds dangerous for this tech” then, sure… except that the reporting says the jailbreak was useful in a different way:
But the information provided by the model in this report would be of more use to people defending computer networks than to those attacking them, she said.
“Who at the White House evaluated this and thought it was a threat?” she said. “It’s a complete overreaction because this is exactly the kind of prompting that defenders would do.”
That almost makes it sound like somebody (NSA?) didn’t want people using this to protect themselves — rather than being worried about malicious uses. It sure wouldn’t be the first time the NSA compromised everyone’s security to make sure they could keep spying on people.
None of this is good or reasonable tech policy — or industrial policy, or any other kind of policy. It’s all just power-seeking Calvinball. Apparently the US government can just scream “national security” with no evidence or explanation and shut down an entire model. That’s ripe for abuse — especially with this administration.
When I wrote recently about how authoritarians seek to grab control over centralized technology choke points, this is the kind of thing I was thinking of, though I didn’t expect them to be so ham-fisted about it.
It’s tempting to read this purely as retaliation by the Trump admin against Anthropic, a company they’re already mad at and already illegally trying to punish. But all of these other issues play into this as well, including Anthropic’s constant refrain of “we’re so dangerous, please regulate us.”
You kept asking for it. Now you’ve got it.
And where are all those Silicon Valley VCs who insisted everyone had to back Trump because Biden was going to seize and shut down LLMs? I looked on X at the feeds of the various of Trump’s biggest supporters who had talked shit about Biden shutting down AI innovation and… of course they’re still supporting Trump. David Sacks came out with a long tweet saying that the administration was totally justified in shutting down Fable because of “safety” saying that Anthropic had “prioritized the continued offering of the consumer model over safety.”
Can you imagine how Sacks would have responded if the Biden admin had demanded an AI company shut down a model because of “safety?” Oh, you don’t have to imagine, because he was pretty clear about how he felt about the Biden EO. He claimed it “hamstrung American AI companies” even though nothing in the Biden admin plans would have ever gotten so far as what the Trump admin did on Friday, shutting down an entire model. All it did was ask companies to voluntarily pre-submit frontier models for an analysis by experts who might make some suggestions on how to keep them secure.
And that was so horrific it was worth effectively blowing up the American democratic order. Yet now Trump goes way further in literally shutting down an LLM and Sacks says it’s all good because it’s for “safety.”
These are not serious people. This is not a serious administration.
They are just power hungry jackasses with poor impulse control.
Here’s what we know: the jailbreak was defensive in nature, according to the cybersecurity expert who reviewed the actual report. Also, the administration offered no public evidence, no due process, and no coherent explanation for why this particular jailbreak required shutting down access for everyone, including Anthropic’s own employees. We also know that this administration pulls out “national security” claims quite frequently that later turn out to be bogus, and thus we shouldn’t trust them without more evidence.
Maybe there’s classified information that changes the picture. But this administration has burned any benefit of the doubt it might have had. What we’re left with is a government that learned it can yell “national security” and make technology disappear — and a roster of Silicon Valley allies who spent years screaming about regulatory overreach from the last administration have suddenly found a new song to sing.
Filed Under: ai ban, claude, dario amodei, donald trump, due process, export controls, fable 5, mythos, national security, trump administration
Companies: anthropic
You must be logged in to post a comment Login