Research that caused the Commerce Department to impose strict limits on the use of Anthropic’s new AI models wasn’t geared toward offensive purposes, according to a cybersecurity CEO who saw the findings.
Late Friday, the department used national security export controls to bar the company from distributing Fable 5 and Mythos 5.
The directive applies to people outside the U.S. and foreign nationals in the U.S., including Anthropic’s own non-citizen employees. Due to the directive’s scope, Anthropic said it had no choice but to disable the models for all users.
The company said it was told that research on a “jailbreak” of Anthropic’s AI that sought to probe bypassing of safeguards sparked the export controls.
“We disagree that the finding of a narrow potential jailbreak should be cause for recalling a commercial model deployed to hundreds of millions of people,” Anthropic wrote in a blog post. “If this standard was applied across the industry, we believe it would essentially halt all new model deployments for all frontier model providers.”
While the company reaffirmed the government’s ability to block unsafe AI, it argued that should be part of a statutory process that’s transparent fair, and based on technical facts. “This action does not adhere to those principles.”
Katie Moussouris, CEO of cybersecurity firm Luta Security, told the Wall Street Journal that Anthropic showed her a copy of the findings, which were produced by Amazon researchers using prompts to obtain information about security vulnerabilities.
“I’ve seen the paper. It’s not a jailbreak. It was Defense Oriented Prompting (DOP), capabilities defenders need,” she explained in a post on X on Saturday.
Moussouris added, “If Nat defense is the goal, this just scored an own goal against us.”
The Commerce Department didn’t immediately respond to requests for comment.
“As a leading cloud provider that serves a large number of private and public sector customers, it’s not uncommon for governments to seek our counsel on potential security risks. When they occur, we don’t share the details of these discussions,” an Amazon spokesperson told Fortune.
Meanwhile, the administration’s directive barring foreign nationals in the U.S. from using Anthropic’s new models raised alarms.
Ben Murphy, a scholar at the Institute for Progress think tank, called it “another step on the balkanization of technology.”
“It might previously have been unthinkable to require proof-of-citizenship to access services; it’s increasingly common across new technologies and, to be honest, this attempt is not surprising in that light,” he added in a post on X.
Murphy also highlighted the unpredictability of the administration’s actions and its consequence for AI developers, warning that labs could keep more models in-house or not make them available.
In addition, labs might be less inclined toward engaging with the government about potential vulnerabilities, he said, with Anthropic’s stance on being transparent seeming to backfire.
Anthropic was already feuding with the administration, which has deemed it a supply-chain risk for Pentagon contractors. Still, the company provided early access to the Mythos model as it warned on its potential cybersecurity implications.
“I don’t know that the government wouldn’t have reached that conclusion themselves, but as a business matter, those pronouncements have not produced a healthy working relationship with the government,” Murphy wrote.