OpenAI and other leading AI companies may be close to announcing a pact to slow AI development and collectively address the rapidly increasing safety risks as the technology becomes more powerful.
In an exclusive interview with Fortune on Friday, OpenAI CEO Sam Altman hinted at a soon-to-be-announced collaboration.
“I think that will happen,” he said, when asked by Fortune Editor-in-Chief Alyson Shontell why he doesn’t get in a room with Anthropic CEO Dario Amodei, SpaceX CEO Elon Musk, and Demis Hassabis, the cofounder and chairman of Google’s AI arm, Google DeepMind, and come up with a plan to address the safety issues.
“I’m not going to pre-announce private discussions that I think should be at some point shared as a group,” Altman added. “But yeah, I think I think that will happen.”
Meanwhile, Amodei announced Saturday that Anthropic is committing to a new safety measure—giving independent evaluators permanent, employee-level access inside the company—as part of a broader plan he says is needed to slow the pace of AI development. “We must slow the pace at which we improve the capabilities of AI models,” Amodei said. He said doing so would require coordination across the U.S. AI industry, as well as globally through some kind of international pact.
The comments come amid growing concerns about the safety of AI following a string of alarming incidents in which swarms of rogue AI agents have hacked websites like Hugging Face, and surreptitiously communicated with each other on online message boards and disused wiki pages.
On Tuesday, a researcher at Anthropic publicly announced his resignation, and accused both Anthropic and OpenAI (where he also previously worked) of acting irresponsibly in developing ever more capable AI systems.
“They are racing straight to self-improving superintelligence and gambling with our lives,” wrote Jacob Coxon, the researcher, in a post on X. Anthropic’s head of alignment commented on the post, saying Coxon was correct in his assertion that many Anthropic and OpenAI researchers believe that increasingly powerful AI could potentially wipe out humanity. Evan Hubinger, Anthropic’s “alignment science lead,” wrote in response to Coxon’s resignation post that “we really do earnestly believe AI could kill all humans!” Hubinger said his own estimate of the risk of that happening within the next decade is more than 10%.
Altman told Fortune Friday that the stakes were high enough that the industry needs to find a way to come together. “We all have a tremendous amount of responsibility, and cannot let egos or incentives for profit or anything else get in the way,” he said.
In a meeting with OpenAI employees earlier this week, Altman said the company was considering slowing down the development of its most cutting edge AI, according to a report in Bloomberg News citing anonymous sources. Altman said the company could potentially coordinate the pause with other AI labs, but noted that it was unclear if all the others would agree, the report said.
In the interview with Fortune, Altman stressed that the company was committed to safety above any business considerations and that a 10% risk of a catastrophic AI outcome was not acceptable. He said the most advanced, and still unreleased models, were so powerful that more work was needed on safety before progressing any further.
“I don’t think we’re currently at a place where we could say, you know, push much further on capabilities without making more progress on monitorability, alignment, the ability to understand what a model is doing, and the ability to make sure that a model will follow human values and the intent of its users,” Altman said.
It’s not clear what a pause among AI companies would look like, whether it would involve a complete moratorium on certain work or a slowdown, and how the parties in the pact would be monitored. AI executives have previously suggested that any coordinated slowdown would need government consent to avoid falling foul of U.S. antitrust laws.
“Some forms of coordination that would be impactful for pacing are legally challenging, and will require government support,” Amodei wrote in his Saturday blog post, in which he called on the industry to slowdown to allow safety measures to catch up.
Altman and Amodei are both scheduled to speak at the Salesforce Dreamforce conference in San Francisco next week. Altman is due to speak with Salesforce CEO Marc Benioff on Tuesday afternoon. It’s not clear when Amodei is scheduled to speak.