Anthropic, the fast-growing AI company currently valued at $60 billion, has quietly hired the husband of cofounder and president Daniela Amodei to help craft safety strategy, adding another family connection to the startup helmed by brother and sister Daniela and CEO Dario Amodei.
Holden Karnofksy, Daniela Amodei’s husband, joined Anthropic last month as a member of the startup’s “technical staff,” a company spokesperson confirmed. The company had not announced Karnofsky’s hiring, but Karnofsky recently updated his LinkedIn profile to reflect his new position.
Karnofsky is an AI safety researcher and philanthropist closely associated with Effective Altruism, a philanthropical movement whose focuses include AI’s existential risks, and which has garnered both influence and controversy in recent years. Karnofsky also once served on OpenAI’s board.
Karnofsky is working on Anthropic’s “Responsible Scaling Policy,” which is the company’s set of protocols to manage public and societal risk around powerful AI models, according to two people with direct knowledge of the matter and confirmed by Anthropic. Both sources, whose identity is known to Fortune, were granted anonymity because they were not authorized to discuss company matters.
Anthropic confirmed that Karnofsky is working on trust and safety issues, describing his title only as “a member of the technical staff” reporting to Chief Science Officer Jared Kaplan.
“He is working on the responsible scaling policy and other aspects around safety planning ahead of highly advanced AI systems,” the spokesperson said.
With Karnofsky’s addition to the company, Anthropic now has three close family members holding influential roles at the company. Siblings Dario and Daniela Amodei were part of a group of early OpenAI employees who left the company in late 2020 to found Anthropic, in part because they were concerned that OpenAI under the leadership of CEO Sam Altman was becoming too focused on commercial products while deemphasizing AI safety research.
Anthropic also recently hired Niki Parmar, the sole female member of the Google AI research team that in 2017 developed the Transformer, a kind of neural network design on which all of today’s generative AI models are based.
Parmar had previously cofounded Adept AI, a startup working on AI agents for work, but left before a majority of that company’s team was hired by Amazon last year. She went on to cofound another AI startup, Essential AI, with Ashish Vaswani, who was one of her coauthors on the landmark Transformer research. Vaswani is still CEO of the AI model and product startup – which emerged from stealth in December 2023 with close to $65 million in funding from investors including Google, Nvidia and AMD.
The new hires come around the same time as Anthropic’s new $2 billion funding round, which valued the company at $60 billion in January. The generative AI industry has become exceedingly competitive as rival OpenAI launches a $500 billion initiative, known as the Stargate project, to build AI data centers around the United States, and as other rivals like Elon Musk’s xAI raise billions in capital to train their own models.
Anthropic is under pressure to move quickly and continue to improve its own large language model, Claude, as quickly as possible. The company has yet to release a “reasoning model” similar to OpenAI’s o1 and o3 models, and Chinese AI company DeepSeek’s R1 model, that use a step-by-step “thinking” process to produce more accurate answers for math, coding, and logic questions.
Karnofsky, who served on OpenAI’s board between 2017 and 2021, co-founded Open Philanthropy with Facebook co-founder Dustin Moskowitz and his wife Cari Tuna in 2017. Open Philanthropy is one of the largest funders associated with the Effective Altruism (EA) movement, which came under scrutiny after the collapse of Sam Bankman-Fried’s cryptocurrency exchange FTX.
The movement is known for its focus on existential risks, including the danger that powerful AI could one day wipe out humanity, and Bankman-Fried backed a number of efforts aimed at trying to mitigate this danger. FTX had invested $500 million into Anthropic, and its stake in the AI company was sold off last year for $884 million, with the proceeds going to repaying FTX customers who lost money in the exchange’s collapse.
Long before officially joining Anthropic, Karnofsky and the Amodei siblings already had close connections. Karnofsky and Dario Amodei were former roommates, and while Karnofsky and Daniela Amodei were engaged, the couple lived with Dario Amodei.
Clarification: This story has updated to note that “member of technical staff” is Karnofsky’s actual title, according to Anthropic.
