- Mark Zuckerberg has laid out his vision for “personal superintelligence” in a new blog post. In it, he acknowledged that the company may need to be “careful about what we choose to open-source” to mitigate the risks of advanced AI. The shift suggests Meta may be preparing to scale back its open-source approach as the company moves closer to “superintelligence,” a hypothetical form of artificial intelligence that surpasses human intelligence across all domains.
Mark Zuckerberg has published his AI manifesto, making a case for a type of “personal superintelligence” that people can use to achieve their individual goals.
In a new blog post, the Meta CEO said he wanted to build a personalized AI that helps you “achieve your goals, create what you want to see in the world, be a better friend, and grow to become the person that you aspire to be.” However, the company’s new aims come with a caveat: This powerful AI may soon be too powerful to be left open to the world.
“We believe the benefits of superintelligence should be shared with the world as broadly as possible. That said, superintelligence will raise novel safety concerns,” Zuckerberg wrote. “We’ll need to be rigorous about mitigating these risks and careful about what we choose to open-source. Still, we believe that building a free society requires that we aim to empower people as much as possible.”
Among those risks: That AI could become “a force focused on replacing large swaths of society,” he wrote.
Zuckerberg has traditionally positioned Meta as a proponent of open-source AI, especially compared with rivals like OpenAI and Google. While many argue the company’s Llama models don’t meet the strict definition of “open-source,” the company has leaned more toward open-sourcing its frontier models than most of its Big Tech peers.
In a blog post last year, Zuckerberg made an impassioned case for open-source, heralding Meta as taking the “next steps towards open-source AI becoming the industry standard.”
“I believe that open-source is necessary for a positive AI future,” Zuckerberg wrote last year. “Open-source will ensure that more people around the world have access to the benefits and opportunities of AI, that power isn’t concentrated in the hands of a small number of companies, and that the technology can be deployed more evenly and safely across society.”
The CEO has left himself some wiggle room, though, saying on a podcast last year that if there was a significant change in AI capabilities, it may not be safe to “open-source” it.
Closed vs. open-source AI
AI safety experts have long debated whether open- or closed-source models are more responsible in advanced AI development. Some argue that open-sourcing AI models democratizes access, accelerates innovation, and allows for broader scrutiny to improve safety and reliability. But others say that releasing powerful AI models openly could increase the risk of misuse by bad actors, including for misinformation, cyberattacks, or biological threats.
There’s a commercial argument against open-source as well, which is why most leading AI labs keep their models private. Open-sourcing powerful AI models can erode a company’s competitive edge by allowing rivals to copy, fine-tune, or commoditize its core technology.
Meta is in a different position here than some of its rivals, as Zuckerberg said last year that Meta’s business isn’t reliant on selling access to AI models. “Releasing Llama doesn’t undercut our revenue, sustainability, or ability to invest in research like it does for closed providers,” he said.
Representatives for Meta did not immediately respond to a request for comment from Fortune, made outside normal working hours.
