Elon Musk and Apple cofounder Steve Wozniak among over 1,100 who sign open letter calling for 6-month ban on creating powerful A.I.

Runaway A.I.

Musk has previously been outspoken about his concerns about runaway A.I. and the threat it may pose to humanity. He was an original cofounder of OpenAI, establishing it as a nonprofit research lab in 2015, and served as its biggest initial donor. In 2018, he broke with the company and left its board. More recently, he has been critical of the company’s decision to launch a for-profit arm and accept billions of dollars in investment from Microsoft.

OpenAI is now among the most prominent companies developing large foundation models, mostly trained on massive amounts of text, images, and videos culled from the internet. These models can perform many different tasks without specific training. Versions of these models power ChatGPT as well as Microsoft’s Bing chat feature and Google’s Bard.

It is the potential of these systems to do many different tasks—many once thought to be the sole province of highly trained people, such as coding or drafting legal documents or and analyzing data—that has made many afraid about the potential for job losses from the deployment of such systems in business. Others fear that such systems are a step on the path towards A.I. that might exceed human intelligence, with potentially dire consequences.

‘Human-competitive’

The letter says that with A.I. systems such as GPT-4 now “becoming human-competitive at general tasks,” there were concerns about risks from such systems being used to generate misinformation on a massive scale as well as about mass automation of jobs. The letter also raises the prospects of these systems being on the path to superintelligence that could pose a grave risk to all human civilization. It says that decisions about A.I. “must not be delegated to unelected tech leaders” and that more powerful A.I. systems should only “be developed once we are confident that their effects will be positive and their risks will be manageable.”

It calls for all A.I. labs to immediately stop training of A.I. systems more powerful than GPT-4 for at least six months and says that the moratorium should be “verifiable.” The letter does not say how such verification would work, but it says that if the companies themselves do not agree to a pause, then governments around the world “should step in and institute a moratorium.”

The letter says that the development and refinement of existing A.I. systems can continue, but that the training of newer, even more powerful ones should be paused. “A.I. research and development should be refocused on making today’s powerful, state-of-the-art systems more accurate, safe, interpretable, transparent, robust, aligned, trustworthy, and loyal,” the letter says.

It also says that during the six-month pause A.I. companies and academic researchers should develop a set of shared safety protocols for A.I. design and development that could be independently audited and overseen by unnamed outside experts.

‘Robust’ governance

The letter also calls on governments to use the six-month window to “dramatically accelerate development of robust A.I. governance systems.”

It says such a regulatory framework should include new authorities capable of tracking and overseeing the development of advanced A.I. and the large data centers used to train it. It also says governments should develop ways to watermark and establish the provenance of A.I.-generated content as both a way to guard against deepfakes and to discover if any companies have violated the moratorium and other governance structures. It adds that governments should also enact liability rules for “A.I.-caused harm” and increase public funding for A.I. safety research.

Finally, it says governments should establish “well-resourced institutions” for dealing with the economic and political disruption advanced A.I. will cause. These should at a minimum include: new and capable regulatory authorities dedicated to A.I.”

The letter was put out under the auspices of the Future of Life Institute. The organization was cofounded by MIT physicist Max Tegmark and former Skype cofounder Jaan Tallinn and has been among the most vocal organizations calling for more regulation of the use of A.I.

Neither OpenAI, Microsoft, or Google has yet commented on the open letter.

A spokesperson for Anthropic, a startup formed of researchers who broke away from OpenAI and which is building its own large language models, said, “We think it’s helpful that people are beginning to debate different approaches to increasing the safety of AI development and deployment.” He then pointed Fortune to a blog Anthropic had previously written on A.I. safety.

Andrew Ng, a computer scientist known for his pioneering work in deep learning and currently the founder and CEO of Landing AI, a startup that helps companies implement computer vision applications, said on Twitter that he was not in favor of a moratorium. “The call for a 6 month moratorium on making A.I. progress beyond GPT-4 is a terrible idea,” he wrote. Ng said he say many new applications of A.I. in sectors such as education, healthcare, and food where advanced A.I. was helping people. He also said there would be no realistic way to implement the moratorium without government enforcement. “Having governments pause emerging technologies they don’t understand is anti-competitive, set a terrible precedent, and is awful innovation policy,” he wrote.

Others took to Twitter to question the letter’s premise. Emily Bender, a computational linguist at the University of Washington, said that the letter seemed to be feeding into the hype around A.I. hype even as it claimed to be trying to point out the technology’s dangers. She alluded to a much cited 2021 research paper she co-wrote on the ethical problems with large language models with then Google A.I. ethics co-head Timnit Gebru (and which contributed to Google’s decision to fire Gebru.) “We wrote a whole paper in late 2020 (Stochastic Parrots, published in 2021) pointing out that this head-long rush to ever larger language models without considering risks was a bad thing,” she wrote. “But the risks and harms have never been about ‘too powerful A.I.’ Instead They’re about concentration of power in the hands of people, about reproducing systems of oppression, about damage to the information ecosystem, and about damage to the natural ecosystem (through profligate use of energy resources).”

Arvind Narayanan, a professor of computer science at Princeton University, wrote on Twitter that “This open letter — ironically but unsurprisingly — further fuels A.I. hype and makes it harder to tackle real, already occurring A.I. harms. I suspect that it will benefit the companies that it is supposed to regulate, and not society.” He said that the real dangers from A.I. were neither mass unemployment nor the idea that A.I. would destroy the human race but that existing large language models like GPT-4, which are increasingly being connected to the Internet through plugins, would make mistakes resulting in real financial or physical harm to individual people.

Update, March 29: This story has been updated to incorporate comments from deep learning pioneer Andrew Ng, Anthropic, Emily Bender, and Arvind Narayanan.