Generative A.I.’s hallucination problem has companies contemplating if they want to ‘move fast and break things’ yet again

In a recent story in the Associated Press, artificial intelligence researchers and industry leaders interrogated generative A.I.’s “hallucination” problem, wherein tools like ChatGPT make up information and write it as fact. The problem stems from the very way that ChatGPT and other generative A.I. tools function—they aren’t really “thinking,” but rather use patterns to predict the next word in a sequence. ChatGPT writes according to what makes sense in a pattern, not according to what’s true.

“This isn’t fixable,” Emily Bender, a linguistics professor and director of the University of Washington’s Computational Linguistics Laboratory, told the AP. “It’s inherent in the mismatch between the technology and the proposed use cases.”

Even OpenAI CEO Sam Altman acknowledges ChatGPT’s penchant for spewing out falsehoods. “I probably trust the answers that come out of ChatGPT the least of anybody on Earth,” he said to a crowd on his university tour, to laughter, according to the article. 

Altman went on to say he thinks the hallucination problem will improve eventually. But even if it can be fixed, how long will it take? How consequential will the errors it makes in the meantime be? And how will A.I. have already upended our world, before we even had a chance to truly contend with the changes? 

There are already several documented cases of generative A.I. hallucinating defamatory information about people or otherwise being used to threaten people’s reputations, such as one instance where Meta’s BlenderBot 3 falsely labeled a former member of the European Parliament a terrorist, for example. And of course, hallucinations are just one of the many concerns surrounding A.I., including issues around privacy, copyright, faulty datasets, A.I.’s ability to perpetuate discrimination, uproot our current systems of work (and thus much of how society operates), and likely more we can’t yet imagine. 

When OpenAI released ChatGPT, it opened the floodgates. Google, which had been holding back on productizing its competing tech over concerns it gave too many wrong answers, released Bard to avoid falling behind and is now experimenting with how it can recreate its search business in the image of generative A.I.  Many companies quickly followed suit, rewriting their product roadmaps to meet the new generative A.I. moment. And as was made clear by artificial intelligence’s explosive prominence in Q2 earnings calls, it’s a big opportunity for business. 

Companies of all stripes now face a common dilemma: The competitive pressure not to fall behind in the A.I. race and the responsibility for the safety of the tools they release. A survey of 400 global senior A.I. professionals just released by Databricks and Dataiku gets at this tension, with the majority of respondents responding both positively about the ROI of A.I. initiatives and that they’re more worried than excited about the future of A.I. 

We’ve already seen what happens when technology that has the ability to uproot how we experience the world is unleashed prematurely or integrated widely despite known issues—for example, Facebook’s role in the genocide in Myanmar, ethnic violence in Sri Lanka, and election disinformation in the U.S. This moment feels like an inflection point for A.I. where we decide if we’re going to “move fast and break things” yet again.

Abhishek Gupta, founder and principal researcher at the Montreal AI Ethics Institute, said “this is one of the biggest issues that is causing a lot of strife within many companies” when it comes to moving forward with generative A.I. plans, specifically citing strife between the business and privacy/legal functions. 

“We are struggling with similar, if not identical, issues around governance approaches to a powerful general-purpose technology that is even more distributed and deep-impacting than other technological waves” he said when asked if he fears we’ll repeat the same mistakes made with social media platforms in terms of not properly mitigating potential harms prior to product release and widespread use.

The nature of the complexity of A.I. systems adds a complicating factor, Gupta said, but he argues that we need to think ahead on how the landscape of problems and solutions is going to evolve.

As if to drive home this point, Facebook-parent company Meta—the originator of the now infamous “move fast and break things” phrase—last month released its powerful LLaMa 2 large language model as open-source code, raising alarms about its potential misuse.

One clear difference between this moment and the early days of the social media era is that policymakers are moving quickly to consider a variety of regulations around A.I. Individuals like comedian Sarah Silverman and companies like Getty have also taken legal aim at companies including Meta, OpenAI, and Stability AI, alleging they broke copyright laws by training models on their work without payment or consent. The Federal Trade Commission is also currently investigating whether OpenAI violated consumer protection laws when it scraped people’s data to train its models. 

“One of the upsides of the current zeitgeist is that there is a lot more discussion around issues of ethics and societal impacts compared to the era when social media platforms were just taking off,” Gupta said. “That said, history may not repeat itself, but it certainly rhymes.”

With that, here’s the rest of this week’s A.I. news.

Sage Lazzaro
sagelazzaro.com

A.I. IN THE NEWS

IBM, Hugging Face, and NASA team up on the release of a new foundation model for climate and earth science A.I.s. That’s according to Engadget, which reports the new open-source geospatial model will serve as the basis for models that can monitor and predict deforestation, greenhouse gas emissions, and crop yields. NASA contributed its Earth-satellite data, while IBM tapped its Watsonx.ai model to serve as the foundation for the project. Hugging Face is hosting the model on its open-source A.I. platform. 

Alibaba open-sources two of its A.I. models. This includes Qwen-7B, a generative A.I. model that creates content in both English and Chinese, as well as a version designed specifically for conversational apps called Qwen-7B-Chat, according to CNBC. The move mirrors Meta’s decision to open-source its LLama 2 model last month in partnership with Microsoft as companies race to accelerate the adoption of their models. 

Zoom backpedals after A.I.-focused changes to its terms of service cause backlash. After recent changes to Zoom's terms of service laid out the company's right to train A.I. models with user data, as well as, in short, the company’s worldwide, royalty-free license to distribute and share it, users on sites like Linkedin and Hacker News called out the company for privacy concerns and pledged to cancel their subscriptions. Even for paid customers, it appeared there was no way to opt-out. Following the backlash, Zoom yesterday published a blog post clarifying that "Zoom will not use audio, video or chat Customer Content to train our artificial intelligence models without your consent" and updated its terms of service with the same language. The ordeal shows these issues are top of mind for users, as well as the increasing importance of clear terms of service in the A.I. age. 

OpenAI releases new features for ChatGPT. The series of small updates include the addition of prompt examples, suggested replies, and keyboard shortcuts. Users will also no longer be automatically signed out after two weeks, and Plus users will now be working with ChatGPT 4 by default when they open a new chat. But perhaps the most interesting update is the new ability to ask ChatGPT to analyze data and deliver insights across multiple files. 

Images and videos are coming to Google’s generative A.I.-powered search. That’s according to The Verge, which reports that users who enabled the Search Generative Experience (SGE) feature will soon start to see more multimedia pop up in the summary box at the top of their results. The SGE experiment, which seeks to generate helpful and accurate answers to queries rather than just surface links, is a clear look into what could be next for the search giant as it scrambles to reenvision its business model for a generative A.I.-powered internet.

EYE ON A.I. RESEARCH

Hackers can use A.I. to "hear" what you type. As with any technological innovation, A.I. is now in the toolbox of not only changemakers and visionaries, but also malicious actors. In a paper published this past week, a group of British university researchers described how machine learning can power “acoustic attacks” capable of analyzing a target’s keystrokes with startling accuracy—up to 95%. These kinds of attacks involve recording a target’s keyboard keystrokes via their device microphone, a nearby microphone, or even a Zoom call and then using a deep learning model to classify the keystrokes, thus unlocking access to their passwords, messages, and other sensitive personal or business information. 

The researchers used a MacBook and recorded the sound emitted by each individual number and letter key on the keyboard (they pressed each one 25 times). They then mapped the results in visual form, using waveforms and spectograms, and used the data to train the CoAtNet image classifier model.

“With recent developments in deep learning, the ubiquity of microphones and the rise in online services via personal devices, acoustic side channel attacks present a greater threat to keyboards than ever,” the researchers wrote, adding that their results prove the practicality of these attacks via off-the-shelf equipment and algorithms. You can read the paper here.

FORTUNE ON A.I.

Tim Cook answers critics about why Apple’s A.I. efforts seem so far behind Microsoft, Google and Elon Musk —Christiaan Hetzner

Andy Jassy summed up Amazon’s A.I. gameplan: Every single business unit has ‘multiple generative A.I. initiatives going on’ —Rachyl Jones

Canva CEO Melanie Perkins sees one major problem with A.I. right now —Emma Hinchliffe

Kenya becomes the first country to suspend Sam Altman’s Worldcoin A.I.-crypto scheme —David Meyer

HR leaders say A.I. will make a positive impact on their roles. CEOs are less convinced —Paige McGlauflin

BRAINFOOD

Lil Wayne vs A.I. As society starts to contend with the implications A.I. could have on creative industries, creatives like writers and musicians are now routinely being asked for their thoughts on the technology. In an interview published this past week on Billboard, rapper Lil Wayne responded to a question asking his opinion on A.I. and its potential effects on creativity by basically daring A.I. to try him.

“Someone asked me about that recently. And they were trying to tell me that A.I. could make a voice that sounds just like me,” he said. “But it’s not me, because I’m amazing. I’m like, is this A.I. thing going to be amazing too? Because I am naturally, organically amazing. I’m one of a kind. So actually, I would love to see that thing try to duplicate this motherf--ker.”

From publications using A.I. to write news articles to the debates unfolding around the technology in the Hollywood Strike, it’s clear some industry executives are bullish on artificial intelligence’s ability to do what creatives do. Wayne’s sharp response reminds us that creatives aren’t going to back down without a fight. Generative A.I. can certainly churn out massive amounts of content, but that doesn’t mean that content is going to be any good. In just a few months since generative A.I. was unleashed into the world, it’s already flooded the internet with junk.