If you’ve been following AI chatter online or stopped by a recent AI conference, chances are you probably heard everyone talking about RAG. 

RAG, or retrieval-augmented generation, is an emerging technique for giving an existing AI model new information in order to have it perform a specific task. Whereas fine-tuning involves actually adapting an existing model using new data, resulting in a derivative model, RAG is simply giving a model a set of information it was never trained on and asking it to consider that information in its response. With RAG, you’re not changing the model, just asking it to temporarily reference external data in order to complete your specific request. It “forgets” the information right away. 

“In the simplest version, I retrieve from my internal database some documents or some text that I want, and I send it to the model along with my ask: ‘Answer this question’ or ‘Summarize this.’ The model does it. The model has not changed. I go do another RAG with a new ask and it keeps going,” Sriram Raghavan, vice president of IBM Research AI, told Eye on AI. “The reason it’s very popular is because it’s simple.”

This simpler approach to working with existing LLMs is only possible because of the sheer power of the latest models. Raghavan estimates this technique started rising around six to nine months ago, trailing the release of ChatGPT by a quarter or two. It’s incredibly useful for anyone who wants to build an application using an existing large language model, and it’s easy to see why it’s catching on. 

Fine-tuning and even prompt engineering, in which users structure and refine text prompts to elicit specific responses from LLMs, are time-consuming and add a lot of complexity. And while fine-tuning typically requires you to provide the model with hundreds or thousands of examples, RAG calls for just a document or two, maybe a few dozen examples. Of course, many more complex tasks will continue to require the more intensive process of fine-tuning, but RAG provides an easy way to supercharge a leading LLM to perform a wide variety of tasks with more recent data, domain-specific data, and even proprietary data. 

“I want to leverage the fact that the model is good at language, knows how to work with it,” Raghavan said. “I want to leverage that, but I want to do it on my data.”

But RAG isn’t a silver bullet. As models get better and better, retrieval is the hard part, according to Raghavan. You need to give the model the correct input, which might mean scouring a large set of documents to narrow it down to the most relevant ones, breaking up documents, or because LLMs can only read text, reformatting more complex PDF documents that contain tables, charts, and graphs.

IBM is currently working on offerings targeted specifically at helping application developers use RAG. For example, they’re creating patterns and “cookbooks” that offer recipes for how to do RAG according to what type of application you want to develop. Microsoft, Google, and Amazon also have their own RAG solutions. Companies want to be able to use the power of LLMs on their own data, which means RAG has a significant role to play in the enterprise in particular.

And with that, here’s more AI news.

Sage Lazzaro
sage.lazzaro@consultant.fortune.com
sagelazzaro.com

A.I. IN THE NEWS

UN Secretary-General launches a High-Level Advisory Body on AI. The body will consider how it can link various AI governance initiatives that are already underway and “it will work fast, because we are against the clock,” said UN Secretary-General António Guterres in his remarks, adding that it will make preliminary recommendations by the end of the year. He emphasized that the body is gender-balanced, geographically diverse, spans generations, and that its members bring a wide range of perspectives with “deep experience across government, businesses, the technology community, civil society, and academia.” The idea of creating such a body has increasingly become central to the discourse around how to handle the technology’s rapidly increasing impacts on society, with some AI leaders like OpenAI CEO Sam Altman being major proponents. 

President Joe Biden is expected to announce an AI executive order on Monday. That’s according to the Washington Post. The order is expected to require "advanced AI models to undergo assessments before they can be used by federal workers" and also make it easier for highly skilled tech workers to immigrate to the U.S. 

GM's Cruise grounds its entire fleet of self-driving cars. Days after California's DMV suspended Cruise's license to test driverless cars in the state and to operate its 24/7 robo-taxi service in San Francisco, Cruise said it will pause driverless operations of its vehicles everywhere. The GM-owned company, which has also been testing its vehicles in cities including Austin, Phoenix, and Miami, said it was taking the pause in order to help restore public trust. California's DMV had cited safety concerns for suspending Cruise's permit after several high-profile collisions.

14 AI inventions make Time’s list of the best 200 inventions of 2023. The list honors several AI models including Open AI’s GPT-4 and DALL-E 3, Meta’s SeamlessM4T, and Stability AI’s Stable Audio. The honorees also include AI systems developed for environmental protection including the AlertCalifornia and Cal Fire AI Wildfire Detector system out of the University of San Diego, as well as TrailGuard AI, which is designed to help monitor endangered species and spot poachers.