Adaption Labs represents a significant bet against the prevailing AI industry wisdom that the best way to create more capable AI models is to make the underlying LLMs bigger and train them on more data. While tech giants pour billions into ever larger training runs, Hooker argues the approach is seeing diminishing returns. “Most labs won’t quadruple the size of their model each year, mainly because we’re seeing saturation in the architecture,” she said.

Hooker said the AI industry was at a “reckoning point” where improvements would no longer come from simply building larger models, but rather by building systems that can more readily and cheaply adapt to the task at hand.

Adaption Labs is not the only “neolab” (so called because they are a new generation of frontier AI labs following the success that more established companies like OpenAI, Anthropic, and Google DeepMind have had) pursuing new AI architectures aimed at cracking continuous learning. Jerry Tworek, a senior OpenAI researcher, left that company in recent weeks to found his own startup, called Core Automation, and has said he is also interested in using new AI methods to create systems that can learn continuously. David Silver, a former Google DeepMind top researcher, left the tech giant last month to launch a startup called Ineffable Intelligence that will focus on using reinforcement learning—where an AI system learns from actions it takes rather than from static data. This could, in some configurations, also lead to AI models that can learn continuously.

Hooker’s startup is organizing its work around three “pillars,” she said: adaptive data (in which AI systems generate and manipulate the data they need to answer a problem on the fly, rather than having to be trained from a large static dataset); adaptive intelligence (automatically adjusting how much compute to spend based on problem difficulty); and adaptive interfaces (learning from how users interact with the system).

Since her days at Google, Hooker has established a reputation within AI circles as an opponent of the “scale is all you need” dogma of many of her fellow AI researchers. In a widely cited 2020 paper titled “The Hardware Lottery,” she argued that ideas in AI often succeed or fail based on whether they happen to fit existing hardware, rather than their inherent merit. More recently, she authored research paper “On the Slow Death of Scaling” that argued smaller models with better training techniques can outperform much larger ones.

At Cohere, she championed the Aya project, a collaboration with 3,000 computer scientists from 119 countries that brought state-of-the-art AI capabilities to dozens of languages for which leading frontier models did not perform well—and did so using relatively compact models. The work demonstrated that creative approaches to data curation and training could compensate for raw scale.

One of the ideas Adaption Labs is investigating is what is called “gradient-free learning.” All of today’s AI models are extremely large neural networks encompassing billions of digital neurons. Traditional neural network training uses a technique called gradient descent, which works a bit like a blindfolded hiker trying to find the lowest point in a valley by taking baby steps and trying to feel whether they are descending a slope. The model makes small adjustments to billions of internal settings called “weights”—which determine how much a given neuron emphasizes the input from any other neuron it is connected to in its own output—checking after each step whether it got closer to the right answer. This process requires enormous computing power and can take weeks or months. And once the model has been trained, these weights are locked in place.

To hone the model for a particular task, users sometimes rely on fine-tuning. This involves further training the model on a smaller, curated dataset—usually still consisting of thousands or tens of thousands of examples—and making further adjustments to the models’ weights. Again, it can be expensive, sometimes running into millions of dollars.

Alternatively, users simply try to give the model highly specific instructions, or prompts, about how it should accomplish the task the user wants the model to undertake. Hooker dismisses this as “prompt acrobatics” and notes that the prompts often stop working and need to be rewritten whenever a new version of the model is released.

She said her goal is “to eliminate prompt engineering.”

Gradient-free learning sidesteps many of the issues with fine-tuning and prompt engineering. Instead of adjusting all of the model’s internal weights through expensive training, Adaption Labs’ approach changes how the model behaves at the moment it responds to a query—what researchers call “inference time.” The model’s core weights remain untouched, but the system can still adapt its behavior based on the task at hand.

“How do you update a model without touching the weights?” Hooker said. “There’s really interesting innovation in the architecture space, and it’s leveraging compute in a much more efficient way.”

She mentioned several different methods for doing this. One is “on-the-fly merging,” in which a system selects from what is essentially a repertoire of adapters—often small models that are separately trained on small datasets. These adapters then shape the large, primary model’s response. The model decides which adapter to use depending on what question the user asks.

Another method is “dynamic decoding.” Decoding refers to how a model selects its output from a range of probable answers. Dynamic decoding changes the probabilities based on the task at hand, without changing the model’s underlying weights.

“We’re moving away from it just being a model,” Hooker said. “This is part of the profound notion—it’s based on the interaction, and a model should change [in] real time based on what the task is.”

Hooker argues that shifting to these methods radically changes AI’s economics. “The most costly compute is pretraining compute, largely because it is a massive amount of compute, a massive amount of time. With inference compute, you get way more bang for [each unit of computing power],” she said.

Roy, Adaption’s CTO, brings deep expertise in making AI systems run efficiently. “My cofounder makes GPUs go extremely fast, which is important for us because of the real-time component,” Hooker said.

Hooker said Adaption will use the funding from its seed round to hire more AI researchers and engineers and also to hire designers to work on different user interfaces for AI beyond just the standard “chat bar” that most AI models use.