Vinod Khosla, famously an early investor in OpenAI, is making a new bet on AI model startup Symbolica, which launched today with a $33 million Series A round led by Khosla Ventures.
In an exclusive interview with Fortune about the latest investment, Khosla said Symbolica offers a “very innovative approach” to one of the biggest challenges facing AI: how to develop smaller, more efficient models that can reason more like humans do. “We love people coming from left field,” he said.
Symbolica is tackling what some have called a moonshot goal: creating a brand-new architecture that it claims improves on transformers, the current best-in-class method used to build AI models from companies like OpenAI, Google, and Anthropic. The secret sauce of transformers is about building bigger and bigger models, using more and more data and computing power—in many cases costing hundreds of millions of dollars. In addition, current models are often “black boxes” that lack transparency and make them tough to trust and audit.
But Symbolica takes a totally different—and even geekier—route: It’s applying a long-standing mathematical theory that the company claims will completely redesign how machines learn—while also enabling developers and users to better interpret and control what the models produce.
Khosla bets on an ex–Tesla founder
While Khosla said he had no inkling of the “ChatGPT moment” awaiting OpenAI when he made his $50 million investment in 2019, the startup already had a large, proven team led by CEO Sam Altman, president Greg Brockman, and chief scientist Ilya Sutskever. On the other hand, with Symbolica at an earlier stage, he is taking a singular bet on its founder—former Tesla senior autopilot engineer George Morgan, who leads a small, 16-person team that includes several Ph.D.-level mathematicians.
In 2022, Morgan quit his job at Tesla, where he had worked for four years after starting as an intern. He said he witnessed how CEO Elon Musk and Tesla’s then head of AI, Andrej Karpathy, tried—and failed—to make their self-driving AI systems perform better, and he disagreed with their emphasis on ever-bigger models with more data and compute. They simply “cranked the scale knob,” said Morgan.
But as full self-driving at Tesla still proved elusive, Morgan began to question their philosophy. “What if scale isn’t all you need?” he said. “What if there needs to be more clever design put into the actual architecture?” In particular, he wondered, what if models could focus on “structured reasoning”—that is, rather than simply being trained to predict the next word, what if models had the ability to logically process, organize, and generate information based on a set of rules or structured understanding of a topic?
When Musk, and especially Karpathy—whom Morgan has said he disagreed with to a “point of great personal contention”—didn’t listen to his concerns that the future of deep learning could not “scale to infinity and solve all problems,” Morgan left to found Symbolica. Khosla first gave Morgan $2 million last year to prove the value of his idea—that a real-world application of a branch of mathematics called category theory, which focuses on the relationships between algebraic structures, could deliver reasoning to AI models.
“He delivered that, very credibly,” Khosla recalled. “So we said, ‘Go hire the best people in this field of category theory.’” He says that while he still believes in OpenAI’s continued success building large language models, he is “relatively bullish” on Morgan’s idea and that it will be a “significant contribution” to AI if it works as expected.
As a result, he invested more money in Symbolica, with participation from Day One Ventures, General Catalyst, Abstract Ventures, and Buckley Ventures. The valuation was not disclosed.
In addition to the Khosla-led fund raise, Symbolica also laid out its focus in a new paper written in collaboration with researchers from Google DeepMind. While Khosla admits he does not understand the math-filled paper—pointing out there are very few people in the world who fully understand category theory—“when these really smart people gravitate to an idea, it’s an important idea,” he said.
Morgan calls the current state of AI ‘alchemy’
Morgan calls the current state of AI a kind of “alchemy,” rather than science. “You hire a bunch of people, and then they get a big cauldron, and then they pour a bunch of stuff into it, they throw in a frog, and then they mix it together—we don’t know how [these AI models] work,” he said.
In addition, he claims that while the transformers architecture underpinning today’s foundation models are “quite a technical marvel,” they are “not really that good—they’re not really that good at coding. They’re not really that good at reasoning.”
But Symbolica’s effort means there is a “general mathematical formal discipline for saying, ‘I want an architecture that does X, Y, and Z,’” he said. “We can know how the models will work. We can know what their internals are; we can know how they’re making decisions.” That said, Morgan emphasized that Symbolica is not working to build one model that is better than those built with transformers.
“We’re going to be building an entire class of models that outperform the transformer,” he said. “I’m exceedingly confident at this point that this will work.”
Khosla sounds more equanimous about the future of Symbolica, as well as all of his AI investments, which he guessed includes 40 to 50 companies since 2012. There is more than one approach to progress in AI, he said. And Morgan and his Symbolica colleagues have a “reasonably good shot of being a more efficient, more capable approach.”
But either way, he added, his willingness to fail is what allows him to succeed: “If I hadn’t bet on a reasonable shot—OpenAI—then we wouldn’t be here.”
