Fortune 500

Get insights into GPU including business profile, stock information, news, and rankings from Fortune.

GPU (GPU): Company Profile, Stock Price, News, Rankings

legacy

Companies

GPU | Fortune

GPU (GPU): Company Profile, Stock Price, News, Rankings | Fortune

Today's AI stack is delivering big productivity gains, but it continues to suffer from serious shortcomings. The AI GPUs on which the stack rests are not optimized for running trained AI models. They consume too much energy per unit of intelligence. SambaNova is trying to solve that problem with a better AI chip. Meanwhile, AI models do not learn continuously or have true memory. That's what Adaption Labs is working on. In conversation, their chief executives discuss how we get from the stack we have to the stack we truly need. 

Sara Hooker, Co-Founder and CEO, Adaption Labs
Rodrigo Liang, Co-Founder and CEO, SambaNova
Jeremy Kahn, AI Editor, Fortune

From the AI We Have to the AI We Need

Conferences

Right, there we go. um. The title of this panel is from the AI we have to the AI We Need or the AI We Want rather, um, and Sarah, I know you've written it before about like the slow death of scaling and why you think that today's sort of vastly uh gargantuan models are maybe not where we're heading. And uh, and Rodrigo, you built a company that's basically, uh, building uh AI chips to run these huge models but much faster, much more efficiently. So I just want to start with maybe one sentence from each of you, um, you know, do you think that the, you know, Sarah, that today's foundation, uh, LLMs are really, uh, you know, where, where we're going, or do you think we'll look back at this and say this was a detour and a bit of a dead end. Yeah, it's lovely to be here. One sentence. I would say today's models are monolithic. They're stuck in time. So a big goal of how we think about what should be fundamentally different is how do we efficiently learn from the environment, because now all models are moving towards interaction. So now it matters more that last mile performance and Uh, for that you need models which can evolve, otherwise you end up, uh, with massive inefficiencies. Rodrigo, are we going to be in this paradigm of huge models for forever? Uh, well, I, I think the world's heterogeneous. I think scale matters and so these large models are, I, I think, are going to be here for a while, and I think there's going to be plenty of room for new models and more efficient models to come in, um, and, um, have a, have a say in the market, right? And do you, do you think we need evolution across. All parts of the stack and maybe talk a little bit about what Samba Denova is doing on chips. Well, I think we're entering the phase of production, and I think if you think about production and at scale, you've got basically three constraints that people are struggling with. One is the models are getting bigger, at least one class of the models and the ones that, you know, we hear most about, I think cost is a huge part of the problem as you go to scale. That's just too expensive and that's largely driven by the infrastructure that we have. And then of course also about. Power and you know when we, when we, we argue about power and and being able to find uh chips, that's not because we're running small models, that's because the large models are taking a disproportionate number of chips to run and so we've got to tackle those problems. Got it. Sarah, at adaption, you're trying to solve some of the remaining challenges that we really haven't been able to solve with the current transformer-based architectures. Can you talk about what some of those gaps are that adaption's really trying to to fill and to solve for? Yeah, I would say, I mean this question kind of amounts to like how did we get here and like where do we go next. So I would say I would agree that the last 15 years what Rodrigo's saying is true. We've just, we've had a very simple cheap formula. You just build bigger and bigger models, but I think we're at an inflection point with massive urgency to change that curve. I would say there's two reasons. Most people intuitively understand. That you shouldn't just apply the same model to all problems, that probably 90% of problems are very easy. Many of the things that you do, um, in bulk processing, for example, you shouldn't be throwing a massive model at 10%, you need a ton of compute. So that is intuitive to people and things like adaptive compute, how do you do much more flexible model. Behavior is very critical, but the second intuitive thing I think most people grasp is that what matters now is that last mile performance, and that typically means you're in a long horizon task. You have many sequential agent actions, and if you just use a monolithic model, error compounds, so. A lot of what we care about is how do you adapt continuously to new data and how do you speed up changes to model behavior and both of those things are pretty critical to our mission because efficiency is key there. So when you have these massive rollouts, um, if you're just naively doing it, which I think a lot of people are feeling right now as they use a general harness monolithic model, um. You just end up painfully inefficient and that reflects as well in all the discussion I think has been at this conference about what am I going to do about my API bills. Right? And you talk a little bit about continuously learning from data that is one of the big unsolved challenges. Um, number of startups such as yourselves have come up in the last year saying they're going to try to address this. What's your intuition about how you solve for kind of continuous learning? Well, firstly, like what is continuous learning? Why do we care about it? Like a lot of why we care about it is now we have these long horizon tasks. If we're not updating model behavior as we go, you're, you're paying in a very expensive way for just blindly doing a rollout. Um, continuous learning is really the, the, the goal that you should be able to update model behavior without forgetting what you already know. For this, I think it's pretty critical for us that we infer automatically what the task is and are changing model behavior every step. For us, the constraint should be real time because unless it's real time, your cost, and maybe Rodrigo will talk about that, but your cost of even leveraging hardware and kind of offloading things to memory and back. Extremely expensive, so we want algorithmic and hardware codesign and that supercharges and I think differentiates what we're doing is that we actually care a lot about that synergy squeezing performance out of hardware and making it as fast as possible to update model behavior. Arrigo, what do you think about this idea that the model and the chip really need to be paired together and are you working on any of that at Somba Nova with any of the providers? Well, I mean, I, I think there's a lot of affinity between kind of the way that the chips run and the models that people build. Some of them we're focused on inference and we'll focus on. Um, premium inference, which is defined by speed and size and so we think that until the great new models are efficient come, you know, the battleground is in these very large 1 trillion parameter sized models, and you've got to run those much, much more efficiently than we are today to get the cost down. And so just give people a sense of how, how much faster you are than some of the competition now. Yeah, so if you look at a large Model only a 1 trillion parameter model these days, and we run full precision. There are a lot of little ways that you might be able to game it by quantizing the models down, but as Sarah will, will say, you lose accuracy on that, and that's not really the right way for hardware companies to go do this. And the way we do it is we run everything original precision. If great, uh, researchers created a wonderful model, it's not for the hardware guys to come and just, you know, chop the accuracy. And so we run a full precision, uh, and we run about 10 times. Faster at 1/10 the power. And so, but we don't necessarily come and say we do everything. What we do is we take the most popular models that people like and we optimize for them. So that's kind of tying into the synergy you're talking about. We look at what's, what are people struggling with, what do people want real time for, and then we drive the efficiency of those up and get the cost and the power way down. And Rodrigo, I know in, in the data centers that you're deploying into. You talk a little bit about heterogeneous compute. It's not all Sombonova chips. You're also using, you know, some CPUs from others, and you've even integrated with other GPUs for running different model nodes, right? I mean this is something Jensen announced a few months ago, and we actually showcased this last week in Taipei called disaggregated inference. And so you can use Nvidia GPUs in the front just to do the prefill and We run the decode of the exact same models significantly faster and we're getting 2 to 3x better than the Blackwell's on the exact same models and so we think that at scale that's the way to at least bring the cost down. You can get 2 to 3x uh cost reduction, 2 to 3x power reduction, and you get faster tokens for these real-time applications. Great. I want to go out to the audience for questions. Uh, who has questions? This gentleman right here got his hand up really fast, so please tell, please tell, please tell us who you are. Hi, I'm Paul Brody with Nightfall Networks, very interested in the case of uh Sombonova and adaption, especially for Sombonova, when you're talking about 2 to 3x improvement in performance, does that apply for training, or does that also apply to in addition to execution, and does that mostly just work with large language models, or can you put other kinds of models into that technology? So, so when I talk about that, this is really ultimately about inference. So we're inferencing models that can be As a single large model, a 1 trillion parameter model, or an aggregate of agents, and so you can aggregate agents that are doing various different things like Sarah says serially connected, uh, and then in total potentially adding up to 1 trillion parameters that you're switching in and out. But I think those two use cases is what we see the most. And Sarah, I wanted to ask about the question you, you touched on it in your answer to the previous question on um. Often with a, you know, what people really want is, uh, an answer to their question or or they want to run an agentic process at a price point that makes sense for them, and they don't necessarily, not necessarily wedded to one particular model to do that. Uh, and people often are saying, well, I wish I had a way that you could easily route, uh, a question to the right set of models or the right model. Are you looking at this as adoption of how you can potentially route to different kinds of models? Yeah, I think that so candidly, so, so much of what we're looking at is how do we make the full stack adaptable, so um. I think that routing is one, but even things like harness design, how do you automate that? Because everyone now has a different view of a harness, and oftentimes that's the main cost of inefficiency. And also for us it even extends to how do you change the whole innovation cycle. We just released Autoscientist which basically allows anyone to build a frontier model and a radically accelerated timeline. So instead of months it's In 2 days, the goal there is, you know, the people who can build models are actually very narrow. It's probably less than 2000 people who build frontier models. This came up in some of the talks yesterday, but how do we really learn in and teach models some of that knowledge that goes into that optimization process. So for us auto scientists is it automatically. Specifies uh how do you train for a given modality as well as like how does it automate all those choices of harness for a given problem that relates to something that came up earlier how do you do adaptive compute? How do you spend less when you don't need to, um, which is quite fun. um I'll throw in for Rodrigo's point like I just I think this is quite interesting maybe it frames the choice about inference um, so much of current workloads are inference so. As people acknowledge that you don't just want to scale anymore, those are not the returns in the pre-training stage. Most labs, but also most workloads are shifting towards how do you do things test time. So inference, there's a reason we're designing this domain specialized hardware. So I just wanted to add that because I think it's very important as people think about why we have this specialization. Yeah, there's a question. Yeah, one second, wait till the mic gets to you. There we go. Wait, who are you? Tell us who you are. Really interesting one is, you know, you have 1 trillion parameters, and you're designing to cope with that. And that begs the question to me about, well, maybe it will make more sense economically to do more efficient ensembling so you get rid of all of those freaking. But the other is you keep doubling down on inference, and I'm just wondering, I'll use the word in public ontologically, is the meaning of inference changing? As you create a learning loop between, yes, that's a very fun question. I'll share. So traditionally inference is you have a single model, you're basically just optimizing for throughput, and you basically have, you're trying to get squeeze as much performance out of a single model, and you're just passing things through. What's fundamentally different though, and Rodrigo, I think this is also changing the hardware scene critically. Um, is you now are doing agentic long horizon task, which means you have to check in as you go. That is a very painful operation. It means it's basically the whole crisis with memory. So you can think of like how we design hardware and optimize for efficiency as like we have this big oven. We can build the biggest oven in the world, and our only bottleneck is how fast the baker can put stuff in the oven. The baker is memory. What's fascinating. Now is that we're basically taking things out and putting things in the oven multiple times over the course of the recipe. So this makes it so important that you're designing things well. So inference is a different beast. How do you make it efficient and all your choices, like how do you auto optimize that become even more critical. It's also probably Rodrigo can speak to us, that's probably dramatically changing your world, right? I'm guessing. Well, I think it's, you know, to Sarah's analogy, it's actually worse than that. I think we have. 50 ovens, but 50 ovens, and we're actually moving from one oven to another to another and so ideally we, you know, we, we want to find ways to keep things in the oven and not keep opening. It's like the way I actually do turkeys for Thanksgiving. I keep checking in and you shouldn't, you know, so we really want to keep it in there and, uh, and that's what we're, uh, we, we think that the optimizations come from. Got it. Other questions? Other questions, to be honest, otherwise I'll keep going. Anybody? All right, um, uh, this is an interesting, uh, issue around, um. And would you just sort of, how do you, how do you do this though? I mean this is what I want to drill down on. What, what is for Soma Nova, the strategy for, uh, you know, making sure you're optimizing for these really large models? Maybe you don't have to run all those parameters. Are you doing things also on the, on the kind of middleware layer or, um, you know, at the kernel layer to optimize? Well, I think there are two ways to do this. I mean, if you have a hardware cycles, our hardware cycles are 3 to 4 years. Uh, so we're releasing now things that we created 3 years ago, and we've got great minds like Sarah who are inventing things that we could not possibly see 3 years earlier. And so we as hardware vendors, we're, we're having to create general purpose infrastructure, you know, things that allow future innovators to be able to run on our platform. And so we try to create things that allow us to take kernels that people are Going to use and implement them and still get the benefits of new architecture. And so, uh, there are, there are folks that say, hey, I wanna harden existing models, but by the time that hardware shows up, you're, the model is no longer of interest. And so, uh, so our, our view of it is, find the abstraction layer that allows innovators to create those, uh, uh, those architectures and primarily GPU, uh, centric, uh, ecosystems. Training and then allow them to then seamlessly bring it over to accelerators to get the real time and scale. Got it. Well, Rodrigo, I'd be remiss up here if I didn't, if I didn't talk about, uh, you know, Samanova. I'm sure you guys must have watched Cerris go out for its IPO. Um, you know, you guys were in discussions with Intel. Uh, there was talk about a purchase. Ultimately, you formed a very close partnership with them. They did not acquire you. Um, are you looking at IPO now for Samanova? Well, I mean, the market is incredibly hot for for technologies like ours. I think in the end we feel like, you know, there's a market that that requires our technology. And so as, as we build the business and we're able to show consistency in in deployment across a broad range of things, I think that it might be in the future. Got it. And Sarah, I want to ask you. For adaption, you know, you're working on these very interesting problems, yet there's so much attention on just a couple of the frontier labs, Anthropic, OpenAI, and they're sucking in a lot of talent and especially as they're heading towards IPOs soon. A lot of people are very attracted to work there, I'm sure. How are you finding sort of this war for talent as a smaller lab working on a very, very different kinds of architecture? I think when so most smart people like two things. They like very interesting questions and they want to work with people who are talented. I think that adaption has both of those. What I will say is that there's different stages to a company. A company just before IPO attracts a certain type of talent, a company that is at its beginning that resonates with a lot of people because especially if you're building something at the frontier of what's possible, that feels remarkable. It's also a very different type of problem. So the companies that IPO, their identity. To be cheeky has been weight-based, right? Even the discussions we've had here about, you know, it's always about the name of the model. Our, our bet is fundamentally different that the shift and the focus will put the model in the background and that by continuously evolving and interacting. And what matters much more is the critical efficiency of that last mile task. So this is a fundamentally different way of viewing the world. You're not thinking about a monolithic set of weights. Your identity is not the weights. You're thinking about how fast and how efficiently can you get that model to learn from its environment. And I think for many talented researchers that feels like the most important problem they can work on because it's the new era of what's coming next. Which is fun. Yeah, and it's much more about a systems approach to solving a problem as opposed to looking only at the model. 0, 100%. If you, so everything is moving towards interaction, I think existing Frontier Labs is just inertia there because they, their whole trajectory of how they, they shape teams, where they place resources is around stages of training, right? There's a pre-training team, continued learning team. Training RL. This is how the very resources are configured. If you're moving towards interaction, you care about different things. I care a lot about how do we design stuff that plays extremely fast with hardware because we want to update as fast as possible. If I care about interaction, I care about interface because I want to get feedback as quickly as possible to know when our model should update, and this requires a different type of team working much more nimbly and actually. A different type of co-design and that's, I think what attracts a lot of people and makes it a very fun time to be working on innovation again because the formula now is not as simple as like just throw more weights at the problem and that means in some ways it's it's a good era. The era of research is back again. The era of research is back again. All right, that's a good note to end on. We're afraid we're out of time, but thank you very much. Thank you for listening.

This week Jeff and Leo discuss ex-NYC Mayor Eric Adams’ foray in the crypto space, possible insider trading on Polymarket, and the case for concern when it comes to World Liberty Financial’s banking charter application. Leo also talks with Lighter CEO Vlad Novakovski about decentralized exchanges and the bull case for crypto.

 A rugpull from NYC’s ex-mayor, prediction market hijinks & Lighter’s CEO | Fortune’s Crypto Playbook

Fortune_Crypto

Wayve CEO Alex Kendall takes Fortune’s Jeremy Kahn for a ride in London, showcasing the mechanics of the company’s AI-powered vehicle running on a single GPU

Microsoft, NVIDIA, and SoftBank have bet over $1 billion on this self-driving startup—here’s what it’s like to take a ride

Features

Microsoft, NVIDIA, and SoftBank have bet over $1 billion on this self-driving startup

Just engaged autonomy and look at that. The A I is now controlling the speed, the steering, everything about the vehicle and we're straight into traffic. Let's watch how we safely navigate around that cyclist. You can see we're slowing down, giving us some distance. So we're going to make a left turn, which would be interesting to see. So, yeah, we should slow down and see how we let the cyclist go through and then make the left turn when we moved over to make a left turn and we had to pick which lane we were going to, was it wanting to that? It knows from mapping that, that it has to be the left lane or did it actually read the indications on the roadway? So it gets to simply navigation from like drive straight or take a lift here or things like this a bit like what you might get if you use a set net system, some of the way the car is control the speed it chooses or the line it takes. None of this is hand programmed, not following a map, but it's making all of these decisions based on what it sees and the whole driving models on a single GP U in the car. Yeah. Yeah, there's nothing on the cloud. It's all on board the vehicle on a single GP U. A similar amount of computes to what we're seeing in production vehicles today. How did you gather the training data for the system? We use a wide variety of sources of training data to make sure that we learn the most safe and performance system. So of course, we have our own fleet that we test throughout the day and night, getting experience in London and we do some trips to other cities as well. From time to time, we partnered with ASDA and 02 of the largest grocery fleets here. Together we collect data from their grocery fleet and we have some other initiatives like that, that we, we're sort of taking off around the world. You mentioned, it's sort of like a large language model takes in all of that data, including things that you wouldn't necessarily want the model models behavior on. But having that amount of data, it sort of makes the system more robust in the end. But then of course, with large language models that you do have that sort of reinforcement learning through human feedback process. Is there some similar equivalent thing that you do with the data? Yeah, it is. And the fact that the way that we test and we give the feedback or in simulation when we resim and look to understand how the system performs. It's, it's a, it's very similar form of feedback in general. The way that, you know, I've been, I've been working on these A I models for over a decade now and I've seen breakthroughs in computer vision or robotics or natural language. They all tend to converge we're seeing now the same techniques apply in different fields. So the rise of synthetic data, for example, in large language models, we see the same thing with generative A I and what we can do in simulation for driving multi. Now we have vision uh language models, you know, say with the LLN similarly, we have lingo which gives us language capabilities in in this car. So all of these kind of trends are playing out in a very similar way. Of course, the interesting thing in this space is that we have the integration problem of making it run real time in a car and the safety challenges of running in the open world where we can't control what might happen. Ultimately, I think what we're learning is that A I is breaking through a lot of the glass ceiling that we've seen in the industry so far, removing the need for maps and being able to leave the geo fence, enabling the use of cost-effective hardware and sensors. Uh But more importantly, solving the long tail of cases and producing behavior that's intelligent enough to drive in complex environments like London

Nala Robotics is using A.I. to automate jobs in restaurant kitchens.

Robots are moving into restaurant kitchens thanks to A.I.

Tech

a robot is nothing but a mechanical reflection of us. As long as it's given the right mode of payment, the data is being handled properly. Robot actually can prove equal, if not better than a human, other robotics uh is uh ai based technology company. We are specializing in kitchen robotics, uh mainly automating and introducing Ai concepts into the back of the kitchen. Most of the technological advances that happened in the restaurant industry are the front of the house uh has been introduced for awhile, mainly in the system. We realized that the back of the house does not have any many technological advances. And that's when we started introducing robotics. Nala can actually perform two important tasks. One is, it can perform the task of line cook as well as the chef. The robotic arms actually get the recipe and actually perform the tasks that are normally performed. Then the chef part of it is the AI element. The software is the element we have our own Os called uh that's the brain of the, that's more of a chef that can create new recipes, come with new ideas, you know, look at the ingredients and then say, hey, you know what, why don't you make a new dish today? So that's a part of it. It is multi cuisine. So it actually can virtually cook anything that is programmed to do given a few ingredients. It actually can come up with new space on its own uh it actually would use feedback mechanisms that the customers give to enhance the taste and come with with new new cuisines. In fact, The National Restaurant Association has actually surveyed restaurant operators across the country not only about their their thoughts about technology, but what they're using and what they expect to do in the future. And as you can imagine, they view technology as a very, very important competitive differentiator. I was looking into robotics and I always thought it was gonna be like 3-5 years down the road and then meeting AJ and his company. He changed my mind real quick. And uh yeah, it's here, it's it's now and it's happening. Pizza Yola is physically designed for classic american food. So it's not just making a pizza. It actually can make fried wings, fried chicken, it can make salads. Pizza Yola is 12 ft by 12 ft. It's much smaller than the system that can fit into any any pizza chain right now, what's unique about us is everything we do is larger than standard. So everything he has to build for us was custom. We are going to retrofit the first ones to fit some stores eventually will perfect the model which then the kitchen will be built around the pizza yola. When you apply technology to an extremely, extremely labor intensive industry, which the restaurant industry is the typical operator can get some substantial productivity and efficiency gains as we scale into certain markets, it's hard to find skilled labor with Saleh's product. The training will be much more efficient right now labor costs have significantly gone up, California is introducing a new law, increasing the minimum wage even higher. And that that's hindering the operations of a lot of restaurants in fact, so uh can address that ultimately would do, it will replace the jobs in our kitchen that most of our staff don't want to do and also be able to work the hours that most people don't want to work for. Late nights and the weekends. Our customers reaction is one of the biggest concerns. We want to be very clear that this product is not taking any jobs away from anybody, it's there to assist and to help make their lives better and to operate more efficiently. The cost of computing has come down significantly. The improvements in Gpu. The improvements in in the computing world has significantly improved the quality of current Ai machines. In fact added to that the cost of robots have significantly come down, robotic arms are more affordable than what it was 10 years ago that the advances that happened in the fields of vision in fields of you know, speech recognition made it all 10 years from now. I don't see restaurants not having some type of Ai assisted robotic in their in their operation. The industry has always demonstrated its innovativeness and ability to rapidly respond to consumer changing wants and needs. And so the ability to to continue to innovate and rapidly adapt. Including the integration of technology um becomes pivotal for their success and growth going forward.

GPU

GPU stock chart

GPU in Fortune Rankings

Videos