Name: Brainstorm Tech 2024: Google’s Gemini Man
Uploaded: 2024-07-16
Duration: 17 min 35 s
Description: Jeff Dean, Chief Scientist, Google DeepMind and Google Research, Google Interviewer: Jeremy Kahn, Fortune

Jeff Dean, Chief Scientist, Google DeepMind and Google Research, Google Interviewer: Jeremy Kahn, Fortune

Brainstorm Tech 2024: Google’s Gemini Man

Conferences

Everybody. Thanks for joining us, Jeff, thanks for being here. Um I'm gonna go to the audience for Q and A pretty early on in the session. So please have your questions ready. Wait for Pat Mike handler to come to you and then stand up and please say your name and state your question. Um So Jeff in the introduction, Andrew talked a little bit about the merger between Google Brain and Google, Deepmind, Google had these two separate A I fundamental research units. What was the reason for combining them and what benefit have you seen from that, that merger? Yeah, I mean, I think both organizations were doing fantastic work and you know, a lot of different A I innovations have come out of both groups prior to the combination, you know, things like Transformers and Burt and alphago and various other Alpha related projects on the Deep Mind side, Alpha fold. But I think as we realized that we were both striving, you know, both organizations were striving to build, you know, large scale multimodal models that can do lots of interesting things. It made sense to kind of combine the people working on those ideas so that we have, you know, a better set of ideas to build on to pool the compute. So we sort of focus on training one, you know, large scale effort like Gemini rather than multiple fragmented efforts. And so that's been really good. I think it's been really great to see people come together and really just kind of work together on, you know, building awesome stuff. Right. And has there been a change in sort of the tempo with which you're able to sort of move things from research to product as a result of that merger or a change in the culture at all? From the from that merger? I mean, I think obviously we want to build, you know, things like Gemini that can sit underneath a lot of different products. And so we have a pretty strong focus on, you know, how do we make great models that are great for, you know, cloud applications, but also, you know, sitting underneath various search features or Gmail or other kinds of, you know, existing Google products, but also can create new kinds of products. So that sort of large escape of product focus is really important, but we're really focused on the fundamental research and making the best things possible and then rely on product teams to kind of, you know, take those building blocks and figure out how best to use them in their setting. Great when I talk to companies about using generative A I one of the things that always comes up is the rate of hallucinations from these models. And it seems to be one of the things that's created a bit of a bottleneck to actually deploying them in any kind of real use cases. I know you guys are working very hard on this issue. There was some hope that when multimodal models came around like like Gemini that that would reduce hallucination rates because the model would understand both the language and also the visual imagery that went with that language, it's made some difference, but that hasn't completely gone away. How are we doing on solving the hallucination problem? And is there kind of a, you know, do you think it's a solvable problem in the current architectures? Yeah, I mean, I do think we're making progress like it's a difficult problem for sure because the models are trained to generate, you know, probabilistically plausible sentences and those are not always true. So, but I do think there are some glimmers of hope. So, one of the things in the latest Gemini models is the fact that it has this very long context capability. So you can put lots of information that you're trying to sort of use and in answering questions that you that you're doing, and that information to the model is much clearer than sort of the model, the training data that the model was exposed to in the pre training phase, which is sort of like a big jumble of tokens that are used to generate these probabilistic models but is not as clear as you know, information in the context, right? So that information I think is much easier and the model tends to hallucinate less about that information. I think that's, you know, one promising direction and we're obviously pushing on lots of different other directions, right? At Google IO. This year, I know you guys made announced that you have this thing called project Astra, which is an attempt to kind of build an a a model that have agency. What what kinds of actions will astra be able to perform and how soon is this going to be released? Yeah, I mean, we're hoping to have something out into the hands of test users by the end of the year. So it's pretty exciting. I think the ability to combine Gemini models with models that actually have agency and can perceive the world around you in a multi modal way is going to be, you know, quite quite powerful. I think we're obviously approaching this responsibly. So we want to make of the technology is ready and that it doesn't have unforeseen consequences, which is why we'll roll it out first to a smaller set of initial test users. But I think it's going to be pretty interesting because all of a sudden, you know, the model can be perceiving the world around it around you and doing, doing stuff on your behalf. Right. Interesting Bill Gates recently said he thought, you know, large language models, there were sort of two more turns of the crank in terms of scaling up these models further. But he also said that he didn't think that those two turns of the crank would kind of get us A G. I'm curious what you think, how many turns of the crank are left on sort of transformer based LM models and where will those turns? Get us? Yeah, I mean, I think as you see the models grow in capability, some of that is due to scaling. So every time you increase the scale of the model and the amount of training data that these models are trained on with the model capacity as well. You do see new capabilities kind of emerge, you see the models, you know, hallucinate less, you see them sort of able to do things that they couldn't do before at smaller scale. So that that is a promising sign and you know, a couple more generations of scaling will get us considerably farther. I do think we're going to need some additional algorithmic breakthroughs, which is something that we've always focused on is how do we sort of take scaling but combine it with algorithmic approaches that combine well with scaling to sort of improve the capabilities of the model. I think things like factuality and reasoning capability where the model can sort of uh sort of imagine plausible outputs and you know, reason its way through which one makes the most sense or the kinds of things that are gonna be uh important to really make these models robust and, and more reliable than they already are. Right? I want to go to questions from the audience. We have a question, please raise your hand and we'll get a, get a mic handler to you. There's one right down here. Um If we can please get a mic to this gentleman and if you could please stand up and say who you are? Alright. Thanks, Mark Paper Master. I'm CTO at A MD. And uh Jeff, we're all um excited with the advancements like you, you all are doing with Gemini, but we also all are talking about the the demands of power. How do you think about it holistically from your role in terms of everything from the chip engineering all the way through the uh models in the application stack? Yeah, it's a good question. I think one of the things that, that I really like about Google is we have a fully integrated system. So we have, you know, we design our own chips for A I, although we also use other people's chips and then we have our own sort of system software on top of that, we can make efficient use of those chips. When we first introduced the tensor processing unit in 2016, you know, that was quite a big advance in terms of energy efficiency. It was about 30 to 80 times more efficient than CPUS and GPUS of the day. And we've now been through multiple successive generations of CPUS that have also made pretty significant improvements in energy efficiency. And we've also, you know, for many, many years focused on the overall energy efficiency of our data centers. We have very low P we have, you know, a lot of data centers of ours are powered by percent or more clean energy, which is super important as despite this, Jeff, you just reported that your co two emissions have gone up over the last several years, despite these sort of improvements and despite these optimizations, so can you talk a little about what's going on there and sort of how, what's the path out of this? I mean, if, if we have, you know, carbon footprint scaling at a rate that's, you know, going up at the same kind of slope as capability, maybe that's not where we really want to be. Yeah, I mean, we have a public commitment to by the end of 2030 you have essentially 100% of our data center operations powered by clean energy. And that's not necessarily a linear thing because often we make, you know, we uh work with clean energy providers, solar, solar, you know, capacity providers or wind providers and some of those things come online in 2027 2028 29. And so those things will provide sort of significant, you know, jumps in the percentage of our energy usage that is from energy. But then also we want to focus on making our systems as efficient as possible, right? And, and is, is, is the access renewable power become a constraint on sort of how quickly you can scale the technology. Um You know, I think there's been a lot of focus on the increasing energy usage of A I and you know, from a very small base that usage is definitely increasing. But I think people often conflate that with overall data center usage of which A I is a very small portion right now, but growing fast and then sort of attribute the growth rate of A I based computing to the overall data center usage. So I think it's really important to look carefully at all data and really understand the true trends that underlie this cool other questions from the audience. Please raise your hand. If you have a question, if not, I've got plenty more for Jeff. But if someone else in the audience has a question over here in the back, the gentleman in the colorful shirt first, I think, and then if you can please stand up and state your name, hey, Will Wilson from antithesis, Jeff, what is your current take on the debate over A I risk and alignment and all that stuff? Yeah, I mean, I think it's a, it's a good and healthy debate for sure. I mean, I think there's, you know, many different viewpoints along a spectrum of, you know, optimism and pessimism and, you know, how much concern should we have for, for, you know, disastrous consequences? I guess I'm somewhat in the middle. I think there's a huge amount of benefit from A I technologies to the world. So we should make sure that we sort of do, do our best to make sure that the world gets access to those things for things like education and health care. But we also need to make sure that some of the, the risks of A I are sort of mitigated or diminished in, in rolling these things out for the positive benefits. So it's, you know, obviously hard to predict the future in a very fast moving field. Uh I think probably the most extreme viewpoints on either side are probably not, not the most uh likely outcome. Do you have a take on, on the California Senate bill? 1047? Do you have a view on whether that's a good approach to regulation or not? I don't, I mean, I would say in general, it is really good for governments at all different levels to be thinking about how they want to regulate A I, I think there's certainly, you know, I sort of think of it as a very general technology and often the regulatory process probably should focus on how is that technology being used in a particular setting, right. So like if you have autonomous vehicles that makes sense to have like a certain set of regulations there in health care settings, there's already sort of a relatively comprehensive framework around roll out of new technologies and health. And so I think, you know, adapting those for new A I based things makes a lot of sense. Cool. I know we have at least one other question because I saw a hand up before. So why don't we go to the uh someone else? Um Just here. Yeah, please say it stand up and say your name. Yeah, hi. This is Sidar Taral with s software company work. So Jeff, I'm a zoo. So I have a soft spot in my heart for Google. But having said that one of the challenges we have is as we look at models, we don't want to be locked into one particular model because we don't know which model or which company is going to drive innovation the fastest. So what's your advice for companies like us in terms of how we should think about not locking ourselves out of future innovation by picking Gemini. Yeah, I mean, I think I hear those concerns. I think, you know, we think we're going to innovate quite quickly and then give people what they want in terms of models. But we also recognize there's a whole bunch of choices. This is partly why we, for example, have open source models called Gemma that we've recently introduced. We just provided Gemma two updates a few weeks ago of 9 billion and 27 billion parameter models and the 27 billion parameter model on the LM SIS um sort of elo rankings actually is the top ranked open model at the moment or it was at least a week ago. I haven't looked this week. This is a fast moving field but, you know, open source models can provide people some comfort that they have sort of a little bit more control. But I also think a lot of people want sort of a PS that they don't have to necessarily spin up a whole bunch of GPUS themselves. They can just sort of use it as a building block for, for getting stuff done. Cool. Uh Other questions down here in front. There's a question I see. Uh if we can get a mic to this gentleman just here right here in the front row. Oh Can we get a mic to this to him right here? Sorry. There we go. Thank you. Please say your name. Hey, my name is Ken. I'm uh CU and co-founder at union. Firstly, I'm a huge fan of your work. 20 years. Every paper, everything. This is what we have built on. Secondly, I think this is mostly looking into the future, purely technical question, not talking about the implications of business, but what's the one thing uh in a that will surprise you in a good way, in a bad way in those cranks that we're spinning. Like let's say we have this two, the next generation. I'm not even talking about the second one. What is the one thing that will surprise you in a positive way? And what will disappoint you if it doesn't happen? Like, yeah, I, I mean, I think um one of the things that is probably going to be a surprise to people is the improvement in multimodal capabilities. You know, I think the first couple of iterations of Gemini really sort of introduced multimodality, but I think it's going to be a much more naturally integrated thing for many different kinds of modalities, audio video imagery, both on the input and generation side. So I think that's going to be an exciting new, new direction. Uh you know, I'm the, the what will surprise me on the downside that's always hard to predict, right? But it's always gonna be disappointing when it happens because you're like, oh, I could have sworn this thing was gonna work out but it didn't. And you know, I think this is part of the nature of kind of a a research driven but still fairly close to product kind of development framework is you're constantly trying to bring new research innovations into the thing you're building at the largest scale and make sure that those things have been de risked at a smaller scale. And sometimes those interactions, you can really predict really well. And we, we do our best to sort of scientifically rigorously evaluate how they're gonna interact. But when they scale up, sometimes they interact in ways that you can't predict. Ok, let's, let's get another question while we still have time. Let's go right here. You got, hey, Healy Ier boom pop. Thanks for being here. Um One thing I think about a bunch is the fact that education in the States is like we're 60 or 70th out of 194 countries. And I think with access to A I, it's wonderful because as internet and mobile phones are around, everyone can ask questions and get answers the trick, I guess. And this is kind of a very high level question which I think is the stuff you think about probably is if you're getting the answer all the time, how do you also balance the responsibility of giving that person who might not have education to think critically? Do you think about the answer? And how do you caveat? How do you say look a I may be right? May be wrong. And how do you make sure that I guess critical thought remains center in humanity as A I becomes more prevalent. A good question. Yeah, it's a very good question. First. I think there's huge, huge potential for A I to help in education, right? We know educational outcomes when you have a personalized tutor are two standard deviations away from those when you have a much more sort of group focused setting where you're not being challenged at the level of an individual and, you know, someone who knows what, you know, and knows what you don't know. So I think that's the potential. I think, you know, part of deploying these systems in education if you want to have the system really challenge people in how they learn best and make sure that they're actually mastering the material before they sort of go off, but it does enable them to kind of take the right educational path for them. And I think that's, that's hugely huge potential. Yeah, I just want to ask a quick, there's been a lot of concern that the use of generative A I is churning out a lot of misinformation. A lot of poor quality content. Google obviously is a company that its entire business is based on finding good quality information on the internet. And there's a lot of concern about the poisoning of the information ecosystem. How are you looking at that at Google and how you're concerned that this technology you're helping create is also kind of rooting the very thing you built the business on? Yeah, I mean, I think this gets to our responsible A I principles like we've, we've always focused on how do we roll out A I technologies in ways that sort of benefit people and that sort of minimize things like harmful bias or misinformation or other things like that. And so it is true. These models can create misinformation. Misinformation though is not a new problem. It's something that's existed for years and years and, but A I also gives us tools to battle misinformation so you can do both. I'm afraid we're out of time, but I want to thank you all for listening. It's been great, Jeff. Thank you so much for joining us. Thank you for having me.