Prabhakar Raghavan, Senior Vice President, Google 
Interviewer: Brian O’Keefe, FORTUNE

Brainstorm Tech 2022: Organizing The World’s Information

Conferences

Brainstorm Tech 2022: Organizing The World’s Information


Hey. So Alan says good morning. Um, I want to start with a quick question for you. How many things have you googled this morning so far today. What 45, would you mind sharing the last thing you googled? Um oh boy, the last thing I googled was uh, someone said uh, this thing is running on tech me, can you go take a look? And I'm like, okay, I don't know what it is. So I went and googled it. We'll find it. Of course. Exactly. All right. So we had a chance to talk in new york a couple of months ago, which was really interesting. Um, and you gave me uh, you know, some insight and previews into some of the technology you're working on and the advances in search. And I want to dive right into that for instance, you demonstrated um this new type of image search that you're marrying with uh the shopping function on google where you can take a picture of an image or even of a pattern and you and connect it to something that you might buy. You know, something that you're looking for. Could you just talk through the technology that's behind that, the advance that that represents and you know, where you see that path going. Because I think that illustrates sort of, you know, some of the progress you're making research. Okay. I think the example I gave you was I took a picture of a rug in the room and said, uh girl you should help me find blind matching, which may not be a testimonial to my credentials and interior decorator. But uh, the technology behind this is amazing. Imagine allowing a user to express their intent, but taking an image and appending to it, the need for blinds or curtains. Uh, so that level of expressivity requires first of all, a ton of underlying technology to be able to take the description of that rug. A picture is worth 1000 words translated into a query append to it blinds and be able to shop for blinds matching the drug. And that's brian that's an evolution of the expressivity that we're bringing to users over the time. If you go back 25 years, you met Mandela, the artist era, Anyone here remember, I remember there's a couple of people here who are over the age of 30, evidently. Uh, so so back then your type queries like pyramids height and you've got a slew of blue links that contain the world's pyramids in height and you hope for the best. Then we opened it up, especially using voice to let users be more expressive and they would say things like what is the height of the great Pyramid of Giza. And you had, they expected you to say 455ft not regurgitate a whole bunch of pages. So we opened up expressivity for users using natural language understanding, uh, Voice became a powerful driver of that in some countries, we're seeing 30% of all queries, voice queries, right. And the next step in the journey is images and being able to search by image. But really the practical use cases like the one described. Yeah, well, just to stick with the voice thing for a second because another thing that you, you that we talked about that's really interesting is that younger users are using search in different ways and sometimes surprising and sometimes challenging to your, your algorithms. But maybe you could just, you know, elaborate on that, how people in the developing world, for instance, for instance, are using search in a different way and how you have to revise the we all keep learning over and over again. Is that new internet users don't have the expectations and the mindset that we have become accustomed to. Right? So, so I mentioned that if a third of your queries of voice queries, that's not typical in uh, in Western europe and in the United States. Right. So, uh, having rich expressivity through voice that comes naturally to them. Uh, the other thing is as a result, the queries they ask are completely different. They're not, You know what you and I used typing 2.6 keyword queries. They don't do that. Uh, beyond that, younger people don't come with preset expectations. So I'll give you just two examples in our studies, something like almost 40% of young people when they're looking for a place for lunch, They don't go to google maps or such. They go to Tiktok or instagram. So think about it. Right? So the journey begins in different forms than before, visually rich forms. Right? I'll give you a different example. Uh, and this is some has to do with my maps, responsibility. People are coming online for the first time, but it's young people, new users in emerging markets, they've never seen a paper map yet. Maps products today looks like a paper map stuck on the phone, which is wrong. Right? So we're not meeting the user's expectations. We got to change so they don't have the contact that you or I might have. That's right. They don't have this notion of the projection of and so we have to conjure up completely new expectations. And that takes altogether new technology underpinning this and I want to ask a follow up about that. I just want to remind everybody, I'm sure that there are people with questions for Prabhakar. So we will have some time for that, raise your hand if you'd like a mic and we'll get a mike handler to you and then we can call on you. So I know that you recently announced, um, some coming advances in map technology, more immersive types of maps that combine, uh, you know, augmented reality and maybe you could explain how that works. Let me give, give you a preview of how that's working and it's still early stage. Right? So the first use case we tackled was one that may be common to some of you, you're in a large city for the first time? For me, it happens anytime I go to London use the tube, I come out of the tube and maps is telling me to go that way except I don't know which way that is. So we brought an augmented reality uh, with an experience where for directions, you hold up your phone and it figures out exactly which way that way. So you don't walk 50 yards the wrong way and then turn around, which is experience. I'm sure you can all relate to, Right? So that's a powerful use of augmented reality in real time. In fact, it became so compelling that users uh, using it would walk into other pedestrians. So we had to introduce afford ince's to make them put down the phone every now and then to look where they were going, right uh then we would all put down our phone every now and then. Especially when walking in a busy city as somebody walking around new york a lot, we have to avoid people. Good idea. Uh, the other thing we're doing is bringing in three d maps uh, for, to begin with in five cities, but later this year we hope to get to 50 of key uh, points of interest. So that if it's Hyde Park in London for instance, you can get a good preview of it. Get a good sense of what it's like, is it crowded etcetera before you even get there and augment that with real time information like how busy is a place and this is actually super important, especially during the pandemic. People are very concerned about whether the place is too crowded and so on. So bringing all that information together leads to some very exciting experience. So you're talking you're using the word augmented a lot, we're talking a lot about metaverse here at this this gathering this week. Google is not talking so much about metaverse. You're you're you're advancing the technology in ways where it's we're going to have an altered or enhanced form of reality. But how do you think about that? The augmented versus you know, the metaverse, like totally digital realm. Yeah, I don't think it's essential to presume that you're restricted to an alternate universe. We still want to serve users in the real world, but using these technologies of augmented reality that we've actually been working on for many years, some of you have heard of google class that we were working on over a decade ago. So the idea is to bring these A. R. And we are techniques really to deliver services to users in the real world. So, you know, a lot of what you're describing is a I behind the scenes. I think people don't really see all the furious paddling below the surface that goes on with with with developing search, but like you've been, you know, you're an academic for a long time before you took over this role running, you know, a giant company, a giant business operation. Now, what you're seeing with ai and how it's combining with sort of like connecting the dots of what you're talking about. What can we expect of where searches going like over the next couple of years and how we're gonna be using it and how it's gonna be, you know, interacting with how businesses use it as well. Yeah, I'll give it a shot. Right. Uh you're absolutely right that the underpinning of a lot of this is fairly complicated machine intelligence that very few people actually understand. Uh just the notion of language. Do we really understand language? Uh we give the illusion of understanding language, but we don't actually uh as the models ai models get larger and larger, we get deeper and deeper understanding of the nuance of language so that we can tell the difference between, for instance, ah what causes Alzheimer's disease, to what is the most common cause of Alzheimer's disease. Right. And being able to tease out these nuances uh requires very fine grained understanding of language. And you've seen us announce a lot of breakthroughs and natural language understanding that I think will let users get more and even more expressive about their intent, Right? So that's one trend that I expect to see continuing the other trend that absolutely will continue is users bringing in more rich content, images, the analysis and understanding of video to the same level we did with documents. So I'll give you an example if you go to google now and say how do I change a flat tire? Well in the old days we used to give you back pages and then we started showing you videos of how to change a flat tire. But now we've actually analyzed the video to say, well seven seconds in we tell you about loosening the lug map 11 seconds and it's about raising the jack and on and on some of the hubcap etc. So getting that level of deeper understanding is the journey was still on, right? But we want to get to the point where you should be able to say, how do I do? You know? So if I'm stuck on lug nuts, um, to the exact point right, and all of this requires prodigious amounts of ai uh, to get done, raise your hand for mike handler if you have a question. So, you know, what you're describing is spectacular and amazing. It's also, you know, the more that the algorithms understand us and understand what we want even more than we want ourselves. You know, also tends to raise the question of privacy about how much you know about us, how much data you're you're processing about us to reach these conclusions and obviously google gets a lot of scrutiny for that. How are you thinking about it? How do you, how do you sort of give us what we want without exploiting it? And you know, what's the, what's the current state of thinking in google about how to do that? That's a great question. Uh So thank you for that. Let me begin by getting uh putting out of the way the obvious Google does not sell your data, your search queries, but then I'm going to give you a personal anecdote that seems to fly in the face of this. Uh some weeks ago, my wife at home searched for Roomba vacuum cleaners. And then for some days after that, whenever I was at home, I would get ads for Roomba vacuum cleaners. And you say, aha, caught google selling my wife Square isn't also somehow figured out and I was my wife. Right. Actually, no, that's not what happened. Right. Uh So step back, maintaining trust on the internet requires robust privacy prince. How do you know? It wasn't what happened uh because we don't celebrate as you know what you do and don't like, we don't know, but but uh you know, if you think about a bit about it, uh you look at what's going on behind the cows, uh there's actually players in the industry that fingerprint hardware devices in this case, the router at home and use that to track users across the internet. Uh This is a fairly unfortunate practice. It's a shady practice because users aren't aware this is happening. They aren't told this. They have no control to opt out of it. So that's not good. We think the answer actually is comprehensive privacy, like regulation that shuts these practices down because that's what you need to restore the public's faith in the internet. And so so we're all for get you know, so you might might surprise you to know that I'm arguing for regulation, but I think that's the right answer. Uh, so big journalism scoop potential there. Who's who's tracking uh, baca's information. But also, so you you favor privacy regulation. What do you think are the chances that there's going to be movement on that? In Washington? I am optimistic. But in that there'll be as much moment in Washington as editor's sounded tentative when he said he was optimistic. I think the emphasis was on movement in Washington. Yeah, right, let's um, tackle quickly a related topic misinformation, which is on everybody's minds. Uh, we have fortune just around a big story about misinformation and disinformation, which is permeating all parts of our lives. And and as usual, people are looking to google for leadership in the digital realm on this? What, you know, what are your internal discussions? What are the actions you're taking to address misinformation to help people guide, you know, get guided through to accurate information. This is a great question, because I often get the question, why don't you just detect all the misinformation. Just shut it down, right? Uh, if I knew how to do that, I would The problem is what philosophers call the relativism of truth. So let me give you an example. If you tell, if you ask google a question, how far is it from When they like to Paris? We would give you the answer 10,000 something km. Right. If you ask, what's the Communist cause of Alzheimer's disease will say, well, we really don't know because we don't. And the reason we don't know is the way we answer all questions is by assembling a consensus of historically expert opinion, historically dependent right on on the distance between manila and paris. There is consensus That we can assemble and and all the computers 10,000 so many kilometers on the question of the communist cause of Alzheimer's. you're not able to assemble such a consensus. Now, what that means is there will be cases where we have to admit that we don't know the answer. And so we have additional metadata on every search result that says about this result and you can go in there and explore whether it's from a reliable source, how long the sword's been around, whether it's a secure site, etcetera. There will be cases when events are developing so fast that it's impossible for us to compute what's right and wrong. So was this from two months or so ago was the hospital in Mariupol bound by Russian forces, right? And until consensus was established, we would put up a banner saying results here are changing really fast. We don't know. So treat the following with caution. Right? So that's our way of telling users caveat emptor. So there will be cases, we'll have to do that. How many people are involved in that? How automated is that system to determine something so subtle and complicated? It is by and large gets done mechanically. But there'll be difficult policy choices like when the european union said, you need to get rid of all Russian State Media, right? We had some difficult choices because we do stand for free and open access to information. If you're a journalist, you do want to have access to Russian State Media, we don't think we have the right to shut it down, but we get we get a court order, we will do that. So in europe, we actually took it off and reported to the Harvard uh, lumen database. Right? So those are some difficult decisions. We had to thread right. In other cases, you know, it goes pretty academically great. Well, Prabhakar thanks so much for joining us and for shedding some light on not only what's happening with search now, but where it's going really exciting stuff and thanks for being here. Thanks, Take care