Name: Don’t buy the ‘big data hype’ says co-founder of Google Brain
Uploaded: 2021-07-30
Duration: 13 min 23 s
Description: Andrew Ng says A.I. development needs to focus on quality of data, not quantity.

Andrew Ng says A.I. development needs to focus on quality of data, not quantity.

Don’t buy the ‘big data hype’ says co-founder of Google Brain

Tech

Don’t buy the ‘big data hype’ says Andrew Ng

many of the uh folks in the, in the room today are entrepreneurs who are starting smaller companies and are looking at these giant companies uh you know, uh by dance, uh amazon alibaba 10 cents that are now very well established and controlled vast sets of data. And they're thinking of their own companies and then wondering how can I possibly compete with giants like that who controls so much data? If the advantage in the age of data always goes to whoever has the most data, you know, we always heard here is it sprays the data is the new oil. Um how do smaller companies compete and use AI to grow their business if they're not able to command those vast pools of data? I think that the importance of giant data says may have been a little bit overhyped having lots of data is great, but the value of data is also very vertical is so maybe you have tons of web search data or tons of economic data but that Yukon data is not very helpful for finding defective parts manufacturing line or not very helpful for understanding medical records. So I think for the many tens of thousands or maybe more of possible air applications there is actually a need and an opportunity for many new businesses and companies and products to be built using data for those verticals which maybe doesn't actually exist in large amounts in anywhere including the large internet companies. It's really interesting. You know, um we've talked a lot of carefully about this and I'm sure you're familiar with his argument about, you know, we're now in the age of implementation not the age of innovation when it comes to big data and ai and that if he argues that as long as you control a lot of data and can just crunch it endlessly. Um you win. And it doesn't even matter if you have really brilliant data scientists working on the data, they just have to be kind of competent and that it's kind of a numbers game and um you know that I, well I hear you you sort of have a different uh different view of the landscape. Yeah, well there's lots of options for implementation. I think there are also lots of options in addition. In fact right now, one of the most exciting shift I see in a I is I remember 10 years ago, starting 10 years ago, you know, a bunch of people including me trying to shift the A. I. World to use deep learning. I think in this decade there's another big technology shift coming which is a shift from um model centric data, centric aI. And what that means basically is we're used to a lot of companies taking whether the data, they have just running A. I against it. But I think that there's a new wave of technology were probably tens of thousands of innovations, research papers, patterns, inventions to be had two special ways for businesses to acquire very high quality data. You know, a certain with C suite in many industries and what I realized was that re collectively in AI world have a certain playbook for getting A I to work in consumer internet companies, companies with hundreds of millions or sometimes billions of users and tons of data. So there's a certain set of things that work that's creating tons of value. But once you look outside consumer software internet, I think the number and scale of practical Ai deployments is just not yet there. And there's actually a new playbook and new set of tools and technologies. So I think we'll need because the challenges in these other cities are quite different. That's quite interesting. So your senses that we're really kind of the early stages of understanding how how ai can work for. I mean it's not just necessarily even smaller businesses. Right. I mean, the uh types of uh analysis that an internet company and consumer facing internet company would require are quite different than say what a manufacturer or uh you know, even if it's a large manufacturers might use. Is that correct? Yeah. So I feel like over the last few years I've been seeing the same problems over and over hampering their adoption and everything from manufacturing, healthcare, agriculture, traditional industries and more. Um I think one datasets are much smaller. So if I'm in a manufacturing plant trying to detect stretches on smartphones that does not exist anywhere in the universe, A data set of a million pictures of stretch smartphones because fortunately has not made that many stretch smartphones. So I want, you know, build computer vision systems using hundreds of millions of images. That technology does not work. We have 50 images. So I think the new data essentially philosophy, the Ai is critical new text to make things work with smaller data because all the data that exists in the world. Um, I think that a I also has a long tail customization problem. We figured out how to get, you know, maybe 200 engineers to build an online that system that generates us dollars, billion dollars worth of gravity or more so that works. But once you're in manufacturing, healthcare, agriculture and so on. What I see is instead of a handful of monolithic AI systems that serve massive numbers of users. I see instead tens of thousands of maybe $125 million project, which in aggregate incredibly valuable. But you cannot have 200 engineers go and solve a $5 million project. So one trend I'm excited about for investors is the rise of M. O. Ops to the machine learning operations tools that makes it possible for teams to be much more efficient in building these, you know, tens of thousands potentially of custom Ai projects with an aggregate will create tremendous value. So that solving all of these little customization problems efficiently would create maybe even more value than one or two more realistic things. Uh, and and finally, I think we also got to get better at bridging the proof of concept to production gap was learned that in the I team can get a prototype, our demo, maybe even a research paper working that that research paper or their prototype with a demo to practical production. Even my teams used to take, You know, 12-18 months to take things to production. And then again, I think the new generation And then the last two I think will make that 10 times faster. So I think a lot of, a lot of opportunities for entrepreneurs. So investors, I think the big companies will also jump into those games. I think there's an open field to try to build and realize these use cases. That's fascinating. So if you're a company that's at an early stage, just get excited and you're looking at a kind of revenue, as you say, opportunity of In the beginning, you know, maybe it's five, maybe it's $50 million, maybe it's even, you know, $500 million. What I mean, what sort of an A what size of an AI team should those companies be thinking is is adequate using mls as you describe? I think um maybe one framing is um if you project into the future, what percentage of the software engineers? You know? I. T. Data scientists. Software oriented people should be A. I. People. And I think it should be a significant fraction of those companies. Uh is not 100%. I think there's gonna be less than 50%. Maybe. Maybe not that far From 50%. I think it varies a lot by companies and industries. How to give it one size fits all. You think that in the future the material faction of all the software people in most companies are software engineers will be a. I. Data scientists. I think that would be one way to plan it out recognizing that there's gonna be a lot of variations. It's quite interesting. You know, again, if you read his book, you know, he presents a picture of the technology almost gonna reaching a plateau leveling off. And but you can also kind of think of an alternative of as you're suggesting here that maybe there are these kind of multiple breakthroughs that small breakthroughs that proceed simultaneously. Is your sense that we're in a kind of period of relative stasis in terms of our understanding and our youth of ai and deep learning? Or do you think that uh we're on the cusp of yet another kind of punctuated equilibrium, if you want to call it that a kind of period of rapid breakthrough, There is one rapid break through that I'm excited about, and I see a lot of innovators playing with and that is the shift data centric ai. So it turns out that the way a lot of a i research was developed over the last not just 10 years, the last maybe 30 40 years was, researchers will download the data set uh and then you try to tune the software to work well in that data set. And this is great. Just drove four tons of A. I research. But thanks to this mode of development over the last few years, especially with the rise of deep learning, what I think I find that for a lot of practical applications, the AI model or the code, the neural network architecture is basically a solved problem. But what is missing is how do you get good enough data to feed into the open source neural networks? You can download off the internet for free. How do you get the data you need? And often accustomed data, you need to feel that in your network. And you know, I remember to be maybe maybe overly technical. I remember when a software version control was a very manual process. My friends and I would email C plus the files as email attachments and say, hey, I'm done everything that you edited now. And so we knew software version control was important, but until we had tools to help manage how we edit software, it was very error prone. So I think today we all know data is important in the top five days for a long time. But what we're still missing is the tools and processes to quickly get you the data that you need to train in the I system. Uh, maybe one example. There's a lot of work in voice search in english and chinese and if you were to get an audio clip, but once someone saying um today's weather, well, how do you label that? You transcribe it on? Um m comma today's weather or um M dot dot dot today's whether or almost noise. Just transcribing today's weather. I don't know and I shared teams transcribed it inconsistently. So it turns out that it can clean up the data. Just pick one transcription, then your speech system were better. But today we just don't have good tools to do that. And when I go to factories as well, landing A. I. Manufacturing vision inspection, we routinely find two inspectors have been doing the job for years and those students sectors, they don't even agree with each other and what is the defense and they've been doing their job for five years. So how do we get good day to all of these deep domain experience, which is the best qualified people in the world to define what is a stretch cycle and what is not. I think today we all know this is important problem, but we're lacking the tools to help systematically rather than ad hoc luck, hard work turned away systematically edit the data to then make the way I work. It's just as there were tens of thousands of innovations in deep learning over the last decade, I think they're going to be tens of thousands of innovations in this technology piece. And this I think is important from blocking a lot of the I value. And I think this is not just something that technical people should look at, just something that this is the dish. Look at that really has profound implications for both investors and entrepreneurs. I mean, what tactical advice would you give young, uh, founders and and entrepreneurs as they launch companies and get started about how to harness uh, these new technologies. How do they, how should they learn more about it? How can they find the people that they need to be able to, you know, have the right skill mix to take advantage? I think um you know, I think many people are assigned to talk about this. Uh My team development on AI tries to publish materials on Wechat on our website uh to share what we know um in terms of advice entrepreneurs, I find that there is a difference between how one builds A. I. Products versus how one builds traditional software products. And so I think that new playbook is being professional as well. Uh Lots of differences stemmed from the observation that for the traditional software product you need codes, you can use interviews, figure market demand product code and launch it. But Ai products, you need both code and data. And so I see for example, more a I started to go through the nrg route, the non recurring engine of the consulting route uh seniors in multiple. Now very large chinese companies that go through the NRB routes, which is not scalable to get the data you need. But I find that the playbook for building a I project is long different than traditional software. So actually, we we we've been talking about this in some of the materials we published at developed AI uh as well.