Freedom Dumlao, Chief Technology Officer, Zipcar
Alok Gupta, Head of Data Science and Machine Learning, DoorDash
Moderator: Dr. Rana el Kaliouby, Smart Eye

Fortune Brainstorm A.I. 2021: A.I and the global marketplace

Conferences

aI and machine learning techniques underpin the flourishing ecosystem of app based on demand services, helping organize informal markets and leading to new forms of competition, all while providing expanded opportunities for independent app based workers having weathered dramatic shifts in demand as covid lockdowns came and went. Executives from the center of the collaborative economy share how their companies are working to improve the user experiences and enhance profitability. We have freedom. Do not hope chief Zipcar is chief technology officer and a lock Gupta, head of data science and machine learning at doordash. Please welcome him to the stage together with dr rana El Halabi. Yeah, yeah. Welcome. Great to have you here. Um So we're gonna start with some questions and then again we'll open the floor up for a conversation and questions from all of you, including our virtual audience as well. Um I'll expect start with you. So how has door dash pivoted during the pandemic? And I guess the role that a I played um as you thought about all of this, one of the things we saw when the pandemic started was uh an increased demand by people at home for food delivery et cetera. At the same time it was coupled with a few people commuting less traffic on the road. So what that meant was we had a surge in demand and all of our food um prediction times for how long it would take to deliver an item. Got a little confused because it's far quicker to travel the roads. So we have to readjust all of our models to instead of training on, say the last three months of data shrink it to the last two days or two weeks of data. And so it's a scramble to go through all of our models, what do we need to update? Um That was one thing, the second thing was there was a a really big demand from users to access grocery and convenience items from home more than than there had been. So we have to accelerate some of our ai there in problem areas. We haven't really done much before in the past, like predicting inventory, recommending substitution sort of problems that you don't see in the food delivery space, but you see in the grocery delivery freedom. How about you? How how, I guess how did the demand change during the pandemic? And then as we came out of the pandemic and power, were you able to leverage ai for that? Yeah, it was, it was a pretty dramatic shift, you know, similarly to what you were speaking to look all of a sudden the traffic disappeared. Uh Some of that was, people just simply were staying home. So at the very beginning of the pandemic, nobody was driving, nobody was doing anything. Um and then all of a sudden everyone was driving again, at least everyone was getting into zip cars account for us. Uh that came as a shock because it was sort of all at once. There wasn't anything that really helps us to predict that that was about to happen. And then once we saw it happening, we noticed right away that the the types of drives and the driving behavior that our members had were completely different right before the pandemic, people would wake up in the morning, Maybe they would take a Zipcar to the office or to do their daily chores. Um but people had all decided to, you know, people were all forced to work from home, uh which meant that they were spending more time where they maybe had an opportunity in the middle of the day to do something different. Uh And so the models for how people were using cars dramatically changed during the pandemic. And even now, even as we've seen some offices reopening, we're still seeing lots of people who are still, you know, staying at home. And so the way that they use our our cars has changed. Uh fortunately before the pandemic, we've made some pretty big investments in uh some systems that helped us to predict what we would need in terms of positioning vehicles to make sure that every member had access to a vehicle when they needed it. So we were able to tune that system once we started seeing how people were driving again and get those predictions to be back on target so that members could get access to vehicles when they needed them again. So this is very interesting because a lot of machine learning models are very data hungry. But if you can't use historical data, how do you gather, I guess. How do you get enough data to train these machine learning models accurately? I think at first one, doesn't you? Except at the beginning there's gonna be some noise and um elevated levels of inaccuracy and you sort of use that cost or that budget to explore the data space maybe more actively than you would otherwise to then build up your training set, then you can then take advantage of and train the model. But yeah, there's no shortcut in the beginning of a lot of trial and more, more error than usual. Yeah, for us we were able fortunately to to leverage some comparable models, so for example holidays and vacation weeks and things like that helped us to bridge quite a bit, but you know, we did have to sort of drop that time period of like the bizarre, nobody's leaving their house anymore time period, all of that was basically useless for us as we restarted. So we had to sort of skip that period in time. Hello, can you share some of the more creative uses of ai that you've seen within doordash. Yeah, so one thing we see with a lot of our restaurants are merchants on the platform. They're opening and closing hours can be quite dynamic and if we send an order when they're not open, it's a poor experience for everybody. So um one of the functionalities, our dash have the dash is the folks at doordash that pick up the item from the restaurant, deliver it to the customer is when they go to a restaurant and they see it's closed, they take a photo to verify that it's closed and to help us process and adjust our timings. We used that bank of photos and we saw that was actually pretty good information. They're using computer vision, we could actually read the updated opening and closing times and use that to dynamically update the restaurants our um, and other use cases based on some of our best performing dashes. We're able to improve our hyper local understanding of restaurants and residences to try and surmise what are the best Routes or places to park and then routes to walk to get inside the restaurant inside an apartment building and it may seem trivial, but shaving 10 seconds here, five seconds there really adds up to improve the experience for the dasher, the customer, the merchant, everyone. And so we're going that sort of into that level of detail. So I want to build on that. How do you build an organization? And I guess a machine learning architecture that allows, you know, people to experiment on the platform? Like, can you talk a little bit about the architecture underlying all of this Zipcar. Sure. So at Zipcar, the uh, creating this, uh starts off obviously with member privacy. Right? So we want to build a great experience for members, but we're not willing to compromise on member privacy. So how do we make it possible for our engineering team to uh come up with creative new ways uh, to solve problems without uh providing them access to information that they just simply shouldn't have access to. So the first thing we take into mind is that then the next thing is we figure out, you know, what are some of the ideas that we have out there and who's willing to pick up the banner and charge ahead on those initiatives? So we have several Zipcar almost all around improving the member experience, whether that's making cars available, uh, making sure that the cars that are available have been properly serviced and are getting the attention that they deserve. Um and also making sure that our fleet associates who take care of our cars are getting scheduled to do the work appropriately and quickly. So doing that, simply dividing it up into those multiple missions allows you to make sure that you're focusing on those different areas. How about you? Um, I would love us to move to a world where non data scientists, non ml practitioners can start to experiment with their own feature ideas or their own objective functions and loss functions, etcetera. We're not there yet and some tech companies are there. Um, but first and foremost, I think it starts from driving a i hypothesis driven culture of you want to try this? What's the hypothesis? The hypothesis usually starts with something like we believe we can improve the consumers experience by offering them this feature or this experience and then, Okay, what's the data we need in order to create this feature and then it goes from there. Um first we're starting with the culture, so I want to actually talk about that a little bit. How do you get, I mean, I guess how are your teams organized? Is it organized functionally? Is it organized by products? I feature um and a related question, how do you get, you know, these teams that may not be domain experts in Ai um on board with leveraging AI and machine learning freedom. So at Zipcar were organized by mission. So we have multiple mission organizations, each with their own uh dedicated goals that they're trying to achieve and that gives us the ability to focus on specific objectives that we're trying to come up with. Um now, as I'm sure many folks here who have tried to build an Ai or data science organization, No, uh, those individuals are difficult to hire and hard to find, so where possible. We like to embed capable people on each mission, uh that's not always possible. So the folks that we do have tend to work with the different mission teams to make sure that there's some resources available. Another great idea though, I think, look, something you mentioned is, you know, I'd love to get people in every organization, uh every part of your organization have access to some of this. And something that I did a company previously was, I created an initiative called Machine learning everywhere and we partnered with a one of these um digital learning platforms to provide a course. Um and I think it was Andrew wings course actually uh to anybody who was interested in the company. We built a program with accountability but also some incentives for folks to do that. And it gave us a machine learning competent engineers that we could we could place all around the company. So everyone at least had a working understanding of that and that worked incredibly well for us at that company. I love that idea. I might steal it. I think we're very similar. We have a centralized hiring, onboarding, performance management, career progress um process. So all data scientists and managed are hired by the same team, but we're extremely embedded in the different mission areas or cross functional pods we call them. And so the data scientists on my team, I tell them, yes, you belong to this data science group but your primary team and those people with whom you share a business objective and seats you should be meeting with them daily working with them. They are your primary team, you're embedded with your product manager Nge Partners but we use the centralist process to ensure a bar of quality. Uh but they're embedded to meet the business objectives. And I would imagine to sharing tools and resources and data warehouses. Yeah, exactly. And that can come at a cost sometimes where people in these different pods hyper embedded don't end up showing as much as they would if they're all essentially managed and project to project assigned. But yes, I think. Alright, let's open the floor up for questions. Do we have any Okay, while people are thinking about their questions, um how do you, I mean, what is really amazing about what you do is you have multiple stakeholders. So how do you use machine learning slash ai to improve the customer experience for these various stakeholders and freedom? Why don't you take that first? Sure. Uh So for us it's, you know, I mentioned earlier, uh are our greatest stakeholders are members. So, and the best experience we can give them is access to a vehicle, right? Like that's the thing that they're coming to us for. Um so we use it to make sure that we understand where we need to have vehicles positioned so that members get access. But we also have stakeholders who are our partners who help us with servicing the vehicles as well. And so we also use machine learning to make sure that we're predicting when a vehicle will need maintenance or any other type of service before it becomes something that's an issue for the vehicle. So those are two key ways and then of course, I'm sure there's others for us, I think on the customer side because there is so much to discover on the door dash, consumer app, food, groceries, convenience, etcetera. It's how do we personalize it? So we push the information they're most likely to want to the top for our merchants. It's how do we make them as successful as possible? How do we help them curate their menus? And then production is how do we help them meet their earning objectives in a way which suits them? And then there's everything else support fraud marketing optimized. And there's, there's a bunch going, you need a hunger detector for your, for your okay. We have a question here. Yeah, rick chavis, my octopus um, question for a look really both of you, you both have an opinion and that is around the idea when you're talking about making a i accessible to, you know, the more casual user or business user, um, what do you think the future is for no code ai you know, to really be at an enterprise scale uh, as opposed to a tiny use case, uh, applicability. I think it's coming, I think we we can progress to a world where we have sufficient guard rails best practices, um, to avoid anyone doing something which is not statistically robust. And so for 95% of use cases we can say to a user, if you have this idea, go ahead and try it. Here's some data you can pull something you want to predict and there's enough guard roles in place where um, you know, you you won't do anything. Uh that doesn't make sense. I'm pretty hopeful. I think there's there's plenty of tools we're already seeing emerge in the NLP and computer vision space as well that are really starting to make it possible for hobbyists, even people who don't have any formal software engineering training uh, to get access to these capabilities. So I think it's inevitable that for the common case is we'll see commoditized set of tools that are available off the shelf. Uh and that's, you know, going to help us to propel into the next tier of problems because our our engineers, our scientists are going to be able to focus on those problems which haven't already been solved next. Yeah, high vocational is steadily black and decker. So this is about Zipcar alright, what can you share about your plants for auto over striving, what should be out out of us. So we're keeping an eye on it. Uh, you know, autonomous driving is something that I think is an inevitability. Uh but for us it's really important that whatever experience we offer to our members, it's something that's safe. Uh and something that we we trust. Um today, I think we're learning a lot about, you know, where autonomous driving works well and where it doesn't and we're keeping an eye on it. Um, I don't, I don't think we have much to share about any upcoming investment, but it certainly is on our radar. Have a question for you. So we do a lot of work in the interior sensing space. Um, so I would imagine one use case or application would be, you know, is the car clean? Are there any objects left behind? Have you thought about that at all? We have actually, so that is an area that we are making some active investments in right now. Uh, you know, when we are the world's largest car sharing network and that means we have over a million members and some members are a little bit more courteous than others. And so for us it's important that we, we find ways to make sure that the next member who gets into the car has a great experience when they sit down. Great. Okay, we have a question here and then we'll come. Hi, my name is Samir with Snorkel a I had a question for you. It sounds like uh iteration is a key part of your workflow happened with Covid something to learn pretty quickly. I'm curious what parts of your ml processor ml flow from indiana pipeline template ties and made it pretty modular to quickly change adapt and experiment with. Uh, some of the things we can do our things like features sharing. So we can have one team building predictive features for one model type, like no estimate the food preparation time and then we can have a an orthogonal team which is trying to predict when the food will arrive in the consumer app. The E. T. A. Is you see and they can share features across. So features sharing is one model Architectures, something we can share across James as well. So embeddings for food discovery preferences is something we can share across food, grocery delivery uh et cetera. So I say those are the two biggest use cases I imagine using AI allows you to iterate an A. B. Test a lot faster. Yeah, we're an experiment first company. So um highly unlikely we will do anything without it starting as an A. B. Test and slowly rolling out to make sure everything looks good question. Hi the miles from Gap. So uh Great. You talked about feature engineering that's very close to my heart. Can you talk a little bit both of you? What's your approach towards the feature store? Do you have one enterprise feature store that everyone can contribute to? You have a Wikipedia type approach? And how do you democratize the creation of features yet have good governance? Uh So for us we, you know, our our feature store looks like a day to Mark um with highly cleansed data. We make that available to people who are working on projects that we've assigned to them. I would say access is granted to very specific individuals and then we're very careful about what data we allowed to be um scene from the data, mark, et cetera. Um The feature stores are based on you know, mostly on our assets which are our vehicles and what's happening with them. And so we look for that. We always look for new features that we can enhance that with both. Internal from sensors inside the vehicle. Uh sensors that are are sensors is the wrong word, but external data sources, for example, things like weather and traffic conditions, et cetera. We also add uh to that and make that available as well. We have a centralized feature store uh across the company for both real time and batch features that we can use and it's very democratic, anyone can contribute. And again, people with access can view it's already there and leverage it for them. That's right, yeah, that's correct. Yeah. So features can come from from nearly anywhere and not really other than obviously, you know, polluting the store with invalid data. No real reason to to prevent anyone from contributing to the data store. Um I'll tell you some of the problems that you see as you scale up though is is the ability to uh extract data from a from a store like that uh quickly enough to be able to respond to things that are happening. So uh if you're building a large scale data store like that, you might also consider adding a capability that allows you to extract data as it's being written into the data store so so that you can start to get enough data to construct models on. You don't have to wait for long running queries to pull that data out. This is all we have time for, but thank you both for joining the conversation and to be continued. Thank you. Thank you.