Name: Brainstorm Design 2019: Smart Solutions For The Visually Impaired
Uploaded: 2019-03-07
Duration: 18 min 46 s
Description: Fortune's Andrew Nusca interviews Chen Zhao, Research Director, International User Experience & Design at Alibaba

My story is about. This lady her name is Monty. She lost her vision when she was six. She lives in Beijing right now and has two jobs one day. I was chatting with her. I just ask her Monty. What do you usually do and we can. She told me, walk on the street window shopping, running, and watch a movie. I was surprised when I heard that she said watch a movie, because I never saw that she watched a movie. What I learned from that is she tries to Live like, you know, as a daily life we have but what have we done to enable her or support her to live like us, a daily life like us, independent delay. Mobile phone probably is the most Frequently use the device of the world. As a researcher in the HSI community, the community is proud of the graphic user interface. What you can see is what you get, but what if you can't see? There are so many challenges for Mengqi, and millions of people like her, blind people. Actually visually impaired doesn't sound nice in Chinese. They prefer for me to speak, to say blind people. So, thanks to Google and iPoll, they at least provide You know, accessibility technology. With our talk back, for blind people they can touch the user interface and then read. So actually they do read the UI, the code underneath the UI. But most of the engineers who write the UI code, they don't know the accessibility depend on that. So many times the [ UNKNOWN ] components cannot be read, the User Interface cannot be understood, even you know what's on the interface, it's very difficult to complete tasks, just imagine, look for a button Look for an icon on a specific position on a small screen, and you need to click it. many times tried, many times mistake, you may get it. Okay spend three weeks to learn an app, and then the mobile app can be updated quickly So Monty has to start over again. That's all the challenges for people who can't say to use mobile phone. So what's our design to help them? [ BLANK_AUDIO ] We call it Smart Touch. First we have a UI proxy which can understand the user interface components and they radiate out much badge. And the most important one is, we design this button. We gave a name called Braille Buttons, so these buttons actually are designed very carefully. On the left side there are three haptic buttons, which always provide the same function. The first one is a way to use the interface Stop ready and go back to the homepage on the right side. The three buttons actually are dynamically changed based on the different apps different user interface. There was a long way to even design the patterns, right ?. And this is not my work. This is the team's work. The idea comes from the Professor [ UNKNOWN ] Tsinghua University, part of our joint research lab. Currently, we're working with a group of engineers and product managers. At Alibaba eight math. If you do not know eight math which is largest navigation app in China there are already seven million users in China. So we are working together to deliver this E- Math. In Chinese, called a math to help the people, the lady on my picture, she can go out, she can take a walk in her neighborhood independently. So, next I'm going to give you a demo to show how smarttouch works on e- math. [ BLANK_AUDIO ] Danielle. [ BLANK_AUDIO ] Can I just say I think it's wonderful to see the entire team up in your presentation, that's very nice. Thank you. So this is exciting, show me how this works. Okay, first I'd like to show you this little tube. This one, actually, is designed by a great designer on our team to help even put the [ UNKNOWN ] buttons on the phone. So it's just press and the buttons can stay on the phone. So the buttons adhere to the screen. Is that right? And you use this To I guess accommodate different devices? Correct, it's [ UNKNOWN ] out. Because this one, it's not for business. We will distribute, sell them to the offline store Alibaba in China, so we have to many it works for every phone, different size, different models So once you put the buttons on then what? Pardon. What happens once you put the buttons on? You have to install software. Yes, so you all installed the But as it hear my [ UNKNOWN ] work outside China so I use GPS simulator tl simulate this beat it's a live demo. So we're doing a live demo and we are using soms servers in China right? Yes, okay It should work. So it's talking to us? Yes, that's how blind people use the phone. You touch and it talks, yes, okay? Let me know if you guys cannot see it. All right, can we get this up on the screen? Great, yes. So just tap. So you're opening the app. Yes, so I'm going to translate So, [ BLANK_AUDIO ] [ FOREIGN ] So this one tells you, talks about what's on the user interface. [ FOREIGN ] Stop reading. [ FOREIGN ] Go back to the home page. [ FOREIGN ] Lost internet connection. No. Didn't I say it was a live demo? I did. Okay, let me try it again. Let me ask if it tell where I am right now. [ FOREIGN ] It said, GPS cannot locate it. No. All right, but you were walking through these other menus and you were saying. So these are for the phone. Correct. And these are for the specific app. Right, so if I have the GPS, this one just like search destination. Okay. And this one tells where you are. This one you're like a [ UNKNOWN ] if I touch this one I can search a destination. Then on the next page those functions will dynamically adapt to That to the page. I have gave you another example, because we paired this with [ INAUDIBLE ], and the share actually used this. When [ INAUDIBLE ] took a taxi, took a subway, what she shared with us, she really liked the taxi function. Once you use the app, call a taxi, there are three functions here. The first one is call the driver. The second one tells you how far the driver is to you and the third one is cancel our reservation, in case accidentally, so that's What we're working right now, so next week we are going to pile out this with 200 [ UNKNOWN ] people around 100 senior people in future they are other people like 50 60 years older so they cannot go out because so many cars but we hope with this they help they can at least work a little bit without a little bit and also we will pull out another hundred people. In Georgia, they have a school for blind people to say we will formally launch it in June, but we want to do the iterative design process tests in the field. And hopefully when we launch it it works well. That's incredible, incredible. So You basically navigate through this phone using the audio feedback, using the buttons, and find your way through. I think it's incredible just that you can use such a visual app, I mean a smartphone, just a smartphone itself is so highly visual. For you to figure out a way to navigate this is something I couldn't imagine. Thank you. So I have a couple questions for you. Yep. You are part of Damo Academy. What is the scope of what Damo does? Tell me a little bit about working in this group. You had a bit more of an aligned with the business role. Before, and now it's very research driven. Yeah so Dama has different labs focused on technologies. We have big AI lab, we have a big an LP nature language processing lab. We also have a team to work on the smart chip And also something very front, and quantitative computing. In my lab specialized in human computer interaction, because we think other technologies actually serve people, and HCI is the only field in computer science To connect technology, bridge technology with people. And tell me how you well, as much as you can possibly tell me, it's the lab, you're working on things you can't talk about. What are the different modes of interaction you're looking at? So you're showing me two, right? You're showing me touch and you're showing me audio. Yeah, yeah, Yes, so my team actually works on five things. We think [ UNKNOWN ] technology enable every technology trend. The PC type is more keyboard click. In the mobile time, you know, it's touch. But touch is design for a small screen without the voice we think the new experience, new interaction should really leverage humans five senses. So, we work on the Mixed reality, we work on the haptics. This one, the project is one of the haptic interaction. We also work on something who digitalize the sense of smile. Yeah, are there other, I mean, so this is focused on the world's most popular device. Based from my informal observations are there other kinds of environments you're working on. The other interaction technology we are working on actually crosses different divides. It's an offline store, a hotel. You think about Alibaba economy. We're not only e- commerce, we also have so many, we call it new retail. There is no very clear line between online and offline, so and also our technology could work on Alibaba's Tmall Genie, its smart voice Action as well. There are different scenarios of business used case this technology can be applied. I mentioned your former role. So a few years ago you were the UX Design Director. Is that right? Research. Research Director. That role was much more directly aligned with Ali Baba's business. Mm- hm. You're in a research role now. One of the things that Fortune 500 and Global 500 companies struggle with the most is trying to figure out how to How to innovate within an established company, right? It's the innovator's dilemma, how do you treat RND? How do you see that now that you're in this dedicated RND unit? How much do you think about tying it back to Alibaba's revenue? Yeah, that's a Great question. When [ INAUDIBLE ] was first established, we spent a lot of time to think, what's [ INAUDIBLE ] position? Is it like Microsoft research, IBN research? How is it different? I think in my own words, I think It's more like we believe if you work on some very technology, which bring the new value and the research papers patterns just come up as byproducts. So for our mission is really to innovate new products As well as to benefit millers into users from those new technologies. The technology industry's relationship with accessibility has been an interesting one over the years. Apple, was very early in it, parts of Microsoft were early in it and you used to work at Microsoft. And its current CEO Satya Nadella told me a couple years ago when I was writing a story about him for us how passionate he was about this subject because his own family, his children are affected by this. I feel like We're getting to a place now where more companies are trying to be more inclusive of people with different abilities, it's the best way to put it. Do you feel like there's more support for this kind of work now either because of interest, or technology, or revenue. What's the driver if you think that's true. I definitely feel there are more people who care about this. Though why i start this project is just like one of my five sense. Until I ask [ UNKNOWN ] and then she told to me [ UNKNOWN ] It just made me realize okay, so for people like us, we have so many technologies tools, apps, which help make us faster, happier. We're spoiled by technology, but there are so many people, 300 million people They even don't have what to support the car. So this kind of technology to enable from can't to can. There's a big difference to make it better. So once I share this with my They map teams with other teams, they just think that's the right thing to do. That's the thing, you can provide values for different people. Right, right, and clearly, your company thinks the same, because it's putting money behind it, right? Yep Tell me a little bit about, and frankly so does the folks here, we talked about empathetic design pretty much through this entire conference, it's definitely a theme. Tell me a little bit about where we go from here. So, solving the visual challenges of a smartphone Is definitely a big one. Where do you see are the biggest opportunities for human computer interaction, HCI? That's a very big question. Yes it it. So currently for this particular research This work, we are working really to turn it from a research prototype in the lab to a real product. So we start from a map, navigation app, since that's the most frequently used for People and also brand people. And our plan next is really to try to enable this technology in different Alibaba's app. Today Alibaba's app concept about not only navigation, shopping, order delivery, food, which is very important, the food. For blind people in China, so that is actually the next step. But in terms of the HTCI, I was inspired by the haptics feedback and also when I tested this at But too bad you guys couldn't hear that. The speed, they talk so fast. The first time I heard that I said, I told my product manager, it's not correct. Let's change it. Actually, that's how the [ UNKNOWN ] are tuned. I listened to [ UNKNOWN ] for three times faster than this one. I even couldn't Hear that, it just made me think okay there are different capabilities, how could HSCI research and design can really Build on that, and make it more helpful. Well that couldn't be more inclusive, right? Chen thank you so much for a peak at the work inside your lab. Thank you, a round of applause please. Thank you. [ APPLAUSE ] [ BLANK_AUDIO ]

Brainstorm Design 2019: Smart Solutions For The Visually Impaired

Design

Fortune's Andrew Nusca interviews Chen Zhao, Research Director, International User Experience & Design at Alibaba