Amazon’s Alexa is becoming a member of the family

Eight years ago, Amazon launched the Echo smart speaker with its Alexa Voice System, now simply dubbed Alexa. At the time, the idea of speaking into a machine from a distance with the wake word “Alexa”—and having the machine respond quickly and intelligently—was a ground-breaking consumer product. Today, that technology has evolved to more than 130,000 skills, including the ability to “drop-in” on other Alexa device owners, offer up recipe recommendations for dinner, add orange juice to the grocery list, and play a variety of news shows and music genres. The tech that powers it all? Artificial intelligence.

“You can tell Alexa that you’re vegetarian, and now Alexa personalizes [its content and recommendations] on that basis of that information you’ve shared,” says Rohit Prasad, senior vice president and head scientist for Alexa at Amazon. “Alexa is going to use that vegetarian preference when showing recipes to you or restaurant choices for where to eat. That’s how A.I. should operate—with transparency, what data has been collected, and how it’s being used.”

Prasad has worked on the Alexa product since the beginning, when the breakthrough in the field was simply the spoken word. What made it possible, he says, is the large amounts of computing power and graphical processing units of technologies like deep learning, a machine learning technique under the artificial intelligence umbrella that teaches computers to learn by example. As Alexa provided more guidance to users’ commands, the machine got smarter and more efficient at generalized intelligence and its human-based reasoning.

With more data points, the Alexa team of developers were able to add technologies, such as the Alexa skills kit. The device is particularly adept at transactional requests, whereby a user issues a request and the machine answers. For example, a user may ask Alexa the weather for the day or to play a favorite podcast from Spotify.

Prasad says they are now working to take that a step further to tasks that are cognitively difficult, such as vacation planning or extended communication about a particular topic, like the history of a sports team’s performance. For him, the dream would be to have Alexa engage with users in more human-like ways, including turn-taking conversation for as long as 20 minutes and more intuitive suggestions, based on its learnings about how a user interacts with their Alexa.

“We are moving up the A.I. stack to power, what I call ambient intelligence,” Prasad says. “The A.I. should anticipate your needs and help you by saying, ‘Hey, I think you left your bedroom light on’ and you’re in the kitchen.”

Prasad aims to develop an Alexa that is more self-aware and makes use of multimodal intelligence. One major goal is to help Alexa accomplish more than one task at a time, in the way that humans multitask throughout the day. He wants Alexa to be helpful to the user without the user issuing a specific vocal request—the device should foresee requests based on the user’s preferences and environment as well as learn from its own mistakes. It’s as if Alexa will be self-aware of its surroundings in a home or office as it continues to build a relationship with its user.

He says his team is striving toward that by listening to their customers, who he admits “demand a lot from Alexa,” and investigating how technology can support those innovations. He believes that Alexa should join in the conversation, as if Alexa is a relevant member of the household. To do that though, users need to help train the A.I. to a point where it can anticipate a user’s morning routine. For example, if you wake up and turn off your alarm, ask Alexa for the weather, and ask Alexa to play the news, he wants Alexa to construct that routine for you. Then, the next time you contemplate hitting snooze, Alexa automatically launches into that morning routine.

“The self-awareness of Alexa can make it a true assistant advisor and companion for you,” Prasad adds.

He likens the current state of Alexa to a kid, one that is still in its early stages of learning. Over the years, he says, A.I. has become more adept at various functionalities. If users continue to be patient and explore the discovery aspects of the device while the A.I. technology advances, then Alexa will keep getting better for consumers.

In fact, the COVID-19 pandemic offered some salient learnings for Prasad’s team. They saw that Alexa’s drop-in feature was used widely by families to check in on loved ones, especially the elderly. They partnered with Boston Children’s Hospital at Cedars Sinai in a similar way, where they installed an Alexa on the bedside table of patients. Patients could connect with family members at any given moment. Given the recent challenges of not being physically present with one another, Alexa offered up the solution by being a portal for human contact—a much more powerful feature than, say, it’s popular ability to play top musical hits.

It also reinforced their hunch that people are looking for things to do at home. Alexa can, and should, Prasad says, be a voice of suggestions, such as what book to read next or what to do on Saturday afternoon. It’s exactly as if Alexa is another family member who offers up an opinion at the dinner table.

“What we think as humans is simple is hard for the A.I., and what we think is hard for A.I. is actually simple,” Prasad says. “We have to work on more of these A.I. methodologies to make Alexa smarter on multiple tasks and develop things like common sense reasoning. It doesn’t exist in its full form, but that’s the way it’s evolving. That is my long-term vision for Alexa.”