Amazon Alexa at 5: A Brief History of the Echo's Digital Assistant

Dave Limp has an Amazon Alexa device in every room in his house. Strike that—Dave Limp has an Alexa in every room in his house and two in his bedroom. But, then again, Limp is Amazon’s senior vice president for devices, meaning he’s in charge of the Internet giant’s Fire TV devices, Kindle e-readers, and, of course, Amazon’s digital assistant, Alexa.

Five years ago Wednesday, Limp’s house was a much emptier—and no doubt quieter—place. That’s because on Nov. 6, 2014, Amazon announced the original Echo speaker. Costing $199 (or $99 for Amazon Prime members) at launch, the smart speaker was originally available by invitation only, for the first few months. Limp had had test versions for more than a year, but all at once, regular consumers could buy an Echo.

The Verge called it compelling but also infuriating. Geekwire’s reviewer stumbled across saved sound recordings from the Echo and presciently warned “Amazon should be (and may be, somewhere on its site) very clear about who has access to the text history and voice recordings.” Fortune concluded Alexa was “a good-enough personal assistant,” though we weren’t sure that we needed more than one Echo in a house.

Consumers, on the other hand, bought them up like M&Ms. Amazon says, to date, more than 100 million Alexa-compatible devices have been sold. “It was very clear, very quickly that customers really did love this,” Limp tells Fortune in an exclusive interview for Alexa’s fifth anniversary. “We were creating something new that’s different than just a product. You’re creating a category, which doesn’t come along in consumer electronics very often.”

But no one, not even Limp, foresaw just how popular Alexa would become or that Amazon would so easily fend off rivals like Apple, Google, and Microsoft as the home digital assistant market grew. Now Limp says his kids, growing up around all those Alexa devices, expect voice controls everywhere. “It’s their new normal,” he says.

Even at a young age, Alexa—and its success—has yielded many lessons for the tech industry. Here are a few that speak to Limp.

Failure is an option

Before Alexa answered it’s first query, Amazon wasn’t exactly known for its hardware prowess. A few months prior to the Echo release in 2014, the company had unveiled its Fire Phone. Despite being crammed with multiple cameras and other gee-whiz features, the Fire Phone was a complete flop. Within about a year, it was off the retailer’s web site and gone forever.

Both the Fire phone and the Echo speaker were developed at virtually the same time, and both by Amazon’s Lab126 unit. Limp says Amazon is willing to take risks to see which innovations connect with customers. “To me it was a good lesson… you have to have conviction when you’re inventing,” Limp says. “Once you think the product is up to par, you have to get it out in front of customers—knowing that many of those are going to fail.”

“I’ll take 20 failures like Fire phone if tomorrow you could tell me we have 100% chance of getting another Alexa and Echo,” Limp adds.

Anyone would. By the year’s end, according to research firm eMarketer, Amazon will have sold an estimated 49 million smart speakers, capturing 63% of the market. (Second-place Google will trail far behind with less than 25% market share.) That’s quite the turnaround.

Open standards can win

Early on in the life of Alexa, the team’s top leaders met with Amazon CEO Jeff Bezos to ask for more engineers and to update the boss on their progress. They were adding new areas of knowledge to Alexa but it wasn’t enough to satisfy Bezos. The team was “moving fast but not fast enough and Jeff thought the only way… was to do similar things to what we did in (Amazon Web Services) for developers,” Limp says.

So instead of developing all of Alexa’s features internally, Amazon pivoted to make it easy for outside developers to plug their apps into Alexa’s code, and to plug Alexa’s code into all kinds of third party devices. Limp himself used the simple tools to code a memory game using the light ring around his Echo.

By 2016, Limp was staggered as he walked the aisles of the Consumer Electronics Show in Las Vegas. “The number of people that had enabled a skill or enabled a piece of hardware that was connecting to Alexa that I’d never heard of—and my job was to hear about these things—was very surprising,” he says. Among the surprises, Limp encountered a voice-controlled robot and an air purifier from Korea connected to Alexa. “That wasn’t anything I had ever contemplated,” he says.

Today, there are over 100,000 skills, allowing Alexa users to play word games, hear jokes, access their calendars, translate from one language to another, and more. And on the hardware side, Amazon has rolled out a massive array of Alexa-compatible devices—everything from televisions and thermostats to headphones and the aforementioned microwave oven. Partnerships have brought Alexa to popular products from other brands, like Fitbit smartwatches and Sonos high-end speakers. Next year the voice assistant is coming to GM cars.

And the openness could expand further, as Amazon executives say they are willing to share their platform with other digital assistants. “The whole pie is getting bigger as customers are embracing this technology,” Tom Taylor, who heads the Alexa effort, tells Fortune. “As we look at the future of these voice assistants, we don’t think that you just have one any more than you would just have one friend.”

Keep it simple (at least at first)

While there were already voice-controlled assistants on smartphones before Alexa, all of those apps had a backup. If a user was frustrated by an assistant like Apple’s Siri or Microsoft’s Cortana, they could use their phone’s regular touch interface to get what they wanted the usual way. But Echo had no other way for user control except voice commands.

The lesson is that when you are rolling out a new tech, says Carolina Milanesi, principal analyst at Creative Strategies, be focussed. “If you are trying to establish new behaviors, make sure not to give users the option to fall back on what they know,” she says. “With Alexa and Echo, voice was the only option.”

The number of devices with Alexa quickly expanded, however. Echo Dot, perhaps the most important single product in Amazon’s line, differentiated the company’s hardware offerings in 2016, with a small, hockey puck-sized smart speaker that currently costs just $40. According to analysts, the Dot is Amazon’s best selling Alexa device. (Amazon itself doesn’t disclose any sales figures for the Alexa line.)

And over time, Alexa has also grown far more complex and capable. Smart home control wasn’t even foreseen as a possible use case by Amazon initially, Limp says. Now it’s one of the most popular functions, with users controlling lights, thermostats, and many other items.

Don’t ignore privacy and security

As that early Geekwire reviewer highlighted, Alexa users did not completely understand how their data, including recordings of their voices, was being collected and shared—even just within Amazon. After a few minor incidents, like police officers seeking Alexa recordings to help investigate a murder, the privacy issue blew up this year.

Reporters learned that thousands of people working for Amazon were listening to a small portion of Alexa customer recordings in order to improve the service’s voice recognition and other features. Some critics charged Amazon hadn’t been clear that its employees could listen to customer recordings.

“We thought we were telling customers clearly and we thought it was a pretty known industry thing,” Limp says. Humans had been reviewing users’ dictation recordings and map requests and other data collected by their devices, he notes. “Humans were in the loop on all these things. At some level, because we thought that was normal, and that was normal in academia, we thought we were communicating it well enough.”

But when reporters kept writing about the practice and people kept reading the stories, “that to me meant customers were surprised,” he admits. “When that happens, our job is to listen to that and make the changes we are think are the right thing on behalf of the customer.”

As a result, Amazon addressed the issues, improving its policies on disclosure and customer control. It cleaned up its public explanations on the web and offered customers the opportunity to opt out completely of having their recordings reviewed by humans. Users can also review and delete their recordings.

“We have to listen to what the world around us is saying and we can’t be static in the way that we do things,” Limp says.