For most of modern history, it was the same routine. Whenever we needed a new lipstick for a big event or a more hydrating moisturizer in preparation for winter, a visit to our favorite store was in order—be it the finest beauty counters at Saks Fifth Avenue, the nearest Sephora, or the local CVS. With the rise of e-commerce in the 2000s, it seemed we could finally buy makeup and skin care with the ease of a few clicks.
But it soon became clear that this newfound flexibility came at the cost of a necessary buying experience. Even with every imaginable cosmetics product available online, we could no longer try on eight shades of foundation to find that one perfect match, and consulting the wisdom of a salesperson on which cleanser is best for sensitive skin was but a distant memory.
This disparity became even more glaring last year, when the COVID-19 pandemic forced much of the world to stay at home and temporarily closed the doors of nearly all brick-and-mortar stores. Suddenly, we had no option but to procure our beauty online, and even after some stores reopened, trying products before buying was rendered obsolete thanks to new health and safety precautions, and asking the advice of a salesperson presented the risk of spreading or contracting the deadly virus. So, with global beauty-industry revenues expected to fall 20% to 30% and online sales far from offsetting the decline in in-store sales, some brands turned to two technologies that had long been on the fringes of the beauty world: artificial intelligence (A.I.) and augmented reality (A.R.).
For many years, digital innovation has been quietly disrupting the beauty industry, but the touch points consumers saw most were often gimmicky, faulty, or simply far too niche. Early players in the space utilized primitive VR abilities to enable users to virtually try on makeup—an exciting prospect in theory but frequently disappointing and unrealistic in practice.
In 2014, however, cosmetics giant L’Oréal launched Makeup Genius, a virtual makeup mirror that worked on smartphones, and it become the first adoption of augmented reality in beauty to really go mainstream.
“It had over 20 million downloads in the first year and led to our team becoming quite visible, but it also started our movement toward A.I.-based digital services,” says Guive Balooch, global vice president of L’Oréal’s Technology Incubator. “The organization thought this kind of technology would soon be everywhere in the shopping experience, and our CEO wanted to internalize that, so that’s when our chief digital officer led the acquisition of Modiface.”
At that stage a leading technology in this arena, Modiface quickly became an integral part of L’Oréal’s tech strategy and has played a key role in many of the company’s digital innovations since, but it also brought a new problem to Balooch’s attention.
“Sometimes there aren’t enough products, and we need to create them, but other times, there are too many products, and we need to know which ones are right for us,” he explains. With seemingly limitless options available to customers to virtually try on, many users felt overwhelmed and experienced purchasing paralysis as a result. “We needed to help them find what was right for them, so we started using A.I.,” Balooch says.
Starting with Lancôme’s Le Teint Particulier foundation and continuing with SkinCeuticals’ SkinScope, L’Oréal developed an in-store tool that could measure skin tone and skin conditions and then use A.I. to create a personalized formula for the customer.
The pandemic and the new demand it’s brought for digital beauty solutions have served as somewhat of a validation for Balooch and his team, whose innovations are seeing more need than ever.
“It’s made people more curious and knowledgeable about the space, and there’s been earlier adoption and connection,” he says. “We’ve also seen how people are wanting to interact with products at home, so we’ve accelerated aspects of our strategy to keep up with how the world is moving.”
One such aspect is the launch of L’Oréal’s Perso, a project the technology incubator restarted last year after putting it on hold years earlier. The latest in what Balooch calls “phygital” projects, Perso is a physical device that uses A.I. and A.R. technologies to power it.
This week, L’Oréal will launch the first consumer product using the technology, Yves Saint Laurent Rouge Sur Mesure Powered by Perso, which will utilize color cartridge sets of YSL’s iconic Velvet Cream Matte Finish lipsticks to create thousands of bespoke shades with a single touch. In the future, L’Oréal plans to create similar Perso-powered devices with many of its brands, from luxury makeup to drugstore skin care, to enable customers to fulfill all of their changing cosmetics needs easily and at home.
As one of only a few beauty brands to devote substantial attention and investment to developing technology so early on, L’Oréal was at an obvious advantage when the pandemic hit in early 2020 and consumer behavior shifted nearly overnight.
“I’m obviously a bit biased, but what I think is amazing about what L’Oreal did was that they invested in the vision of the future even when there were challenges, and we now have this amazing team,” Balooch says. “In a way, I think the major advantage that we have is the infrastructure that we’ve developed over the last 10 years, and that’s why we’re able to do things that are a little bit more complicated.”
Other brands that lacked the same history of digital innovation or the internal resources necessary to meet rising demand for A.I.- and A.R.-powered services looked to solutions beyond their company gates. As consumer interest in such technology has grown in recent years, colossal brands, retailers, and indie brands alike have turned to Perfect Corp, a Taiwan-based beauty tech provider that has pioneered A.I. and A.R. solutions. In business since 2015, the company has built several services harnessing A.R. technology, facial recognition, and color matching, and it’s proved invaluable to beauty’s push toward digital.
“At the very beginning, it was a little bit hard to sell because the beauty industry is very traditional, and there was the thinking that digital would skew colors,” says Wayne Liu, senior vice president and general manager of Perfect Corp. “But after showing proof of concept with a brand like Estée Lauder, which was really the first company we worked with, we saw a huge response.”
In 2019, Estée Lauder called upon Perfect Corp’s YouCam Makeup virtual try-on-color match experience to create the iMatch Virtual Shade Expert, which offers customers a way to find their preferred shade of Double Wear Stay-in-Place Makeup and Futurist Hydra Rescue Moisturizing Makeup and see it virtually applied in real time. Following the Shade Expert’s success and aware of the mounting need for at-home solutions in the wake of the pandemic, Estée once again partnered with Perfect Corp late last year to build iMatch Virtual Skin Analysis, a self-guided digital skin care diagnostic tool that leverages the latest YouCam A.I. technology to reveal what skin needs with a personalized skin care regimen.
“Our goal is to provide our customer with the most personalized and engaging experiences, something that is at the core of our brand DNA,” says Jon Roman, Estée Lauder’s senior vice president of global consumer marketing and online. “And leveraging A.I. technologies allows us to provide personalized and accurate product recommendations, while enabling her to have the same high-touch experience whether she’s shopping in-store or online.”
Similarly, Neutrogena, owned by personal care and medical behemoth Johnson & Johnson, sought the help of Perfect Corp in early 2020 to create an updated version of its Skin360 app, a skin analysis tool which previously required a separate $60 phone attachment to use. Perfect Corp’s YouCam technology allowed the app to instead provide a 180-degree selfie analysis simply using a smartphone camera, and the algorithm, developed in partnership between the Taiwan firm and Johnson & Johnson’s Research & Development team, analyzes more than 100,000 skin pixels over 2,000 facial attributes and offers more than 2.5 million possible product recommendations. Since its relaunch, the Skin360 app has seen more than 115,000 downloads, a rise in popularity that Neutrogena believes can be attributed to the increasing demand for at-home, dermatologist-grade skin assessment.
The brand also collaborated with Google and YouTube late last year to utilize its new Virtual Try On technology, allowing consumers to try on Neutrogena’s lip shades alongside their favorite beauty influencers while watching their YouTube videos, and it is currently working with Amazon’s makeup try-on technology. But after paying close attention to the consumer shift to online shopping during the pandemic, Neutrogena has created its own virtual try-on solution, with the help of Perfect Corp’s YouCam Makeup technology, and it is set to launch within the first quarter of 2021.
“Ultimately, the evolution of digital capabilities addresses market opportunities as e-commerce shopping grows and as retailers remove physical in-store testers to avoid hygiene concerns,” says Logan McGill, the brand’s director of acne innovation and skin tech. “Through platforms like our Neutrogena Skin360 app and technologies that enable virtual try-on of Neutrogena Makeup products, our aim is to put the power of skin health directly in the hands of our consumers, enabling them to learn more about their skin and to more easily find the products that are right for their unique needs.”
While A.I.- and A.R.-powered beauty innovations were in the works long before COVID-19 was even a part of our collective vocabulary, the pandemic and the problems it’s presented to the beauty industry have undoubtedly sped up the consumer adoption of such solutions. As such, it seems all but certain that demand for beauty tech will continue well into the future, even as traditional in-store buying experiences once again become feasible.
But many of beauty’s biggest players aren’t yet convinced that there won’t be a place for brick-and-mortar stores in the future, and the tech innovators are hedging their bets accordingly.
“On the one hand, as a technology provider, we provide the technology that customers need to recreate an in-store experience at home,” explains Perfect Corp’s Liu. “But on the other hand, some of the beauty brands that we work with really do rely on the store, and they do still need to get the consumer back to the store and think about new ways to do that, which is where in-store technology can come into play.” Rather than create solutions simply for today’s problems, he says, it’s about solving the problems that will be important for many years to come.
