Leaders from Adobe, Wayfair, Wipro, Freshworks and Medtronic, who gathered for a lunch discussion at Fortune‘s Brainstorm Tech conference on Tuesday, said their companies were finding many useful ways to implement generative AI in their organizations. But many added that the cost and reliability of GenAI tools remained key challenges and discussed how they were addressing those issues. In addition, several pointed out that choosing the right tools for the right use cases was essential.
For example, Fiona Tan, chief technology officer of online retailer Wayfair, said that the company is currently working with a second generation of AI-driven “copilots” to help support customer service and sales employees, as well as offering image-generation capabilities for customers to envision room designs.
However, Tan said the company came up with a matrix to identify whether generative AI would work for specific tasks and how success would be measured. That was especially important given the risk of hallucinations—the inaccurate or nonsensical outputs that generative AI models sometimes produce.
There are tasks where customers might be more forgiving about hallucinations, Tan explained. She pointed to a decorating ideas feature on the website, where “if the chair leg was maybe a little askew, or it messed a little bit with some of the stuff in their room, they just appreciated the ability to see their rooms in different styles.”
Bedrooms versus operating rooms
On the other hand, Ken Washington, SVP and chief technology officer at medical device company Medtronic, said that while the company uses machine learning extensively, operating rooms are no place for generative AI right now. “You just can’t tolerate hallucinations,” he explained. “That’s why you don’t see the use of generative AI in medical technology development and there are no generative AI FDA approved therapies or diagnostic tools.”
For Adobe, dealing with the cost and reliability of generative AI is the same as any other technology. “We’re asking, what is the problem we’re trying to solve? If it’s a new problem, that’s a red flag,” said Ely Greenfield, CTO of digital media. “We would probably look at problems that we already know are out there, we just didn’t have a good solution for them before and this might be the right solution.” Then, he explained, knowing that there won’t be 100% accuracy and that the cost is expensive, “we go in and try to do an early triage on what problems we want to tackle and can we tell upfront whether the cost and reliability issues are problems for this work?”
But, he warned against focusing prematurely on cost. “I think the trick is that the dynamics of this world is changing so fast that step one is to get in and figure out whether you can solve a problem with the AI today,” he said. “By focusing on different models, swapping out, getting some smart people to go distill a model, or wait a month, wait a week, these things are getting better. They’re getting cheaper. So get involved in finding the solutions now, and then, worry about the cost.”
For Freshworks, however, a software-as-a-service company providing tools for customer service, IT and sales, cost is definitely an issue when it comes to adopting generative AI, said Siddhartha Agarwal, the company’s SVP, product strategy and operations.
“We’re a little company, and cost is important for us,” he said. What the company has done is created a “model garden” for its cloud engineering team, with 40 to 60 different models available to developers, who can experiment with a model, determine its cost profile and performance and decide if it is the right and most cost-efficient model to use. “It gives them the ability to experiment and optimize on that cost curve,” he explained.
Cost is also a pain point for IT consulting firm Wipro, which hosts a variety of models on its platform, said Chief Technology Officer Subha Tatavarti. Because of the computing power costs, Wipro is excited about smaller models, which use less computing power. If the company can reduce the cost of serving up models, that can “translate into customer cost savings as well,” she explained.
Wipro has also focused on RAG, or retrieval-augmented generation, a technique that can reduce hallucinations and improve model output. “From what we’ve done in the last couple of months, it has given us very encouraging results,” she said.
Read more coverage from Brainstorm Tech 2024:
AI adoption in ad industry needs ‘non-optional mandates’, says Interpublic CEO Philippe Krakowsky
Why business leaders view AI as an opportunity to take ‘toil out of our work’