A.I.’s role in financial transactions needs the human element of ‘explainability’

Given the complex nature of these algorithms and the need for customers to feel comfortable with what’s happening with their money, there needs to be a level of transparency from the financial institutions. It’s a challenge, as Morgan Stanley’s chief analytics and data officer Jeff McMillan said at the Fortune Brainstorm A.I. conference in Boston, “because we’re not just giving you movies. We’re giving you investment advice.” He stressed that “explainability” is key to keeping customers happy and firms honest.

“You can go get these algorithms off of AWS and Azure. They’re not secret. And creating recommendations, it’s not super hard on a relative basis,” he said on Tuesday. “But explaining the recommendation to someone who has zero expertise in data science is really hard. Explaining why you are saying, ‘This idea is a good idea,’ in very layman’s terms is very, very difficult. We spent as much time trying to communicate the complex in simple terms as we did making the thing super complex.”

Focusing on that, he said, can help a corporation understand a client’s needs better, which in turn helps improve the efficacy of their A.I. tools. “You don’t fail because your machine-learning algorithm isn’t ‘good enough,’” he said. “In my experience, you fail because you can’t get that last mile where the machine learning intersects with human behavior. It’s that intersection where you’re trying to sort of cross that chasm, we have to say, ‘Listen, we’re not here to replace your job. We’re here to make your job actually better and more efficient,’ where I think organizations really struggle. This is an A.I. conference, but it’s really about the A.I. human element where I think organizations like all of us really need to focus our time and attention.”

Carol Juel, the executive vice president and chief technology and operating officer at consumer financial services company Synchrony, agreed and also pointed out that the pandemic had huge effects on digital transacting. She cited her firm’s acquisition of Pets Best, a pet health insurance company, and how the increase in people getting animals during the period of isolation also gave the company much more information about their customers.

“In financial services, in general, we have a significant volume of data,” she said. “On every individual customer, we have thousands of attributes that tell us about them and then we have behavioral attributes. We combine that with proprietary partner data and that’s the key piece, because those then can drive the opportunities for personalization. Understanding someone’s digital engagement, knowing how much they spend, knowing what their behavior patterns are digitally, [then] you can drive that experience. And as the world moves massively digitally over the last 20 months, how we then improve authentication to reduce fraud, how we know more about those customers, so that that experience can be frictionless. You can ultimately allow for those high-risk track transactions to flow through digitally, as more and more is happening through those channels. We measure everything from volume of customer satisfaction to reduction in call center volume. If you have that underlying data to build those models that you can rely on to reduce fraud and improve experience, that’s sort of the holy grail of digital transformation in financial services.”

Speaking on a running theme of the conference, both panelists addressed the need for ethics, accountability, and transparency in the way they use A.I. and machine learning to make decisions. McMillan said it’s one of the most important things a firm can do. 

“You have to define, ‘What does it mean to be ethical?’ How are you defining that, because in my business it’s going to be different for pharma than it’s going to be for manufacturing. So you have to define what it means for the model to operate within the bounds of conformity. Organizationally, you have to have those conversations, you have to bring the right people to the table to define that. Then you have to have an independent group of people to monitor the efficacy of what you do and hold you—hold me as the guy that does this for a living—accountable and create transparency. If I go out of bounds, you have to call me to task and there has to be those forums to say, ‘Jeff is operating outside of the defined boundaries of the standards that we’ve defined for this, and we need to have a conversation.’”

In closing, Juel offered some words of wisdom, and caution, for all companies relying on A.I. “There’s a lot of, oftentimes, hype about the technology. The technology is amazing, I love tech, but I think you have to take a step back from that and really bring people along the journey. A.I. is and can be transformational for your organization. But it’s not the technology, it’s how does it apply to the business that you are in? What are the challenges? How fast do you want to move? Will your culture allow you to move at the speed you want to? What are the things you need to adapt in order to take advantage of the opportunities there? Because A.I. is not going to solve the problems of any organization if you think that’s the solution. There’s a whole host of culture, previous investment, how it’s viewed, how important it is. If something’s not working, [you need to say] ‘Hey, we can’t do that. Let’s move on.’ And that’s not a failure, that’s learning. A.I. has given us the opportunity to say, ‘That data didn’t meet what we intended it to do, that model didn’t work, let’s move on.’”

More must-read business news and analysis from Fortune:

• From Delta to Southwest, the airlines in the best—and worst—shape going into a chaotic holiday season
• How a risky bet on the Shiba Inu coin made this warehouse manager a millionaire
• Patagonia doesn’t use the word ‘sustainable.’ Here’s why
• Will monthly child tax credit payments continue in 2022? Their future rests on Biden’s Build Back Better bill
• ‘I’m afraid we’re going to have a food crisis’: The energy crunch has made fertilizer too expensive to produce, says Yara CEO