“Look at the data.”
I’ve heard that business bromide many times, and it isn’t reassuring.
Companies face pressure to be data-driven, data-centric. Hey, we’re all data scientists now. But data can be manipulated and misunderstood. It can also be just plain wrong.
Flubs abound. In 2022, Equifax provided millions of inaccurate consumer credit scores to lenders. A class action lawsuit followed. Then there’s Uber, whose data quality controls missed an accounting screwup that cost it $45 million in back pay for drivers.
For trust in business data, it’s the best and the worst of times.
Thanks to technological advances over the past five years, software now ably handles “plumbing-like tasks” such as capturing and moving data, says Sean Taylor, cofounder and chief scientist at startup Motif Analytics. So people can focus on more complex analysis, whether it’s data visualization or using data to answer questions.
Overall, data is more trustworthy too. “As the technology is getting better and more reliable, we can believe that what we have in the dataset is more likely to be reflective of reality,” Taylor says.
But it’s a doubled-edged sword. Data literacy and tools have improved so much that many employees can do analysis on their own, which gives Taylor pause. “It’s very hard to know if it’s good, to give broad access to more data to more people.”
One obvious hazard is misinterpretation. “At best, it’s just random mistakes,” says Taylor, a Facebook and Lyft alumnus. “At worst, it’s some kind of systematic mistake that leads to a big blind spot, or something that’s disastrous for your company.”
Another pitfall: motivated reasoning. “Someone wants the answer to be a certain thing, and they can torture the data and use their lack of skill as a way to come up with whatever conclusion that they want,” Taylor says.
Charles Pensig, founding partner of Stratus Data, sees a trade-off between ease of use and discipline. “I think we’re going through with data what we’ve been going through with media,” says Pensig, whose boutique data analytics consulting firm has worked with Disney, Foxconn, and General Electric. “We’ve realized that it is paramount to have an editorial process to ensure that the products that are generated are high-quality.”
For Taylor, that starts with figuring out what data gets tracked or captured. “The first step in quality is designing processes that emit the kind of data that are suitable for the questions that you have.” At Lyft, for example, Taylor and his colleagues tracked drivers canceling rides and measured how long users waited before that happened. Combined, those two things helped show if driver cancellations caused bad experiences for customers.
Second, businesses can seek out flaws or defects. That’s a job for Superconductive, whose Great Expectations platform lets customers test data to confirm their beliefs about it. For example, if a company thinks it has 10,000 product SKUs and testing reveals that it actually has 20,000, something’s wrong with the data.
The third step is having someone check the work, which Taylor compares to a peer review. “If you do the analysis multiple ways and you keep getting the same finding, then you feel confident.”
Pensig’s advice: Don’t collect reams of data now and ask questions later. “How do I encode this stuff so that I can actually use it?”
Make no mistake, that’s a key data point.
Nick Rockel
nick.rockel@consultant.fortune.com
IN OTHER NEWS
What’s that smell?
Ah, yes, it’s crypto exchange Binance trying to cover up the stench of scandal with a new luxury perfume. Named "CRYPTO," the product is the company’s overture to women, who account for less than 20% of U.S. cryptocurrency holders. Its launch won’t change the fate of Binance cofounder and former CEO Changpeng Zhao, due for sentencing next month on money laundering charges.
Image problem
In yet another blow to trust in artificial intelligence, a Microsoft engineer has alerted regulators about the potential dangers of the tech titan’s AI image-generation tool. It’s all too easy to create offensive and harmful content with Copilot Designer, whistleblower Shane Jones wrote in a letter to the Federal Trade Commission. Jones wants the tool pulled from the market until his employer makes it safer.
Investor alert
Beijing would have us believe that China’s faltering economy will grow about 5% this year. Sharmin Mossavar-Rahmani, CIO of Goldman Sachs’ wealth management arm, is less bullish on the country. In fact, she’s warned investors to avoid China, citing real estate, infrastructure, and exports as three main weak spots.
Temperatures rising
The Securities and Exchange Commission may have passed its new rule that companies disclose climate change risk, but it wasn’t exactly a vote of confidence. Commissioner Hester Peirce, who opposed the requirement in a 3-2 decision, said it could give the SEC license to “spam investors with the commission’s pet topic of the day: climate.”
TRUST EXERCISE
“An overwhelming 72% of [food] shoppers report that more transparent product information and labeling are important to them. They want complete ingredient listings in plain English and more complete nutritional information than current labeling provides…That transparency leads to greater trust from consumers as well as purchasing decisions that can improve population health. That’s why the FDA must redouble its efforts to improve the clarity and completeness of food labeling.”
U.S. consumers need better food labeling, argues Julie Chapon, cofounder of Yuka. She’s not wrong. Chapon, whose company makes an app that deciphers food and cosmetics product labels, points to poor diet as a key contributor to Americans’ falling life expectancy.
To help reverse that alarming trend, she calls on the FDA to follow Europe’s lead by requiring food manufacturers to clearly explain the nutritional value of their products. The solution: an easy-to-read label that gives foods a score based on salt, sugar, fat, fiber, protein, and other content. Should be a piece of cake for the marketing whizzes that brought us Twinkies, Doritos, and Diet Cherry Vanilla Dr Pepper.