FORTUNE — People in the business of reviewing business advice get it by the truckload. After a while, every book delivered to your door looks just like the last one. All strategy prescriptions are backed by comprehensive research, and every author is impressively credentialed. It is hard to determine who is adding value to the conversation for two reasons: One, no one has time to read all these books; two, there’s tremendous incentive for an author to spin hard conclusions out of mucky data.
“It’s very tempting, consciously or subconsciously, to impose a pattern on data that isn’t really there in order to support a hypothesis,” says Michael Raynor, co-author with Mumtaz Ahmed of The Three Rules: How Exceptional Companies Think. “After all, if you stare at the poundcake long enough, Elvis’s profile will surely appear.”
The Three Rules conforms to type by citing impressive study numbers — 25,000 companies over 45 years — then allocates several pages to unpacking their study methodology. So I asked Raynor how he reads business books. Is there a way to assess research claims quickly, respectfully, but skeptically?
MORE: What if Steve Ballmer ran Apple?
Raynor’s first prescription is to remember that persuasive storytelling requires that the storyteller leave out the weeds. This is especially relevant to corporate biographies, since the form requires the narrator to omit people and events that turn out to be irrelevant only in hindsight.
His second note of precaution is about what to do when presented with causal claims. Most smart people know not to mistake correlation for causality, but we do it all the time. Or we dismiss someone else’s claims by saying that they haven’t proved causality (just because one event happened after another doesn’t mean the first happening caused the second). True enough, says Raynor, but “nobody has evidence of causality.” Causation exists — there would be less incentive to leave the house in the morning if it didn’t — but it’s difficult to prove in complex systems (and any system that includes humans is a complex system).
Raynor also advises watching out for what Phil Rosenzweig dubbed “the halo effect.” In other words, make sure you aren’t letting the reflected glory of a company’s signature achievement in one arena color your view of their performance in other areas.
Next, be aware of the data’s limitations and your own. Why dwell on your own limitations? Our intuition as to what’s statistically significant can be terrible. When we pick up a book that profiles certain companies, we tend to assume that the companies being profiled have, in fact, delivered noteworthy performance.
But “that’s an assumption that’s worth questioning,” says Raynor. “If two companies differ in profitability by 0.1% in return on assets over a five-year period, would you study those two companies to understand behavioral differences that drive performance differences? Of course not. Because it’s too small a difference over too short a period of time.”
So watch for sample selection and time frame. “In a short season, luck can overcome skill.”
Raynor’s last note concerns an all too common criticism of business success studies. Say a company praised in a popular business book — for example, Circuit City in Jim Collins’s 2001 Good to Great — ultimately disappoints. Critics then pile on to say that the author botched the analysis. (“Hey wait a minute, you said that company was great and then three years later they’re in bankruptcy. You don’t know what you’re talking about.”) That’s unfair — and shortsighted. “This whole notion that you have to study a company that is perpetually excellent before you can learn something [from it] is nonsense,” Raynor says.
MORE: Searching for freedom at work
The best rebuttal, he says, is to point out that Usain Bolt will probably not be an Olympic gold medal winner at age 60, but that doesn’t mean the techniques he uses now will not be worthy of study in years to come.
Our best defense against seeing Elvis in the poundcake, however, is one both authors and readers can use daily: Realize that the smartest people in any room appreciate it when you acknowledge data that doesn’t support your conclusions. So, in cases where the rules you’ve devised don’t appear to hold up, say so. Mention how you might be wrong, and then present a case for why you believe what you believe anyway, says Raynor. That kind of candor is flattering to your audience’s intelligence and — most importantly — memorable.
