FORTUNE — Can companies learn to become better innovators? At a time when economic growth is stagnating in mature economies and slowing in developing ones, and experts far and wide are fretting over the degree and quality of innovation here and abroad, this has turned into a (multi) million-dollar question.
For the past eight years, our firm, Booz & Company, has conducted an annual study on R&D spending among the 1,000 public companies that spend the most on innovation. Every year, we affirm that there is no correlation between how much a company spends on R&D and its overall financial performance. Apple (AAPL), for instance, has consistently been named the most innovative company in our study, yet it spends just 2.2% of its revenue on R&D, well below the 6.5% of revenue spent by the computing and electronics industry as a whole.
Overall spending among all 1,000 companies increased by 9.6% in 2011 compared to the previous year, to $603 billion. But a great deal of that money is not being spent wisely. As part of this year’s study, we surveyed and interviewed executives about their activities during the early stages of innovation, when companies generate and then vet the ideas that will eventually become new products and services. The results were not encouraging.
MORE: Corporate America: Don’t give up on your workers
Forty-six percent of the executives we surveyed admitted that their efforts to generate good new ideas and move them into product development stage were only marginally effective, and just a quarter of them said their companies were good at both idea generation and development. These results vary considerably based on the strategy that these companies follow in developing new products and taking them to market.
Companies that directly engage customers in hopes of understanding their needs and wants, and then try to be first to market with their new products, tend to be more effective in their early-stage innovation efforts.
Companies that depend on a deep understanding of their chosen markets and prefer to innovate through incremental improvement, even if they aren’t first to market with the resulting products, are somewhat less effective at generating and vetting new ideas. Also less effective are companies that emphasize technology solutions and need a robust pipeline of game-changing technology.
But even these distinctions don’t matter as much as how well a company can execute on its chosen strategy (assuming that it actually has a strategy for innovation; nearly 20% of companies don’t). A company must first strive to get its innovation and overall business strategies on the same page. Then it must make sure it has the agility to carry out this strategy, and make sure its culture and organization are on board.
Hewlett-Packard (HPQ), for example, whose success has long been driven by its technological wizardry, would likely be much less successful if it decided to take its innovation cues primarily from extensive market research into what PC consumers want. “Consumers don’t really know what is possible,” says Girish Nair, HP’s senior vice president of corporate strategy and alliances. “They can describe what they want, but they don’t know what the technology can do, especially because what the technology can do changes rapidly. You have to create it….”
Where the best new ideas come from is surprisingly consistent among successful companies, although the tools and processes each company uses varies. Consider Caterpillar Inc. (CAT), whose innovation strategy depends not on being first to market with the latest, greatest technology, but rather on understanding what the market wants by making good use of tools like customer feedback. Tana Utley, Caterpillar’s chief technology officer, underscores this point: “If it doesn’t relieve a particular customer pain point, then there’s no compelling case to pull it through all the gates into production.”
MORE: Vote: Businessperson of the Year Round 4
Excellent execution are just as choosing which ideas to develop into marketable products, but timing is essential as well. The success of instrument maker Agilent Technologies Inc. (A) depends on making sure that the right technology gets into the marketplace at the right time, so the company works hard at deciding which projects to develop and which to kill.
The process has to be rigorous, says Darlene Solomon, Agilent’s chief technology officer. “We constantly ask ourselves, ‘What have we learned about the technology that makes it more or less attractive than [it was] six months ago? What have we learned about the market, … and what’s going on in the world that might help us decide whether the technology is now even more valuable or perhaps becoming a “me-too” technology?’ ”
In our study, we found that the 100 biggest R&D spenders are just half as likely as smaller ones to be effective at this stage, because they often lack the ability to move products into development quickly.
Our study shows that companies with enough discipline to match their innovation strategy with what they are capable of pulling off perform considerably better than those companies who do not do this. Successful innovation doesn’t require flashy new techniques like social networking, just the knowledge of what works and what doesn’t, and the determination to stick to these principles.
Barry Jaruzelski is a senior partner and John Loehr and Richard Holman are partners at Booz & Company.
Editor’s note: Hewlett-Packard and Caterpillar are clients of Booz & Company.
