Tony Ulwick is the pioneer of jobs-to-be-done theory, the inventor of the Outcome-Driven Innovation® (ODI) process, and the founder of the strategy and innovation consulting firm Strategyn. He is the author of  “Jobs to be Done: Theory to Practice” (IDEA BITE PRESS) and numerous articles in Harvard Business Review and Sloan Management Review.

In more than 95 percent of the hundreds of companies our firm has assisted, managers haven’t even been able to agree on the answer to the most fundamental question: what innovation is.

A definition is in order. Innovation is not an initiative; it is a business process. The process begins with market selection and includes steps to uncover customer needs, determine which needs are unmet, formulate a growth strategy, and devise and evaluate product and service concepts. Approved concepts then enter into development—a separate process. When the innovation process is executed effectively, only winning products enter the development process, and product success rates can exceed 80 percent—a vast improvement from today’s 70–90 percent failure rates.

Today’s most popular approaches to innovation fall into one of two types: those that revolve around ideas and those that revolve around needs.

In what we call the “ideas-first” approach, companies brainstorm or otherwise come up with product or service ideas and then test them with customers to see how well the ideas address the customer’s needs.

In the “needs-first” approach, companies first learn what the customer’s needs are, then discover which needs are unmet, and then devise a concept that addresses those unmet needs.

Unfortunately, the “ideas-first” approach is inherently flawed and cannot work, and the “needs-first” approach, although superior, often fails because it is structurally flawed. It can work, however, if those structural flaws are overcome.

This is where marketers have a role to play if they seize the opportunity.

The Ideas-First Approach Is Inherently Flawed

Many companies adhere to the “ideas-first” approach and have developed support systems and organizational cultures that reinforce its use. Companies that follow this paradigm believe that the key to success in innovation is to be able to generate a large number of ideas (the more, the better) and to be able to quickly and inexpensively filter out the ideas that will likely fail. They believe this approach gives them a better chance of coming up with a greater number of breakthrough ideas.

This thinking is supported by many academics, managers, and consultants.

Creators and supporters of many of the popular gated or “phase gate” development processes, for example, state that the first step of the development process is idea generation. Approximately 68 percent¹ of firms have adopted some form of gated development, which means that this same percentage have adopted, at least to some degree, the ideas-first mentality.

Examples demonstrating the prevalence of this mind-set abound. In their book, Innovation to the Core, Strategos CEO Peter Skarzynski and Rowan Gibson say that “successful innovation is a numbers game… the chance of finding a big, new opportunity is very much a function of how many ideas you generate, how many you pick out and test with low-cost experiments.”² Harvard Business School professor Teresa Amabile states in a frequently cited article that “all innovation begins with creative ideas.”³

Nearly everyone in a major corporation has participated in a brainstorming session in which, without knowing the customer’s needs, they were encouraged to generate hundreds of ideas and were told that there is no such thing as a bad idea. You can probably still picture walls of Post-It notes!

Others who support the ideas-first approach have promoted the benefits of executing the approach quickly. Many refer to this accelerated ideas-first approach as “failing fast,” the idea being that when many ideas are generated and tested quickly, the best ideas are revealed faster.

Since it is accepted that an ideas-first approach is going to generate many failures, it seems logical to try and weed out the failures quickly.

This concept was touted by Tom Peters in Thriving on Chaos. In that book, Peters said companies should, “test fast, fail fast, adjust fast—pursue new business ideas on a small scale and in a way that generates quick feedback about whether an idea is viable.”⁴ IBM founder Thomas Watson, who years ago said, “If you want to succeed, double your failure rate,” also supported this thinking and adopted a management style that did not punish failure. The fail-fast approach is still well supported today. For example, the authors of Innovators Guide to Growth believe that “if you fail fast and fail cheap, you have actually done your company a great service.”⁵

As a result of this ideas-first thinking, an entire ideation industry has evolved to compete on developing ways to generate and evaluate more and more ideas, faster and faster. But there is a problem: despite its popularity, academic support, and widespread use, the ideas-first approach to innovation cannot be counted on for predictable growth and is inherently doomed to failure. There are two reasons for this:

First, generating more ideas does not meaningfully improve the probability that someone will come up with the optimal idea to satisfy unmet customer needs.

People are in effect brainstorming ideas without ever knowing what the customer’s needs are or which of those needs are unmet. We know that in any given market a customer has 50 to 150 needs (how we know this will be discussed later) and that anywhere from 5 to 80 percent of those needs may be unmet. The mathematical probability of someone coming up with an idea that satisfactorily addresses all the customer’s unmet needs without knowing what they are or whether or not they are satisfied is close to zero.⁶

Generating more ideas that don’t meet customers’ needs is misguided, and doing something bad faster does not lead to better results.

This approach to innovation is analogous to expecting a sharpshooter to hit a target without knowing what the target is. It is like expecting a doctor to recommend the right treatment without knowing what is wrong or what the symptoms are.

This brings us to a second reason why the ideas-first approach is doomed to failure: the evaluation and filtering processes are flawed.

Because the customer’s unmet needs are unknown, the evaluation and filtering processes used today can easily miss great ideas and fail to filter out bad ideas. Let’s remember what the evaluation and filtering process is supposed to do: separate the useful ideas from the useless ones. Or, in other words, choose the ideas that best address the customer’s unmet needs. And yet, this evaluation and filtering process is typically executed without knowing what the customer’s needs are.

Lacking explicit knowledge of customers’ unmet needs, managers rely on intuition or evaluate proposed concepts using methods such as conjoint analysis, paired comparisons, and forced-choice scaling techniques, along with surveys and qualitative methods such as focus groups. These methods and others like them rely on customers to evaluate how well a proposed idea will address their unmet needs without truly understanding the product or technology and how it explicitly relates to those needs. Such an evaluation and filtering process is faulty in several respects. The first and most obvious one, mentioned earlier, is that chances are great that the best solution is not even in the consideration set.

But there is also the fact that customers may not be able to make the connection between the technology and their needs. It is not surprising, then, that companies using the ideas-first approach to innovation struggle to achieve success rates greater than 10 to 20 percent.

The Needs-First Approach Is Structurally Flawed

Those who have recognized the inherent flaws in the ideas-first approach often attempt to follow a needs-first approach to innovation. Using this approach, companies first attempt to understand the customer’s needs, then figure out which are unmet and devise a concept that addresses those unmet needs.

This thinking, although very different from the ideas-first approach, is also supported by many academics, businesses, and suppliers. Theodore Levitt, for example, in his 1960 landmark Harvard Business Review article, “Marketing Myopia,” states, “an industry begins with the customer and his or her needs, not with a patent, a raw material, or a selling skill.”⁷

Since then, others have drawn a similar conclusion. Harvard Business School professor David Garvin has noted that “studies comparing successful and unsuccessful innovation have found that the primary discriminator was the degree to which user needs were fully understood.”⁸ In theory, if all the customer’s unmet needs are known, then ideas can be generated to address them—and these ideas will have obvious value.

Over the years, many methods have been utilized to capture customer needs. These include focus groups, personal interviews, customer visits, and ethnographic, contextual, and observational research methods in addition to interviewing techniques such as voice of the customer (VOC), lead user analysis, and storytelling.

A comprehensive supplier industry has been developed to offer these services, and yet companies nearly always fail to uncover all or even most of the customer’s needs. The reason for this is twofold. First, there is no universally accepted definition of a customer need, and second, there is an assumption that customers have latent needs, or needs that cannot be articulated. As a result, most companies don’t know what customer inputs they are looking for or when they have them all—they assume that it is impossible to capture a complete set of customer need statements and that they have no choice but to execute the innovation process without knowing all of them.

For 20 years, this belief has been supported and perpetuated by many well-respected individuals and organizations. In their 1991 best seller, Competing for the Future, Gary Hamel and C. K. Prahalad warn companies of the risk they run if they cannot get a view of the needs customers can’t articulate.⁹

The Product Development Management Association (PDMA) states that “customer needs, either expressed or yet-to-be-articulated, provide new product development opportunities for the firm.”¹⁰

Peter Sharzynski and Rowan Gibson explain in Innovation to the Core that “radical innovators are deeply empathetic; they understand—and feel—the unvoiced need of customers.”¹¹

Even the process-oriented P&G CEO, A. G. Lafley, says in The Game-Changer that “great innovations come from understanding the customer’s unmet needs and desires, both articulated and unarticulated—that is, not only what they say, but, more important, what they cannot articulate or do not want to say.”¹²

Given those attitudes, it is not surprising that companies think that customers cannot articulate their needs and that capturing all the customer’s needs is impossible. But the truth is that customers can articulate their needs and that all the needs can be captured.

And then there is the more basic problem of defining exactly what a customer need is. At most companies, 95 percent of managers will say there is disagreement regarding how a need should be defined. Even more significant, the companies tasked with capturing customer needs also disagree on the definition. The sad reality is that despite all the talk about satisfying customer needs, there is very little understanding of what a customer need is or what the purpose, structure, content, and syntax of a need statement should be.

Abbie Griffith and John Hauser loosely defined “customer need” in their 1991 article “Voice of the Customer” as “a description, in the customer’s own words, of the benefit that he, she or they want fulfilled by the product or service.”¹³ Unfortunately, this definition, and the notion that it is acceptable to capture the literal voice of the customer, took companies down the wrong path. Today we know that obtaining inputs in the customer’s own words will more often than not result in the wrong inputs.

Most managers, consultants, and academics agree that companies must look beyond the customer’s own words to extract the kind of input that is needed, but they cannot seem to agree on whether or not a need is a description of customer benefit, a measure of customer value, a statement of a problem, or something else entirely. We also find that they cannot agree on how the statement should look, what information it should contain, how it should be grammatically structured, or what types of words and phrases should be used or avoided to ensure variability is not introduced into the statement—variability that can adversely affect later prioritization of unmet needs. Managers find themselves in a position that is analogous to that of a chef who knows that certain ingredients are required to produce a certain taste but is unable to figure out precisely what combination to use. And once forced into that position, getting it right becomes a process of trial and error.

Many academics, consultants, supplier firms, and others end up regarding the collection of these customer inputs as an art. In fact, some of the most popular approaches today utilize anthropologists to “seek out epiphanies through a sense of Vuja De,”¹⁴ as IDEO general manager Tom Kelley says in The Ten Faces of Innovation. Although we believe that observation can be an effective way to obtain customer inputs, we do not recommend relying on Vuja De, intuition, or what Harvard Business School professor Dorothy Leonard calls “deep smarts.” We hold that the collection of inputs, like any other business process, ought to be well controlled and optimized for success. An artful approach may result in success on occasion, but process variability must be well controlled in order to overcome the 70 to 90 percent failure rates these methods deliver.

Unlike the ideas-first approach to innovation, however, the needs-first approach is not inherently flawed, only structurally flawed—it can be made to work, as evidenced by the creation of the Outcome-Driven Innovation (ODI) methodology. ODI is an needs-first approach to innovation that has an 86% success rate—5 times the industry average. It corrects the flaws in the methods that have been used to date: namely, it links a company’s value creation activities to customer-defined metrics—a truly revolutionary concept in the field. By doing so, it supplies a definition of customer needs that the entire organization can embrace, and it offers a rigorous, controlled approach to collecting needs statements, to formulating growth strategies and to generating and validating breakthrough ideas.

Finally, ODI does not fall back on the notion that there are needs that customers cannot articulate. We’ll save that for a future article.

—

1 Robert Cooper, “Winning at New Products: Accelerating the Process from Idea to Launch,” 3rd ed. (Da Capo Press, 2001), 311.

2 Peter Skarzynski and Rowan Gibson, “Innovation to the Core” (Chicago: Strategos, 2008), 137.

3 Teresa M. Amabile, Regina Conti, Heather Coon, Jeffrey Lazenby, and Michael Herron, “Assessing the Work Environment for Creativity,” Academy of Management Journal 39, no. 5 (October 1996), 1154.

4 Tom Peters, Thriving on Chaos: Handbook for a Management Revolution (New York: Knopf/Random House, 1987), 479.

5 Scott D. Anthony, Mark W. Johnson, Joseph V. Sinfield, and Elizabeth J. Altman, The Innovator’s Guide to Growth, Putting Disruptive Innovation to Work, (Harvard Business Press, 2008), 94.

6 Given the number of possible ways that just 15 unmet needs could be satisfied by products and services in any given market, millions of ideas would have to be generated before an exhaustive set of ideas could be created. If you assume three competing ideas for each of 15 unmet needs in various combination, then you are generating ideas on the order of three to the power of 15, which is 14 million ideas. The chances of any one idea effectively addressing 15 unmet needs are one in 14 million. Furthermore, in most markets, we find there are more than 15 unmet needs.

7 Theodore Levitt, “Marketing Myopia,” Harvard Business Review 38, no. 4 (July-August 1960).

8 David Garvin, A Note on Corporate Venturing and New Business Creation (Boston: Harvard Business School Press, 2002), 5.

10 From the definition of “customer needs” in The PDMA Glossary for New Product Development (Mount Laurel, NJ: PDMA, 2006), http://www.pdma.org/npd_glossary.cfm.

11 Peter Skarzynski and Rowan Gibson, Innovation to the Core, 69.

12 A. G. Lafley and Ram Charan, The Game-Changer (New York: Crown Business, 2008), 45.

13 Abbie Griffin and John Hauser, “Voice of the Customer,” Marketing Science 12, no. 1 (Winter 1993), 4.

14 Tom Kelley makes that statement on page 17 of The Ten Faces of Innovation (New York: Doubleday, 2005). He goes on to say that anthropologists have a half a dozen distinguishing characteristics that include, for example, practicing the Zen principle of “beginner’s mind,” embracing human behavior with all its surprises, and drawing inferences by listening to their intuition. Our opinion is that this mind-set makes it all too easy for dangerous variability to creep into the need statements and the inputs themselves.