Product Discovery Anti-Patterns

TL;DR

Scrum has proven to be an effective product delivery framework for digital products like applications or apps. However, Scrum is equally suited to build the wrong product efficiently as its Achilles heel has always been the product discovery part. What product discovery part, you may think now. And this is precisely the point: The product owner miraculously identifies what is the best way to proceed as a team by gating and prioritizing the product backlog. How that is supposed to happen is nowhere described in the Scrum Guide. Consequently, when everyone is for himself, product discovery anti-patterns emerge.

Learn more about the numerous product discovery anti-patterns that can manifest themselves when you try to fill Scrum’s product discovery void.

Scrum’s Achilles Heel: Product Discovery

In the attempt to fill Scrum’s product discovery void, product delivery organizations regularly turn to other agile frameworks like lean UX, jobs-to-be-done, lean startup, design thinking, design sprint — just name a few. The wave of agile transition projects particularly in large, established organizations over the course of the recent years has provided those frameworks with a tremendous tailwind. Some ideas have meanwhile gained buzzword status in the process causing the occasional collateral damage along the way. (I stopped correcting stakeholders of my current project — mainly from the business side — when they use “MVP” in an incorrect way, for example.)

Generally, it is safe to assume that agile product discovery anti-patterns result from a broader set of issues by comparison to the anti-patterns of a particular framework such as Scrum. (Download for free: The Scrum Anti-Patterns Guide.)

The main contributing variables to various product discovery anti-patterns are:

• Existing organizational dysfunctions, for example, the organization is structured in functional silos,
• A substantial degree of ego issues among individual players — the what-is-in-for-me-syndrome — resulting in personal agendas being pursued,
• A complex, multi-layered reporting structure within organizations that filters as well as delays the flow of information, thus impeding communication and decision-making.

Additionally, particularly larger organizations struggle with the necessary transition at the core of the process of becoming an agile, learning organization: The move from allocating budgets to projects staffed with temporary teams group of FTEs to building and funding lasting product teams.

I do believe that a separation between product discovery on the one side (product management), and product delivery (engineering) on the other side is nowadays no longer a viable approach. To prevail in today’s game of an accelerated innovation-based competition — software is eating the world — , every organization needs to acquire a holistic understanding of an agile product creation process:

A vision leads to a strategy which (probably) results in a portfolio of products (and services). Each of those products has a product roadmap that needs to be reflected in the product’s backlog, which ultimately provides the sprint backlog. This core agile product creation process is agnostic to the product delivery framework, be it Scrum, XP, or any other framework.

The organization needs to inspect and adapt every layer of this hierarchy continuously within appropriate time intervals. Apparently, there is a difference in the inspect & adapt cadence when strategy and sprint backlog are compared to each other. (Example: The Secret Tesla Motors Master Plan (just between you and me) from August 2nd, 2006.)

Keeping this holistic product creation process in mind, you can often observe some of the following product discovery anti-patterns in practice.

Please click the “clapping hands” 👏_, if you found this post useful–it would mean a lot to me!_

Product Discovery Anti-Patterns: The Fallacy of Knowing Better

• ‘We know what to build’: The is no user research, nor any other interactions of the product delivery organization with customers. (There are several reasons causing this phenomenon ranging from a founder or entrepreneur who pursues his or her product vision without engaging in customer discovery activities. Or the product delivery organization is solely briefed indirectly by key account managers. Probably, the sales department deems a direct contact of the product people with customers too risky and hence prevents it from happening. What these patterns share is either a bias that is hurting the learning effort or a personal agenda. While the former can be overcome by education, the latter is more difficult to come by as the culprits typically reject the idea that they are guided by selfish motives. For becoming an effective product delivery organization it is essential that the team directly communicates with customers at a regular base.)
• Loving the solution: This is a variation of the ‘we know what to build’ syndrome. (There are no ugly children at least not from their parents’ perspective. This bias seems to apply to products and services, too, think of Ikea: people value what they created over other products. Which is also the reason that you avoid asking strangers for their opinion: They might not be sharing your conviction. The solution to this product discovery anti-pattern is simple: Fall in love with the problem, not your solution.)
• No shared understanding: The product delivery organization does not make any attempts to create a shared understanding among the team and the stakeholders on what to build for which reason. (The best way to secure an engaged and effective communication with internal and external stakeholders is to include them in the product creation process. People care for what they help create. The best way to do so in my experience is Jeff Patton’s user story mapping. It is not just a great way to turn ‘requirements’ into feasible user stories at a tactical level. User story mapping is also very well suited to come to an understanding on product roadmap and release planning.)
• Persona-driven self-fulfilling prophecy: You create the personas first based on what you believe to be the customers or users of our next [add an adjective with a positive connotation here] product or service will be. (This is a potentially dangerous mental model as you may fall victim to a self-fulfilling prophecy. It is not unlikely that you unconsciously create user tests in a way that will verify your personas in the process. Instead, you should flip the process: first, you observe potential customers in their natural habitat to figure out a problem worth solving. Then you create a prototype addressing that previously identified problem. After the prototype, you create preliminary personas from the insight you gained during the related research. “Preliminary” needs to be stressed, too, as the persona design needs to be constantly calibrated with what you learn while continuously running user tests. There are no static personas. Read More: “Creating Personas.”)
• Annual roadmaps: Product roadmaps are planned once a year for twelve months in advance. (Contrary to popular belief in traditional command & control organizations product roadmaps are subject to continuous planning. The agile product delivery process is centered around the idea to work solely on those ideas that provide the highest value to customers and the organization at any given time. To meet that standard the product roadmap needs to be adapted to the learnings from running product experiments regularly in an appropriate cadence.)
• The incrementalism trap: You get so focused on improving incrementally that you miss the available innovative leap ahead. Remedy: Apply Elon Musk’s first principles thinking model and break down a situation into its elementary pieces to put it back together again in a way that serves your purpose better.

Agile Transition — A Manual from the Trenches

The latest, 171 pages strong version of “Agile Transition — A Hands-on Manual from the Trenches” is available right here**,** and it is FREE!

Product Discovery Anti-Patterns: Silos at Work

• Design as a silo: There is a UX/UI design team/department that is organized as functional silo outside the development team. (UX/UI competence is essential for a lot of cross-functional teams. If those competencies are only available by sourcing them from a different entity, the organization creates an artificial hand-over. The interruption will impede both speed and quality of the team’s work as crucial information and learnings will only be relayed but not experienced first hand.)
• User tests/user research as a silo: User tests need to be handed over to a functional silo. The researchers will be reporting back once the results are in. (User tests are an integral and thus continuous part of the product discovery process. Hence, user research shall be run by the team itself by embedding someone with the necessary competencies into the team. Read More: How to Run Lean User Tests.)
• Don’t waste engineers on user tests: The engineers do not participate in user tests because they are deemed too valuable/expensive. (They are supposed to code as it is easier to hire an additional UX designer by comparison. This is the wrong mental model: the earlier engineers participate in identifying a problem worth solving they better the ROI on their engagement will become as they will bring the technical expertise on feasibility to the table. Building the wrong thing due to isolating the engineers is for several reasons incredible expensive. It is a huge waste of resources, and there will be significant costs of delay — the team could have been built something more valuable in the meantime. Lastly, it is incredibly frustrating to see your effort go to waste, no matter what you were paid to make it. (Read More: (1) Daniel Pink: Drive. (2) In Chapter 2 — “The Meaning of Labor” — of his book “The Upside of Irrationality,” Dan Ariely describes an experiment dubbed the ‘Building of Bionicles’ (Paperback edition 2011, pages 66 to 77) where Harvard University students were paid to assemble Lego figures. The experiment aimed at a better understanding of people’s reactions to small reductions in the meaning of their work. His conclusion: “The translation of joy into a willingness to work seems to depend to a large degree on how much meaning we can attribute to our own labor.” (See above, page 74.))
• No engineers or designers working in customer support: Engineers or designers do not work with customer support from time to time. (Have you heard of “support-driven development,” that everyone in a company should spend five percent of his or her time in customer support? This is not a new idea, and a lot of successful companies practice it to simplify the flow of information from a client’s problem to its solution. It proves to be an effective way to discover pain points and create products that clients love. Kayak.com co-founder Paul English: “Customers are a big source of my e-mails. Anytime anyone contacts us with a question, whether it’s by e-mail or telephone, they get a personal reply. The engineers and I handle customer support. When I tell people that, they look at me like I’m smoking crack. They say, “Why would you pay an engineer $150,000 to answer phones when you could pay someone in Arizona $8 an hour?” If you make the engineers answer e-mails and phone calls from the customers, the second or third time they get the same question, they’ll actually stop what they’re doing and fix the code. Then we don’t have those questions anymore.” Read More: Everyone on Support.)
• A sole source of truth: The product delivery organization relies solely on either quantitative or qualitative feedback. (You need to take both sources into account to come to an informed decision. Data does not provide any information on ‘why’ things are happening. Data only addresses ‘what’ is happening. At the same time, focusing exclusively on qualitative feedback may make you vulnerable to bias and unfavorably mental models. The road to self-fulfilling prophecies is paved with good intentions.)

Product Discovery Anti-Patterns: Cargo-Cult Discovery

• Would you use this feature? Your product discovery process equals asking prospective customers what problems they have. (“It’s really hard to design products by focus groups. A lot of times, people don’t know what they want until you show it to them.” The famous Steve Jobs quote that everyone in product design and management has heard of before. Fact is, that it is useless to ask prospective customers in advance what features they would appreciate in a new product. Whether you send questionnaires or you ask them directly in an interview, they will usually fail to imagine both the situation as well as the new product in a way that will provide you with actionable insight. The problem is called “psychological interference” due to the high level of required abstract thinking. Read More: Four Lessons Learned from Making Customer Value Your Priority.)
• Selling non-existing features: What features do you need us to provide to close the deal? Sales managers chase sales objectives by asking prospects for a feature wish-list and provide those to the product delivery organization as requirements. (The problem with customers is that they usually lack the depth of knowledge required to provide useful answers to this question. Most of the time, they also lack the level of abstract thinking necessary to come up with a viable, usable, and feasible solution. As the saying goes: if the only tool you are familiar with is a hammer every problem will look like a nail. Pursuing the sales process in such a way will lead the product into a feature comparison race to the bottom, probably inspired by bonuses and personal agendas. This is the reason why product people like to observe customers in their typical environment using a product to avoid misallocating resources on agenda-driven features. At a systems level, reconsidering individual monetary incentives for salespeople is helpful, too. In a learning organization, teams win not individuals.)
• What features do you need this year? To be fair, some product managers behave in a similar way like sales when they ask internal customers what features are required. (A call to send in your requirements typically ends up with an epic, Christmas-like wish-list of “requirements.“ If the resources of the development team are basically free of charge there is an incentive to grab as much of those as possible by extending the list whether those requests are valuable to the organization or not. You might also observe that stakeholder define requirements that they do not expect to be fulfilled just to gain leverage in future “negotiations” with the product delivery organization. This submission process is normally motivated on the stakeholder side by optimization efforts leading to a local maximum. (Note: Internal stakeholders acting this way behave perfectly rational under such conditions.) Instead, the product delivery organization needs to maximize the outcome of the whole organization. This maximization effort requires entering a competition of ideas across all internal stakeholders to channel resources to the most valuable ideas.)

Product Discovery Anti-Patterns: Ideas, Hypotheses, and Feedback

• #NoTransparency: There is no transparency how the product delivery organization chooses ideas for the hypotheses shortlist, and how those ideas make it from that shortlist to the experiment backlog. (Establishing the principle that internal stakeholders have to enter a competition of ideas to avoid creating local maxima at the expense of the organization is in itself challenging. The prerequisite for a successful transition to that principle is transparency on the one hand, and a competition watchdog on the other hand. Otherwise, stakeholders might opt out of the process as their efforts appear futile, and rumors might arise: Are HiPPOs driving the selection process? Or will those who shout the loudest be privileged when ideas make it into the backlog of experiments? Do I have to pitch my ideas at 2 a.m. in a bar to the product owner to be heard? Make sure that everyone understands how and why decisions are made. Visualizing both the hypotheses funnel as well as the backlog of experiments has proven to be supporting.)
• No feedback process: There is no process how the product delivery organization deals with suggestions for new features or other ideas, and feedback is provided randomly at best. (It’s a good idea to have a process that actively involves stakeholders and members of the organization in the product discovery process. People like to have a purpose and to be a part of something larger than themselves. Providing everyone in the organization an opportunity to contribute, without regard to rank, makes working as a product owner easier. Such a process doesn’t require fancy technology — a simple shared spreadsheet or form is enough to get it started. Initially, a template to suggest new product features could consist of questions that address the why, the how, the what, and the for whom. It could address the tactical or strategic nature of the suggestion, a possible time-frame, or an estimate of the expected return on investment. Most importantly, designing a process to handle product ideas should be kept agile: the process should start with a simple solution, and then be improved once initial feedback from stakeholders on the process has been analyzed.)
• Ignoring sales: The sales organization is not effectively included in the hypotheses creation process. (Beware, though, of the tendency of salespeople to demand new features once the quarter is nearing to meet sales targets to secure bonuses. This opens the discussion ns of bonuses is general; a learning organization should not provide individual financial incentives. The possibility of a moral hazard is otherwise too large: Using the organization’s resources for free to meet objectives that are personally beneficial is all too human. Do you remember the origins of the financial crisis from 2008?)
• Ignoring customer support: The customer support organization is not adequately included in the hypotheses creation process. (To my experience, cooperating closely with the customer care team is most beneficial for any product delivery organization. Failing to include them is one of the worst product discovery anti-patterns.)
• Hypotheses selection without engineers: The product management does not include engineers early in the process of identifying potential hypotheses to test. (Idea is worth nothing, execution is. Usually, a product delivery organization is flooded with ideas from everywhere: customers, sales, internal stakeholders, HiPPOs, friends, and family — you name it. The most important job of the product people is hence bringing order and transparency to the chaos by establishing a process that turns the long list of ideas and requirements into a short-list. The purpose of this short-list is to run experiments that verify or falsify short-listed ideas as only valuable, usable, and feasible ideas can become a part of the product backlog. To do so, it has proven useful that a triumvirate from product management (business), design and engineering decides which of the items on the hypotheses shortlist make it into the backlog of experiments. Excluding the engineers from that decision process flaws the whole process from the beginning.)

Product Discovery Anti-Patterns: Egos at Work

• HiPPO-ism: The product discovery process is at least partly driven by the beliefs of individuals of the higher management caste. (This can be best described with the famous quote from Jim Barksdale, the then CEO of Netscape: “If we have data, let’s look at data. If all we have are opinions, let’s go with mine.” Think about it: how does the pay-grade, and hence the individual’s position in the social hierarchy of the organization, qualify that person to have the ‘right’ idea? The Macintosh was a brain-child of Steve Jobs, and he deserved credit for that. He also almost killed it prematurely by refusing to add expansion slots to the design. I do believe that the Macintosh II made the iPhone possible.)
• ‘My budget’ syndrome: Stakeholders do not pitch for development resources but claim that they allocate “their” budget on feature requests as they see fit. (That process leads to the creation of local optima at a silo or departmental level. The effect can be observed particularly in organizations, that tie additional benefits to individuals. Instead, resources need to be allocated in the spirit of optimization for the whole organization. Note: ‘Pet projects’ also fall in this category.)
• Sunk cost fallacy: “We have already invested so much. Let’s finish it.” (No matter how much has already been spent, past expenditure does not justify continued work on a product that provides little or no value.)
• Bonus in limbo: We are nearing the end of a quarter. Bonus relevant KPIs (key performance indicators) are at risk of not being met. The responsible entity demands product changes or extensions in the hope that those will spur additional sales. (This behavior is comparable with the ‘what features do you need to close the deal’ anti-pattern, but it demanded in more pressing fashion, typically four weeks before the end of a bonus period.) Financial incentives to innovate: The organization incentivizes new ideas and suggestions monetarily. (Contributing to the long list of ideas and thus the hypotheses short-list should be intrinsically motived. Any possible personal gain might inflate the number of suggestions without adding value.)

Product Discovery Anti-Patterns: Conclusion

A holistic product discovery & delivery process is at the heart of any learning organization. Filling the product discovery void of the product delivery framework of your choice becomes, therefore, is a necessity. And there are plenty of agile frameworks and methodologies to choose from for that purpose. Watching out for the product discovery anti-patterns listed above will save your organization significant resources in doing so.

What product discovery anti-patterns am I missing? Please share with me in the comments.

Product Discovery Anti-Patterns — Related Posts

28 Product Backlog and Refinement Anti-Patterns

Download the ’Scrum Anti-Patterns Guide’ for Free

Please click the “clapping hands” 👏_, if you found this post useful–it would mean a lot to me!_

Do you want to read more like this? Well:

• 📰 Join 12,366 peers and sign-up for this weekly newsletter
• 🐦 Follow me on Twitter and subscribe to my blog Age of Product
• 💬 Alternatively, join 1,900-plus peers of the Slack team “Hands-on Agile” for free…

Product Discovery Anti-Patterns Leading to Failure was first published on Age of Product.