Understanding the problem space and its role in development

“People don’t want to buy a quarter-inch drill. They want a quarter-inch hole!” — Theodore Levitt

This quote is the shortest and maybe the best explanation of two concepts: the problem space and the solution space.

Another way of looking at this would be:

“If I had asked people what they wanted, they would’ve said faster horses.” — Henry Ford

You need to work in the problem space not only in the solution space to create real innovation and great products.

In this article, you’ll learn about the problem space and how it leads to and differs from the solution space. You’ll also come to see why the concept is important for success in product management, design, and agile product development.

What is the problem space?

The problem space is a concept or a term that focuses on identifying and understanding the problems and needs of a customer. It’s the foundation for the tasks that are later performed in the solution space and leads to the requirements for the product. These requirements are then used to develop the features which are bundled together to a solution.

The goal is to create a clear basic understanding of the user and their needs in the problem space to create a value proposition and a product that solves the problems of the user or satisfies their needs.

In the problem space, you learn about the market, the user, the needs of the user, and the jobs to be done. Based on this information and research the product team will define a value proposition in the solution spaces and craft a product which, when done right, will lead to a great product market fit.

In other words, you need to invest in both the problem space and the solution space to find great product-market fit. Tasks in the problem space include deep research and analysis to gather insights about the customer’s needs, pain points, behaviors, and experiences of the customers.

The problem space in design thinking

If you want to work user-focused you need a good grasp of the problem space. The role of the problem space in design, especially in user experience (UX) is pivotal for the success of a product manager. Exploration of the problem space leads to great design concepts and ultimately to good product-market fit.

To reach that fit, observing the market, its users, and their needs is key. After broad observation, the product owner has to decide and narrow the field down to conceptualize and focus on the most important needs. The double diamond is a concept that shows exactly that. You have to explore the problem space more widely and deeply (divergent thinking) first, before focusing on your action and conceptualizing (convergent thinking):

To give you a better understanding of the different tasks of a product manager in the problem space, the following tasks are key.

Building empathy with your users

In the discovery, one of the first steps is to build empathy with the users. This involves understanding not just the practical issues and the jobs to be done, but also the emotional and psychological experience of the users. Such empathy is essential to understand the user’s pains and help to resolve them with the right product.

Identifying the right problems

By observing and understanding the users, you can identify the right problems to focus on and narrow the field again to later conceptualize the right design. A well-defined problem space helps clarify the rest of the product development process. This is very critical because the quality of the solution is directly tied to how well the problem is understood.

Guiding to the right design decisions

As soon as the set of problems that need to be solved are identified the product owner can narrow the field more and conceptualize the design and ideate the solution. Insights from the problem space are used to create informed design decisions throughout the whole process. This also includes decisions about features, interfaces, interactions, and even aesthetics in the solution space.

Test the right things

Understanding the problem space allows for more effective testing and iteration of designs. By knowing what needs to be solved, designers can create more focused prototypes and tests, gather relevant feedback, and iterate in a way that progressively hones in on the most effective solution. In the solution space, you have to widen the field again, ideate, and create different solutions to test against the insights gathered from the problem space.

Create the right long-term vision and a winning strategy

A thorough understanding of the problem space contributes to a long-term vision for the product. It helps in anticipating future user needs and market trends, ensuring the product’s relevance and adaptability over time.

Create user-centered designs

The overall goal of great products is to create solutions that are tailored to the user’s needs. The problem space is where designers learn what these needs are, and by thoroughly understanding it, designers create solutions that are innovative, and relevant for the users. The problem space puts the user in the center and delivers valuable solutions for them in the solution space.

The problem space is not just the starting point of the design process, it’s a continuous reference point that ensures the user remains at the center of the design process, leading to more effective, empathetic, and successful design outcomes.

The problem space in agile methodologies

Combining agile methods with the concept of problem space and solution space can help to react to changes in the market and ensure success with a higher probability. Agile methods emphasize adaptability, collaboration with the customer, and the frequent delivery of functional product increments. Let’s take a deeper look at where the problem space can have an impact on agile product development.

Continuous discovery

Agile frameworks prioritize ongoing discovery over fixed planning. The problem space is continuously explored and redefined through user feedback, market changes, and new insights. Discovering the user needs and the market in discovery sessions is therefore a key aspect of an agile development team.

We often use behavior driven development techniques like user story mapping or example mapping to uncover hidden aspects of the problem space.

Since the problem space is also about understanding the emotions of the users we tried to combine user story maps with customer journey maps — emotion graphs. This helped us to understand the pain points of the users better:

User stories and backlog refinement

In agile, user stories often represent problems to be solved rather than features to be built. During backlog refinement sessions, these stories are discussed, prioritized, and detailed. This process is fundamentally about understanding and defining the problem space, ensuring that the team focuses on the most valuable and relevant problems in each iteration.

We often use the following template:

As a <<user>> I want to <<do something>> so that I << accomplish something>>

As a <<who>> I want to <> so that I <<why>>

Example:

As an eBanking customer, I want to send money online so that I am independent of physical banks and their opening hours.

For the development team, it’s often harder to understand the problem space. That’s why the most powerful thing about user stories is that the user story explains the who, what, and why. This helps the development team to dive into the problem space.

Sprint planning and goals

Agile teams use sprint planning to decide what problems they’ll tackle in the upcoming iteration. By focusing on problems rather than solutions, teams can adapt their approach as they learn more, which is crucial in an environment where change is expected and embraced.

Sprint goals serve a similar purpose to user stories. They focus the team on why the outcome of a sprint occurs. We love to use the following template for our sprint goals:

Our focus is on <<output or topic>>
We believe it delivers <<outcome>> to our <<users or customers>>
This will be confirmed when <<measure or event>> happens

Iterative development and feedback loops

Agile methodologies, with their iterative development cycles, allow teams to explore the problem space incrementally. Each iteration or sprint provides an opportunity to receive feedback, learn from real user interactions, and refine the understanding of the problem. This iterative approach reduces the risk of building something that doesn’t meet user needs, but only if you show your product increments to your users, release early and often, and invite real users to the reviews.

We try to invite our key users to every review to gather feedback. But even more important, we show them our work even before a ticket or a feature is done during the sprint and we often reach out to our stakeholders and present them with the implementation. We tend to learn something and work on the feedback right away.

This is key to prepare for the review meeting with your users. We often use the following structure:

• Warm-up phase — To set the stage and invite them to give feedback
• Main feedback phase  —  To gather insights
• Wrap-up phase  —  To summarize and give an outlook

Collaboration and cross-functional teams

Agile teams are often cross-functional, bringing together diverse perspectives. This diversity is beneficial in exploring the problem space, as different team members may see different aspects of a problem. Regular collaboration and communication help in building a shared understanding of the problem space.

Everyone in the team must be focusing on the problem space. Because of this, it’s useful to integrate design and the problem space in the whole cross-functional team by focusing on the why and using models like a value proposition canvas.

Customer collaboration over contract negotiation

Agile values customer collaboration, which is essential in continuously exploring the problem space. Regular interactions with customers and stakeholders help refine the problem understanding and ensure that the team is always working on the most pressing and relevant issues.

But this close collaboration may not be for everyone. Often software engineers tend to avoid contact with customers. To challenge this behavior we constantly work on this mindset in our retrospectives.

Responding to change over following a plan

Agile’s emphasis on responding to change is inherently linked to its approach to the problem space. As understanding of the problem evolves, agile teams are expected to adapt their plans and priorities accordingly, ensuring that the solution remains aligned with real user needs and conditions.

It’s clear to me that if agile teams apply agile methods and understand the concept of the problem space, the likelihood of the product team finding product-market fit increases.

The problem space vs. solution space

I often experience that development teams have a hard time distinguishing between the problem space and solution space. However this distinction between the problem space and the solution space is fundamental in product development and understanding this difference is crucial for successful outcomes. Here’s a table of key aspects of both spaces:

Examples of problem spaces

To help nail down the concept of problem spaces, let’s take a look at some real-world examples from products you may know.

Space Pen

Have you ever heard the story of the Space Pen?

The story of the Space Pen (or Fisher Space Pen) is a famous anecdote in the world of product management, innovation, and design. It’s a perfect example of the differentiation between problem space and solution space.

During the space race in the 1960s, astronauts needed a writing instrument that could function in the zero-gravity environment of space. Traditional pens rely on gravity to draw ink onto the paper, which would not work in space. Once this fact was identified, the Fisher Pen Company set out to build a pen that would work in zero gravity. After Paul Fisher spent about one million dollars on a pressurized ink cartridge NASA sent its first Space Pen on a flight in 1965.

In the meantime on the other side of the world, with an ongoing Cold War, the USSR sent its first cosmonauts into space with a pencil. They faced the same physical challenges (problem space), but had their solution (solution space) a standard pencil.

Often the anecdote ends here, but to be fair, NASA had its reasons and also sent the first astronauts in space equipped with ordinary pencils. The problem with pencils is that they tend to break, leaving debris and graphite dust floating around in the spacecraft, which can be both a fire hazard and a danger for electronic devices, since graphite is a conductive material.

The problem space here was the need for a writing instrument that could function effectively in the unique conditions of space. It involved understanding the challenges of zero gravity, the safety hazards of using pencils, and the requirements of astronauts.

The exploration of this problem space requires a deep understanding of the space environment and the practical needs of astronauts.

The solution space involved the development of the Fisher Space Pen. This was an innovative approach to solving the problem, as it did not rely on gravity to deliver ink.

The development of a pressurized ink cartridge was a technical solution that effectively addressed the defined problem.

The story of the Space Pen illustrates the importance of correctly identifying and understanding the problem space before jumping to solutions. Instead of trying to modify existing pens or persisting with pencils, one has to truly understand the challenges and problems that also arise with other solutions (like in this case pencils, their hazards and risks).

It’s a classic example of innovative thinking that goes beyond the obvious solution and instead rethinks the problem by focusing on the unique conditions and requirements of writing in space.

Dyson vacuum cleaners

Another example of problem space versus solution space would be Dyson vacuum cleaners. The founder, James Dyson, was frustrated with his conventional vacuum cleaner losing suction power over time. He identified a broader problem in the vacuum cleaner market: most vacuums lose suction over time due to their bags and filters filling up (problem space). Dyson therefore developed and patented the first bagless vacuum cleaner using cyclonic separation, addressing the core problem of sustained suction power (solution space).

Spotify

Before Spotify, the music industry was grappling with issues like piracy and inconvenient access to music. The problem space was defined by consumers’ desire for easy, affordable access to a wide range of music (problem space). So Spotify invested in streaming and the streaming service addressed all these issues by offering legal, on-demand access to a cast library of music. Thereby they changed the whole industry, solved the privacy problem for the big labels, and provided a better way for people to access and pay for music (solution space).

Identifying and analyzing problem spaces

Analyzing the problem space is no one-time thing. A successful product development team will always analyze the problem space and identify user needs and market changes. We use different kinds of approaches and methodologies depending on the situation and the phase of the product development.

User research

We use both quantitative and qualitative methods to gather insights from the market and the users. Interviews, focus groups, and ethnographic studies help gather in-depth insights into user needs, behaviors, and motivations. Surveys, questionnaires, and data analysis provide statistical insights into user preferences and trends.

Design thinking

Especially in the beginning design thinking can be used to ideate and brainstorm solutions based on a clear problem understanding and understanding of the users and their needs.

Customer journey mapping and user story maps with emotion graphs

Mapping out the customer’s journey with a product or throughout a process or feature helps identify pain points and moments of truth where improvements are needed.

Personas

We create detailed personas at the beginning of product development and always update them as more is learned on our journey to product success. Detailed personas help us understand various user segments and their unique needs and challenges. We link personas to features and user stories, this helps us to see the user story through the eyes of a persona.

Example mapping

We use example mapping to identify concrete examples often in combination with personas to identify the right test cases and therefore to create the best solutions matching the problem space.

Market research and competitive analysis

We keep an eye on our competition and analyze market trends to stay ahead of our industry and its trends.

Key takeaways

In essence, the exploration of the problem space is a fundamental aspect of successful product management, design, and agile development. The key lies in understanding that real innovation and impactful solutions stem from a deep and empathetic comprehension of the user’s needs, challenges, and experiences. It’s important to distinguish between the problem space, where the focus is on identifying and empathizing with user problems, and the solution space, which is dedicated to developing and implementing solutions to these problems.

For product managers, designers, and agile teams, the continuous journey of exploring and re-evaluating the problem space is crucial. This process involves employing various methodologies like user research, design thinking, customer journey mapping, and competitive analysis. By doing so, teams can ensure that their solutions not only address the current needs of users but are also adaptable to evolving market trends and user preferences.

Featured image source: IconScout

The post Understanding the problem space and its role in development appeared first on LogRocket Blog.