Evaluating UX research methods and types for your product

How we think of and practice UX (user experience) research has evolved rapidly in recent years. What was once a specialized discipline is now viewed as something everyone in an organization can (and should) participate in. As a result, the definition of UX research is also changing. Whether you’re a UX veteran or newly introduced to the concept, user experience research is evolving and branching out.

What is UX research and why is it important?

As UX research becomes more commonplace in organizations big and small, its definitions and applications have naturally evolved. Traditionally, user experience research is the practice of studying user interactions to help with the design of people-first products and experiences.

Nonetheless, the meaning of UX can vary depending on who you’re talking to. For product teams, UX research might mean validating prototypes and concepts, and for marketing teams, it may mean testing brand designs and messaging before a launch. Each team's UX research deliverables will vary, too. In other words, UX research is no longer a practice held in one corner of the business. The most successful organizations empower all teams to collect user and customer insights in order to make better business decisions.

Types of UX research and when to use them

Before we jump into individual research methods, or the tactics used for conducting UX research, there are big-picture questions that need to be addressed first. And that’s: what types of UX research are there?

Qualitative vs. quantitative research

When it comes to understanding your users, you may find yourself wondering if your UX research approach should be qualitative or quantitative. And it’s important to figure that out because the two types uncover very different insights.

To get a full picture of your user experience, you need to understand both what’s happening and why. If you only have quantitative data, you may be missing out on key insights that could make a difference in your understanding of the user experience. And if you only do qualitative research, you won’t be able to tell whether your findings are representative of a larger population.

 

Quantitative research: What's happening with your users?

Quantitative research provides the hard numbers that can highlight patterns and trends in user behavior. This method relies on statistical data to quantify behaviors, opinions, and attitudes. Tools such as surveys, usage statistics, and A/B testing can offer valuable metrics that show what users are doing, how many are doing it, and under what conditions these actions occur.

For instance, quantitative methods can determine the number of users who abandon a shopping cart before completing a purchase, or they can measure how a new feature affects user engagement. These insights are crucial for making data-driven decisions that enhance user interfaces and streamline user interactions.

Qualitative research: Why are your users behaving this way?

On the flip side, qualitative research dives deeper into the motivations behind user actions, filling in the narrative gaps left by quantitative data. Through techniques such as interviews, focus groups, and ethnographic studies, qualitative research explores the underlying reasons for user behaviors, providing context to the numbers.

This method is especially useful when you need to understand the nuances of user satisfaction, discover pain points, and gather detailed feedback about user experiences. It helps to paint a vivid picture of why certain features are loved or disliked, and it can reveal how users emotionally connect with your product.

Integrating qualitative and quantitative research for a comprehensive understanding

The most effective UX research strategy employs both qualitative and quantitative research to create a comprehensive understanding of user experiences. By integrating these approaches, you can both quantify user behavior and understand the motivations behind it.

Starting with quantitative data to identify areas of interest or concern, you can then apply qualitative research to explore those areas more deeply. This combination allows you to not only validate broad patterns found in the data but also to explore the rich, personal insights that give these patterns meaning.

A balanced approach for deeper insights

By balancing quantitative and qualitative research, you achieve a more complete view of the user experience. Quantitative data gives you the necessary breadth to make generalizable conclusions, while qualitative data brings depth and color to those conclusions, providing a clearer direction for actionable improvements. Together, they enable a holistic approach to UX research that aligns business strategies with genuine user needs.

Attitudinal vs. behavioral research

Though sometimes misconstrued as being the same thing, attitudinal and behavioral research are not synonymous. However, as with quantitative and qualitative research, the two can be useful when assessed concurrently. 

Attitudinal research involves the assessment of users’ preconceived attitudes or feelings toward an experience. For example, this could involve asking a user why they like or dislike a feature on your site prior to using it. In contrast, behavioral research is focused on what the user does. 

Drawing another parallel to the distinction between quantitative and qualitative methods, behavioral research will tell you what’s happening, while attitudinal research helps to provide the reason why it’s happening. Always keep in mind that what users say and what users do are often different

Behavioral research: Observing user actions

Behavioral research is grounded in the observation of user behavior as they interact with a product. This method tracks what users actually do, not just what they say they do. By analyzing actions such as clicks, navigation paths, and interaction patterns, behavioral research uncovers how user behavior aligns—or doesn’t—with expected or intended use cases.

For instance, behavioral analytics can reveal if users find and use a new feature, or if they encounter issues completing tasks, allowing teams to objectively measure user engagement and identify problematic areas of a design.

Attitudinal research: Understanding user sentiments

Conversely, attitudinal research delves into the why behind user actions by exploring their attitudes, preferences, and feelings. This type of research typically involves direct feedback from users through methods like surveys, interviews, or sentiment analysis, providing insights into their satisfaction, preferences, and perceptions.

Attitudinal research can explain why a feature that performs well behaviorally might receive negative feedback, or why users request changes to elements that appear to function perfectly well from a behavioral standpoint.

Combining attitudinal and behavioral research for full spectrum insights

To fully understand user experience, it’s beneficial to combine both attitudinal and behavioral research. While behavioral data shows what users are doing, attitudinal data explains why they are doing it. For example, if behavioral data shows an increase in usage of a particular feature, but attitudinal data reveals that users feel frustrated or confused while using it, there may be deeper issues that need addressing.

By assessing both what users do and how they feel about their interactions, you can gain a comprehensive understanding of user experience, leading to more informed decision-making and better product development.

A holistic approach to user research

While attitudinal research provides context and understanding of user feelings and perceptions, behavioral research offers concrete data on user actions. Together, they provide a holistic view of the user experience, enabling UX researchers and designers to create more user-centric products that are not only functionally effective but also emotionally resonant. 

Generative vs evaluation research

The goals of generative and evaluation research (sometimes referred to as evaluative research) are very different. Generative research helps you define the problem you’d like to design a solution for. Evaluation research, on the other hand, helps you evaluate an existing design (in prototype, final, or some other form).

Generative research: Defining the user's world

Generative research, sometimes known as discovery research, is fundamentally about defining the problem space before attempting to design solutions. This type of research is exploratory in nature, aimed at uncovering the deep needs, desires, and behaviors of users that are not yet fully understood. It helps paint a broad picture of the context in which a product will be used and the potential challenges and opportunities that exist within that context.

Methodologies often employed in generative research include:

• User interviews to dive deep into the motivations and needs of individuals.
• Ethnographic studies to observe users in their natural environments, providing a richer, more authentic understanding of user behaviors.
• Field studies which involve direct observation and interaction with the user while they engage in their daily activities, offering insights that are often unattainable through other means.

Generative research is crucial in the early stages of product development when teams are tasked with defining the 'what' and 'why'—what needs to be built and why it is important. This research feeds the creative process with valuable insights that guide the design and innovation of new products.

Evaluative research: Assessing and refining the solution

Once a prototype or product iteration has been developed, evaluative research comes into play. This type of research assesses how well a product or feature performs against the intended user needs and business goals. It is more structured and tends to be quantitative, although qualitative methods can also be used to gain deeper insights.

Common evaluative research techniques include:

• Usability testing to identify how effectively users can use the product and where they encounter issues.
• A/B testing to compare different versions of a product and determine which one performs better in terms of user engagement and satisfaction.
• Surveys with closed-ended questions that provide measurable data about user satisfaction and usability.

Evaluative research is critical for refining and validating the design, ensuring that it not only meets the initial design specifications but also resonates with users and meets their needs effectively. It helps teams iterate on design solutions, making informed adjustments based on user feedback and behavior.

Integrating generative and evaluative research for comprehensive insights

While generative research is ideal for identifying new opportunities and understanding user needs, evaluative research is essential for testing and refining those ideas against real-world applications. The integration of both research types allows teams to create products that are both innovative and effective, ensuring that they not only meet the creative vision but also perform well in practical use.

By understanding the distinct roles of generative and evaluative research, teams can better plan their UX research activities to support all phases of the product development lifecycle—from conception to launch and beyond.

UX research methods

UX needs to be a strategic initiative that drives a culture of user-centric design and thinking—informing everything from the product itself to marketing campaigns and messaging to brand design and social media. There's a variety of UX research tools and user research techniques that help teams collect the insights needed specific for every role.

Let’s take a look at some of the more common methods now.

Remote usability testing

It might be obvious that remote usability testing is a great method for conducting UX research—if only by its name alone. This method of remote research uses an insight platform to record the screen (and voice, depending on the software you choose) of test participants as they interact with your product or experience in their natural environment—at home, in their office, or a specific location.

Through usability testing, designers, product managers, and researchers alike can uncover and understand how real people respond to products and experiences. From what they like and dislike, to where they get stuck and confused, to areas of improvement, the valuable insights gathered from these tests are eye-opening.

Diary studies

Diary studies are a form of longitudinal research (research that takes place over a long period with the same participants). Typically, users self-report their activities at regular intervals to create a log of their activities, thoughts, and frustrations. It’s a useful approach for capturing organic feedback on activities that are repetitive, long, or unpredictable.

The value of diary studies

Optimal scenarios for diary studies:
Diary studies are most beneficial when exploring behaviors or experiences that unfold over longer periods. They are ideal for:

• Extended processes: Observing how users engage with a process that takes time, such as adopting new technology or navigating significant life events like purchasing a home.
• Behavioral triggers: Discovering the catalysts that prompt specific user actions, such as reaching for a health-tracking app or initiating a digital payment.
• Routine integration: Understanding how services or products become part of daily habits, affecting user retention and satisfaction.

These studies provide a depth of insight that snapshots from brief interactions simply cannot match, offering a richer, more contextual understanding of the user journey.

Strengths and challenges of diary studies

Benefits:

• In-depth insights: Over time, diary studies can reveal profound insights into user behavior and psychology that other methods might miss due to their limited observation windows.
• Rich contextual data: By capturing real-time data, diary studies provide a detailed view of how products or services fit into the fabric of everyday life.

Considerations:

• High participant commitment: The intensive nature of diary studies requires a significant time commitment from participants, which could lead to dropout if not properly managed.
• Complex data handling: The qualitative data from diary studies can be voluminous and complex, requiring meticulous analysis to derive meaningful conclusions.

Conducting effective diary studies

Preparation and participant engagement:
Successful diary studies begin with thoughtful planning and careful participant selection. Maintaining flexible demographics and providing clear, comprehensive instructions are crucial to participant retention and data quality. Adequately compensating participants for their time is also essential given the extended nature of their involvement.

Instruction clarity:
Participants should receive explicit instructions detailing the frequency and type of diary entries required, whether textual, visual, or both, and any specific focus areas or questions to address.

Pilot and iterate:
Piloting the study with a few participants can help identify any issues with the study design and instructions, ensuring everything is clear and the study is set up for success.

Enhancing insights with interviews

Initial interviews:
Starting with an in-depth interview can help set clear expectations, build rapport, and encourage thorough participation throughout the study period.

Concluding interviews:
Post-study interviews allow for deeper exploration of diary entries, helping clarify ambiguities and delve deeper into significant findings.

Integrating with other research methods:
While powerful on their own, diary studies can be even more informative when combined with other research methods, such as surveys or usability tests, to build a comprehensive picture of the user experience over time.

In essence, diary studies offer a unique lens through which to view the user experience, capturing the ebb and flow of daily life and its impact on product engagement and satisfaction.

Card sorting

Card sorting is a qualitative research method used to group, label, and describe information more effectively—based on feedback from customers or users.  Card sorting requires you to create a set of cards—sometimes literally—to represent a concept or item. These cards will then be grouped or categorized by your users in ways that make the most sense to them. Most commonly, it’s used when designing (or redesigning) the navigation of a website or the organization of content within it, because it helps to evaluate information architecture.

Surveys

It may seem obvious, but through a series of expertly-phrased and positioned questions, surveys allow you to empathize with your users in order to gain quantitative insights that aren’t as visible to developers, managers, and marketers. Listening to your customer can help you find new problems to solve or devise new ideas, and collecting customer feedback through surveys is an active, receptive, and honest way to do it.

Designing effective surveys

Key considerations for survey design:

1. Ask the right questions: Tailor your questions to elicit the most informative responses. This involves understanding the kind of information you need and framing your questions to minimize bias, ensuring they are clear and direct.
2. Incentivize participation: Motivate your respondents by offering rewards or explaining the value their feedback provides. This not only increases the response rate but also encourages thoughtful, honest answers.
3. Simplify the process: Ensure your survey is concise and straightforward. Managing expectations upfront about the time commitment and providing a seamless user experience can reduce dropout rates.
4. Prioritize accessibility and readability: Good UX design is crucial. Surveys should be easy to read and accessible on all devices, using clear fonts, contrasting colors, and an intuitive layout.

Why customer feedback is crucial

Feedback is essentially a dialogue between you and your customers. It helps you understand their needs, preferences, and pain points. Continuous feedback throughout the product development cycle allows you to make iterative improvements, ensuring that the final product resonates well with your users. Real-time insights from surveys can guide your decision-making process, helping you solve the right problems and innovate effectively.

Leveraging survey feedback

Immediate and long-term benefits:

• Identify new challenges: Surveys can pinpoint areas that need attention, revealing both new opportunities and existing flaws.
• Drive product evolution: Regular customer feedback helps you keep pace with users' evolving expectations, ensuring your product or service remains relevant.
• Enhance customer satisfaction: By acting on survey feedback, you demonstrate that you value customer input, fostering loyalty and enhancing overall user satisfaction.

Building on survey insights

While surveys provide valuable quantitative data, integrating these findings with qualitative research, such as interviews or observational studies, can enrich your understanding. This mixed-method approach allows you to not only quantify trends but also explore the reasons behind them, offering a comprehensive view of your user experience.

Interviews

Live interviews are a great way to collect qualitative insights. By having dynamic discussions, interviewees are able to observe verbal as well as non-verbal cues and ask open-ended questions to uncover those details that surveys and usability testing cannot. Interviewing is an especially useful UX research method for understanding complex feelings and experiences because it allows you to ask follow-up questions.

When should UX research be conducted?

Early on, UX research was most commonly used to solve a problem that was already known. For example, if a company noticed that visitors were dropping off their site at an unusually high rate on a particular page, researchers would look into how to solve that specific problem. As a result, research wasn’t necessarily something that was done for discovery or as a regular part of the development process.

But that’s a thing of the past. The value teams are getting from fast human insights is driving UX research best practices to become embedded in everyday processes for teams across the business. What was once just a problem-solving mindset has evolved to combine the perspective of finding what problems to solve, as well. 

This is where researchers flex their strategic skills, and companies are getting creative with user experience research to provide valuable insights. As any consumer will tell you, there are countless experiences that leave us underwhelmed, if not disappointed, that might not be so obvious to spot.

For example, if you visit an e-commerce site, find a product, and purchase it, there isn’t a lot of information that would trigger anyone to wonder if the purchasing experience needed any attention. After all, a purchase was made, right? Discovering what problems need solving, in addition to solving the ones you know need attention, is a big shift in UX research mentality.

A final word on UX research

UX, CX, usability, user testing. No matter what you call it, putting your customers at the center of your company’s mission and culture has become a competitive advantage that not only attracts new customers but keeps them coming back.

This expanded view and access to user experience research mean that not only are companies better equipped than ever to create and improve great experiences for their customers, it also means that customers will be expecting better and better experiences in the future.