Hello and welcome to our UX research 101. In this comprehensive guide, you’ll discover everything there is to know about UX research (or user research as it's also known). 

Containing thorough how-to guidance for every step of the design and development journey, you’ll be a UX research expert in no time.

Introduction

Companies that integrate user experience (UX) research into the product development lifecycle are nearly twice as likely to outperform their counterparts on critical metrics like customer retention, conversion, and revenue.

That’s the conclusion, based on a survey of UX professionals, in our recent report, ’State of the art of UX 2022’.  

To put this into purely financial terms, UX research has a direct impact on the bottom line.  

Of the respondents that said user research is consistently integrated in their product development process, customer satisfaction scores averaged more than 80%, against 50% for those that said it was not. Net Promoter Scores (NPS) were similarly skewed at 67% versus 32%, and profitability was rated at 66% for companies with UX research fully integrated, against 37% for companies that said UX research was inconsistently, or barely, integrated.

In the words of Alfonso de la Nuez, Co-Founder of UserZoom, “…for UX research to fulfill its business-critical function, it needs to be embedded throughout the product development lifecycle. To achieve this, there has to be board-level responsibility for customer facing-experiences…”

According to McKinsey, in The Business Value of Design, “The best design performers increase their revenues and shareholder returns at nearly twice the rate of their industry counterparts.”

What is UX research?

UX research, or user experience research, is the scientific method that companies use to understand the thoughts, feelings and behaviors of the people who use their products and services. 

UX research methods can be qualitative or quantitative. They can also be moderated or unmoderated. 

Qualitative vs Quantitative user research

The precise UX research method used depends upon where the product is in the development cycle. For example, at the discovery stage (before a product has even been defined), the most popular research method tends to be interviewing stakeholders using open-ended questions. 

Typically, a group of potential customers will be recruited and invited to give their opinions about a market sector, and the products available in that sector. These tests can be carried out in person, in a series of interviews and focus groups, or remotely, using online interview tools.

These are known as qualitative methods. They give results that are verbal rather than numerical, illuminating the emotional responses of existing and potential customers.

Later in the development process, it's important to validate this qualitative data with feedback from a larger sample size of users. This can be done using quantitative research methods.

Quantitative research is any kind of investigation, experiment or study where the results can be presented with numerical values. The data you’ll uncover in quantitative research is all to do with ‘how many, how often and/or how much, etc.’

You can achieve this kind of data using online surveys, A/B testing, click testing or true intent studies.

What is qualitative research?

Qualitative refers to any kind of research where the results take the form of observations, comments, thoughts, and feelings. They tell the story of the participant’s experience. 

This type of research can be further subdivided into behavioral and attitudinal methods. For instance, a behavioral method examines how people use products. An attitudinal method would then investigate what people think of these products.

Benefits of qualitative user research

• Qualitative UX research provides insight into the hearts and minds of the people using the product.
• Qualitative responses reveal things that quantitative data can’t – for instance, if there’s a page on a website losing traffic, a qualitative test can reveal firsthand what happens when a real user visits that page.
• Participants may find it easier to describe their feelings about some things, rather than always assigning them a number or ticking a ‘yes/no’ box.
• Anecdotal evidence can sometimes be more persuasive than hard data – especially if it’s strongly felt and if stakeholders can see and hear those feelings for themselves.

What is quantitative UX research?

Quantitative research is numerical. It tells us “What” the user likes best in a product or service whereas qualitative research tells us “Why.” 

In the design phase of the product development process, quantitative methods such as A/B testing and click testing might be introduced to provide statistical support for a hypothesis developed during qualitative tests. In A/B tests users are asked to choose their preference between two different versions of a feature or design. In click testing, the tester is asked to click on a particular menu item or feature to see how easily they can find it.

These types of tests deliver data that can be subjected to statistical analysis. An A/B test would deliver a score for the two different versions of a feature or a page. A click test would record the success rate, and perhaps the time taken, for a participant to locate and click on a specific item in the menu.

Quantitative refers to any kind of research where the results can be presented in numbers, i.e. ‘how many, how often, or how much’. It is often used to add context (e.g. it took users less time to complete a task on prototype A than on prototype B) and can be thought of as the “What” data.

The analytics of a typical website – pageviews, sessions, bounce-rate, frequency of visits – are all examples of quantitative data. 

Net Promoter Scores, ease of use scores, completion rates, brand perception scores and time on task scores are all examples of quantitative data in UX research.

Benefits of quantitative UX user research

• Quantitative tests generally allow for a larger group of respondents and the data is relatively quick and easy to collect.
• Because of the numerical nature of the results it’s easy to tie UX improvements back to company KPIs, helping to justify investment.
• Quantitative research tends to be objective (i.e. it’s harder to accidentally ‘lead’ participants).
• Because of the larger sample sizes, quantitative research tends to be more robust i.e. they are more representative of the population as a whole.
• It’s easy to make persuasive visualizations from quantitative data.
• People love statistics!

Moderated vs. unmoderated user research

What is moderated UX research?

In moderated testing, a moderator observes the participants, either in-person or remotely, as they carry out specified tasks. 

The moderator can interact with users if they encounter problems, say, or clarify something the user has said. By being present they can understand the user’s behavior better and dig deeper into any issues. They can also ask for opinions and modify the script on the fly if anything unexpected happens.

What is unmoderated UX research?

Unmoderated tests are similar to moderated tests, except that they are unobserved. 

In an unmoderated test, the participant interacts with the product or service while being prompted with pre-selected questions and/or tasks. Different participants can be tested at different times, and at different stages in the development cycle, including post-launch.

Once a test is set up there is no real limit to how many participants can be included, making the sample size larger and creating economies of scale. 

Unmoderated tests require a specialized online platform or service that adds something to the cost, especially if the service provider helps recruit participants, build the study and analyze the data. But because a test can be used multiple times for multiple users, without the services of a moderator, the savings will frequently outweigh the costs. 

In summary, a moderated test can be more flexible and insightful, especially for qualitative tests. An unmoderated test can be quicker, easier and cheaper, and it can have a much larger sample size. This is especially useful for quantitative tests where the data can be analyzed statistically, rather than by hand. 

How to decide between moderated or unmoderated UX research?

As with qualitative versus quantitative testing, the best approach to using moderated and unmoderated testing is a blend of the two. Unmoderated adds a huge amount of value to an otherwise moderated, predominantly qualitative, regime by:

• Quantifying usability research: Consider a customer base that includes many different personalities, usage patterns, and perspectives. The UX team have run qualitative tests to gauge the thoughts and feelings of users about the product. By running large-scale (therefore unmoderated) quantitative tests, they can monitor personas and ensure they are catering for a true representation of their customer base. 
• Testing users in their natural environment:  Testing participants at scale, in their natural environment, introduces into the test data their different systems, setups and configurations. It also tells researchers what participants do when they are acting entirely normally, i.e. when they feel they are “unobserved”.
• To fully understand users’ behaviour: Unmoderated (therefore remote) usability testing uses a combination of web analytics, showing where users go, and surveys (why they go there) to create a complete picture of their behaviour.
• To validate or define your lab-based research: Online user testing not only provides data that quantitatively solidifies previous UX research but also can highlight new critical issues that can be brought back into the lab for further qualitative testing.
• To test globally: International research can be prohibitively expensive and time consuming. Remote user testing allows studies to be conducted in many international markets without the cost of travel, native language speakers or the need to translate results.
• To conduct benchmark studies: Unmoderated user research allows researchers to obtain usability metrics that show how a website performs against other versions of the same site, or against competing sites. Usability testing effectively creates a measure of user experience. The results can be compared against the same product’s performance over time, or against competitors through ‘industry benchmarking’.

For a full description of the most popular UX research methods, and when they are applied, see the next chapter, Which UX research methods?