Learn everything there is to know about UX research in this extensive five part guide.
Hello and welcome to our UX research 101. In this comprehensive guide, you’ll discover everything there is to know about UX research (or user research as it's also known).
Containing thorough how-to guidance for every step of the design and development journey, you’ll be a UX research expert in no time.
Companies that integrate user experience (UX) research into the product development lifecycle are nearly twice as likely to outperform their counterparts on critical metrics like customer retention, conversion, and revenue.
That’s the conclusion, based on a survey of UX professionals, in our recent report, ’State of the art of UX 2022’.
To put this into purely financial terms, UX research has a direct impact on the bottom line.
Of the respondents that said user research is consistently integrated in their product development process, customer satisfaction scores averaged more than 80%, against 50% for those that said it was not. Net Promoter Scores (NPS) were similarly skewed at 67% versus 32%, and profitability was rated at 66% for companies with UX research fully integrated, against 37% for companies that said UX research was inconsistently, or barely, integrated.
In the words of Alfonso de la Nuez, Co-Founder of UserZoom, “…for UX research to fulfill its business-critical function, it needs to be embedded throughout the product development lifecycle. To achieve this, there has to be board-level responsibility for customer facing-experiences…”
According to McKinsey, in The Business Value of Design, “The best design performers increase their revenues and shareholder returns at nearly twice the rate of their industry counterparts.”
UX research, or user experience research, is the scientific method that companies use to understand the thoughts, feelings and behaviors of the people who use their products and services.
The precise UX research method used depends upon where the product is in the development cycle. For example, at the discovery stage (before a product has even been defined), the most popular research method tends to be interviewing stakeholders using open-ended questions.
Typically, a group of potential customers will be recruited and invited to give their opinions about a market sector, and the products available in that sector. These tests can be carried out in person, in a series of interviews and focus groups, or remotely, using online interview tools.
These are known as qualitative methods. They give results that are verbal rather than numerical, illuminating the emotional responses of existing and potential customers.
Later in the development process, it's important to validate this qualitative data with feedback from a larger sample size of users. This can be done using quantitative research methods.
Quantitative research is any kind of investigation, experiment or study where the results can be presented with numerical values. The data you’ll uncover in quantitative research is all to do with ‘how many, how often and/or how much, etc.’
Qualitative refers to any kind of research where the results take the form of observations, comments, thoughts, and feelings. They tell the story of the participant’s experience.
This type of research can be further subdivided into behavioral and attitudinal methods. For instance, a behavioral method examines how people use products. An attitudinal method would then investigate what people think of these products.
Quantitative research is numerical. It tells us “What” the user likes best in a product or service whereas qualitative research tells us “Why.”
In the design phase of the product development process, quantitative methods such as A/B testing and click testing might be introduced to provide statistical support for a hypothesis developed during qualitative tests. In A/B tests users are asked to choose their preference between two different versions of a feature or design. In click testing, the tester is asked to click on a particular menu item or feature to see how easily they can find it.
These types of tests deliver data that can be subjected to statistical analysis. An A/B test would deliver a score for the two different versions of a feature or a page. A click test would record the success rate, and perhaps the time taken, for a participant to locate and click on a specific item in the menu.
Quantitative refers to any kind of research where the results can be presented in numbers, i.e. ‘how many, how often, or how much’. It is often used to add context (e.g. it took users less time to complete a task on prototype A than on prototype B) and can be thought of as the “What” data.
The analytics of a typical website – pageviews, sessions, bounce-rate, frequency of visits – are all examples of quantitative data.
Net Promoter Scores, ease of use scores, completion rates, brand perception scores and time on task scores are all examples of quantitative data in UX research.
In moderated testing, a moderator observes the participants, either in-person or remotely, as they carry out specified tasks.
The moderator can interact with users if they encounter problems, say, or clarify something the user has said. By being present they can understand the user’s behavior better and dig deeper into any issues. They can also ask for opinions and modify the script on the fly if anything unexpected happens.
Unmoderated tests are similar to moderated tests, except that they are unobserved.
In an unmoderated test, the participant interacts with the product or service while being prompted with pre-selected questions and/or tasks. Different participants can be tested at different times, and at different stages in the development cycle, including post-launch.
Once a test is set up there is no real limit to how many participants can be included, making the sample size larger and creating economies of scale.
Unmoderated tests require a specialized online platform or service that adds something to the cost, especially if the service provider helps recruit participants, build the study and analyze the data. But because a test can be used multiple times for multiple users, without the services of a moderator, the savings will frequently outweigh the costs.
In summary, a moderated test can be more flexible and insightful, especially for qualitative tests. An unmoderated test can be quicker, easier and cheaper, and it can have a much larger sample size. This is especially useful for quantitative tests where the data can be analyzed statistically, rather than by hand.
“For every dollar spent to resolve a problem during product design, $10 would be spent on the same problem during development, and multiply to $100 or more if the problem had to be solved after the product’s release.”
Robert Pressman’s ‘Software Engineering: A Practitioner’s Approach’
“Usability rules the web. Simply stated, if the customer can’t find a product, then he or she will not buy it.”
Jakob Nielsen’s ‘Designing Web Usability’
As with qualitative versus quantitative testing, the best approach to using moderated and unmoderated testing is a blend of the two. Unmoderated adds a huge amount of value to an otherwise moderated, predominantly qualitative, regime by:
For a full description of the most popular UX research methods, and when they are applied, see the next chapter, Which UX research methods?