This user research guide is designed to help you take your user research to a more inclusive place
There is a profound diversity and inclusivity problem across society at large—and that problem extends to user research.
Too many groups are underrepresented in user research studies. We need to cast wider nets to better understand and serve all users.
First, some clarity on what diversity and inclusion mean. Yes—we should already know this— but a quick refresher never harmed anyone.
We’re all striving to ensure our usability tests, explorations, or desk research include users from a range of backgrounds, household incomes, life stages, ethnicities, disabilities, sexes, and genders. But can we work even harder to make this happen?
There should be as many users from each group as it takes until themes are thoroughly fleshed out, and no new themes emerge (also called ‘data saturation’).
Not every budget can stretch this far, so it’s important to make any sample of participants as rich as possible to uncover the broadest set of circumstances, points of view, and insights.
Diversity and inclusivity are essential for good user research
To state the obvious, the world is made up of many different nationalities and ethnicities. Here are a few things to get you thinking:
Minority community members deserve to be represented just as much as anyone else. And they have a lot of influence and spending power. It would be immoral to ignore their voices—and it would be a lost opportunity commercially, too.
Yet the industry standard of participant screening includes those from around only six nationalities with little to no mention of differing abilities.
But let’s stop for a moment. I’m going to guess I’m preaching to the choir here. You’re probably reading this because you’re one of the enlightened ones, working to bring greater diversity and inclusivity to user research. You might already know about the dangers of groupthink. And how diversity and inclusivity are making organizations around the world more successful.
So let’s dig a little deeper.
Diversity and inclusion open the door to a mixture of user scenarios, problems, goals, and ideas. It is all too easy to stay inside our digital echo chambers and social circles, leaving us ignorant of the full range of use cases for our apps and services.
We think we know what people want and how they behave. And while intuition, hunches, and personal experience can be valuable, there is simply no substitute for evidence-based decision-making through testing.
Take vehicle seatbelts as an example. Seatbelts were not originally tested with women in mind. But the real scandal is that seatbelts are still not tested with crash dummies that match the average female anatomy. There is a staggering 73% greater chance that women will be injured in a car accident compared to men. The problem isn’t just seatbelts. Most such as stab vests, high-vis vests, and body armor are all made for men and men only.
When it comes to smart voice assistants and accents, anyone who doesn’t use American English without vernacular, regional diction, or accents will know first-hand the struggle to be understood by technology. It’s incredibly frustrating, not least because it could be fixed with more inclusive user research.
The business case
From a business perspective, a product that caters to a wider range of users is more successful in customer satisfaction, retention, loyalty, and revenue. Consumers now have more options to choose from than they’ve ever had before. If your company doesn’t fit what people need, they can easily find another company that does. Conducting research upfront is also often cheaper than fixing what has already been built. Another win for the business.
When conducting your user research tests, remember that even accepted methodologies may come up against cultural differences. For example, many cultures (such as Japan) are not accustomed to the directness of thinking-out-loud testing.
None of us know precisely what cultural or circumstantial differences might be discovered through user research when we maximize diversity and inclusivity. That’s why user research is so important.
While conducting qualitative research with five users is the industry standard, this will only highlight general patterns—it won’t uncover the full gamut of issues. Even Jakob Nielsen, who originally recommended five participants, says five is the bare minimum.
Even basic thinking-out-loud usability studies will need 8-12 participants. Research samples need to go beyond the standard ‘five’ to be able to capture themes across each background difference.
It’s impossible to interview every single type of user, from every background or with every relevant impairment. But you might not need to, depending on what your app or service does.
Start by exploring users and their circumstances and how they match your user base. Then you can arrange your research based on metrics like engagement, number of transactions, or value of transactions.
However, there may be a reason why some users from certain backgrounds may not be using your product in the first place. They might not be able to use your product because their needs haven’t been discovered in diverse research. But this doesn’t mean they don’t need your product or service.
Certain types of users may be easier to recruit than others. So, you may need to investigate how to support and actively recruit those who are less inclined or less able to participate. For example, many people will be employed in roles where they don’t have time to chat with a stranger for an hour, or they may need hours that are outside the ‘traditional’ 9 to 5.
Past research and theory can help shine a light on which areas of user divergence are valuable to understand. For example, race is unlikely to impact an ‘add to cart' experience—while using a screen reader for visual impairment certainly would.
It is both unnecessary and potentially irresponsible to make minority status relevant in situations when it's not relevant or appropriate. Always think about what you’re testing for.
By now, you know whom you’re inviting into research and why. Now we need to understand how to support these participants through the process.
Screeners or research invitations should ask for any special assistance the participant needs during the session. What you need to know is:
Incentives like vouchers should be for websites where users want to shop and can shop – make sure it’s relevant to where the participants live.
Unfortunately, it can be difficult to know what to ask or how to support users when you’re not a part of their world, and you don’t have a diverse team to help reach out to these communities or cultures. It may be worth reaching out to external sources who do.
It’s impossible to get the diversity of users you need using just the common sample size of five participants. While every project should aim for a diverse range of users, it is worth having separate rounds of research focusing on important variables like race or household incomes.
Prototypes should also be diverse and match your audience. People find it hard to relate to something if it looks like it’s not for them. For example, designs filled with race-specific generic names like ‘John Smith’ or ‘Jane Doe’ with photos of generic white people won’t help users of other non-white races feel that the product is aimed at them. There are which helps to keep the design generic enough but relatable for a wider user audience.
There are also prototyping tool plugins that auto-generate avatars and photos. These plugins cycle through different types of users to make tweaking prototypes per audience easier and quicker, especially if you need to do so between sessions.
However, that’s not to say that nothing can be learned from prototyping. Many issues uncovered in testing are often a lack of best practices. Doing a simple audit can go a long way to being inclusive and diverse. Plus, the more research you do, the easier it’ll be to spot issues or design with these in mind from the beginning.
The beauty of user research is that anyone from any social background, race, or ability can walk through your building entrance or dial into your Zoom call. It’s a beautiful moment when you meet that elusive user you have been fighting to meet.
So, don’t be afraid to throw your discussion guide out the window. Ask your planned question, but feel free to digress and ask everything you need to understand their circumstances. Seize the opportunity to use these often powerful user quotes and insights to campaign for further diverse and inclusive research.
You don’t know what you don’t know
A word of warning: brace yourself for blunt feedback from diverse users, especially in the beginning. You don’t know what you don’t know. There’s a high chance that aspects have been missed.
Unvarnished feedback, however difficult it may be to receive, is often what unlocks just what you need to build empathy between stakeholders and users.
While we may not consciously or deliberately exclude minorities from our research, we need to actively promote diverse participant groups in our research samples.
In research and design, diversity and inclusion significantly impact both our users and our business. It means that as a business you’re building the right solution for your users. The earlier you discover these insights, the cheaper and easier it is to include the necessary changes in your solutions.
The more diverse your user research, the richer the experience. Everyone will gain from your product.