Seven top challenges when recruiting participants for UX research

The whole point of UX research is to understand who your customers are, what is important to them, and how they actually use your products.

Gaining real observations from the real people using your product is one of the better methods of UX testing, rather than just indirectly analyzing data about them. However, sometimes finding actual, real-life people to test your products can be more challenging than you might assume. Which brings us to the first of seven common challenges we face when recruiting user testing participants.

Finding the ‘right’ participants

When deciding which participants will potentially give you the "best" data, you’ll probably want to find people who use (or will likely use) your product reasonably often.

The term "best" here means relevant, useful and objective. These participants should also be able tell you honestly what they think about using your product. Sometimes this can be tough to hear, so prepare yourself. If you’re doing UX testing as an exercise in fishing for compliments, you’re doing UX testing wrong.

Deciding how many participants to recruit

If your objective is just to identify "low hanging fruit", a sample size of five is probably enough. However, if you want to go beyond that—bearing in mind there’s tremendous business value in fixing and optimizing several workflows at once—you’ll want to go with a larger group of participants. The higher the group size, the more issues you'll find.

In many cases, especially where design has matured and lots of optimization is needed, it’s recommended to test with groups of 20, 30 or even more participants.

Depending on your intentions, here are our recommendations about sample size.

The bold and blue rows are the most common set of numbers we have seen across hundreds and hundreds of studies.

So immediately, these are the major challenges in participant recruitment: finding the most relevant participants in a volume that makes their input statistically significant.

But before we tackle these issues head-on, let’s discuss other problems that UX researchers come across when recruiting participants.

Clarifying your target audience

Tying into the above point on finding the "right" audience—based on your research project, you may want to focus on clarifying just who is your target demographic.

And finding your target demographic can be tricky. As we've said before, demographic information can be related to age, location, employment, education and household income.

You may also want to explore issues people are currently experiencing while they visit your website via a specific device (iPad, Android phone, etc).

You may also want to research the presence or absence of key behaviors, needs and triggers. As such, behavioral information could include things like purchase decisions (would they or wouldn’t they buy) or usage rate (how much time do they spend).

This is all stuff you have to investigate and define, before you even begin planning your research tasks.

But there is one easy way to get the right people through the door (physically or virtually).

Writing a good screener question

Before running any kind of usability test, you’ll want to find people to participate. People who will carry out the tasks you sent them.

A screener (or screener question) is an opportunity for you to have bit more control over who carries out your test before they begin. This will also help you filter out anybody who wouldn’t necessarily be right for it.  

Here’s an example:

If you’re testing the navigation of a website that has a sole audience of civil engineers, and you want to make sure all the relevant categories are represented and sit in the right menu, there’s little point in recruiting a load of people who literally just this second had to Google the term "civil engineers".

You can define exactly who you want to test with, for any and all research methods by creating your own screener questions—ensuring you’re gaining insights from only the most valuable and relevant participants.

And if even that sounds like too much work, screeners can be saved and copied across multiple studies, or you can use pre-approved Screeners from our screener library.

Incentives

People are sacrificing a significant part of their day to help you, so they will expect to be paid for their time. Obviously the amount you pay will depend on your budget, but the expected rate can be anywhere between $25 – $60 per hour. The more niche your product or market, the more expensive it will be to recruit the users you’re looking for.

If your study is lengthy or complex, you will likely need to incentivize it and this is something you’ll have to budget for.

You could go the route of using a recruitment service to do your recruiting for you, and as part of the service charge, they will be responsible for paying participants for their time. This still costs money, but removes many organizational headaches.

Limited reach

You’ll probably only have a limited pool of potential participants. Most of which will be collected via your own email address book, your existing customers or whoever is sat in your local coffee shop with nothing better to do.

And who hasn’t roped in family or friends at the last minute to make up the numbers? This isn’t necessarily a bad thing but will their insight be truly honest?

Bias

Your own personal biases can come into play the more involved you are in the recruitment process—you may dismiss a perfectly good candidate, for reasons that only your subconscious can determine. However that candidate’s observations and behaviors could still be perfectly valid.

Using a recruitment service

So far, we’ve talked pretty generally about participant recruitment. Whether you’re doing online remote testing (where participants can be spread far and wide across the globe) or in-lab testing (where people are coming to your office and you can watch them test your product with you in the room)—the principles of relevancy and volume are still the same.

However, there's one major thing that you should keep in mind when deciding whether to go remote or in-person—if you’re going the in-person route, you’ll likely be solely responsible for all the recruiting.

There are certainly benefits to this, You control exactly who comes through your door and you can build a relationship with the people you’re testing with—this potentially enables them to open up more, and make them more likely to return for ongoing testing.

However it can be time-consuming. As I said earlier, you could be concentrating on equally important things, and this can be especially detrimental if you are a UX team of one. It also requires you to have very strong project management and organization skills. You’re probably spinning a lot of plates already.

You could avoid all of the above challenges of recruitment by using a participant sourcing company.

Let’s look at the pros and cons of letting someone else do the work for you.

Pros of UX research services

• They will save you time.
• Will have a much wider-net to search for relevant participants. Most platforms should have a global reach, and therefore be able to source much larger segments of people.
• Thanks to the sheer number of participants at their disposal, you’re more likely to get the "right" participants, especially if your needs are a bit more niche.
• They should have expertise – for instance finding people that will provide value even when not exactly matching your profile.
• Simplicity—they will handle incentives for you.

Cons of UX research services

• It can potentially be more expensive than self-recruiting.
• This requires coordination with the external recruitment service.
• This requires you to have your study design planned in advance, so the recruitment service can provide accurate pricing and timing estimates.

Let us do the work for you

Our customers can access automated recruiting for all of our unmoderated UX research methodologies – including Basic Qualitative Usability Testing, Advanced UX Research, Tree Tests, Card Sorts, Click Tests or Surveys (or a combination of all). This means you can automatically launch studies to the right participants and get rapid results, without having to contact anybody directly.

We are the industry’s first solution that offers self-service task building and automated participant recruitment for all your UX research needs. This ultimately means you can build, recruit and launch an advanced UX test completely on your own. No fuss, no hassle, just UX insights delivered with scale and speed.