Seven top challenges when recruiting participants for UX research (and how to avoid them)
The whole point of UX research is to understand who your customers are, what is important to them, and how they actually use your products.
Gaining real observations from the real people using your product is one of the better methods of UX testing, rather than just indirectly analyzing data about them. HOWEVER… finding actual, real-life people to test your products can be more challenging than you might assume.
For a start, people just don’t like doing stuff. Especially stuff you’re not paying them to do. But how else could you do to get people to come and do some testing? Beg them? Trick them? Lay traps?
Even if you managed to gather together a bunch of angry, humiliated and potentially wounded people in one room, you’ll probably realise you’ve wasted a whole bunch of resources and time, when you should have been doing other important stuff – like writing the tests themselves.
And who’s to say these people are even the ‘right people’. Basically you’ve got yourself a group of testers whose sole connection to one another is that they’re all easily fooled by a rudimentary bear trap.
Which brings me neatly to the first couple of things we need to discuss…
Finding the ‘right’ participants
When deciding which participants will potentially give you the ‘best’ data, you’ll probably want to find people who use (or will likely use) your product reasonably often.
The term ‘best’ here means relevant, useful and objective. These participants should also be able tell you honestly what they think about using your product. Sometimes this can be tough to hear, so prepare yourself. If you’re doing UX testing as an exercise in fishing for compliments, you’re doing UX testing wrong.
For an in-depth and entertaining guide to all the different elements that go into recruiting the right participants, download our comprehensive new ebook:
Deciding how many participants to recruit
Kuldeep Kelkar, our SVP Global UX Research Services, answered this question in his article on estimating sample sizes for UX research.
If your objective is just to identify ‘low hanging fruits’, a sample size of five is probably enough. However, if you want to go beyond that – bearing in mind there’s tremendous business value in fixing and optimizing several workflows at once – you’ll want to go with a larger group of participants. The higher the group size the more issues you will find.
In many cases, especially where design has matured and lots of optimization is needed, it’s recommended to test with groups of 20, 30 or even more participants.
Depending on your intentions, here are our recommendations about sample size….
The bold and blue rows are the most common set of numbers we see across hundreds and hundreds of studies here at UserZoom.
So immediately, these are the major challenges in participant recruitment: finding the most relevant participants in a volume that makes their input statistically significant.
But before we tackle these issues head-on, let’s discuss other problems that UX researchers come across when recruiting participants…
Clarifying your target audience
Tying into the above point on finding the ‘right’ audience – based on your research project, you may want to focus on clarifying just who is your target demographic.
And finding your target demographic can be tricky.As we stated in our previous guide to recruiting the right participants, demographic information can be related to age, location, employment, education and household income.
You may also want to explore issues people are currently experiencing while they visit your website via a specific device (iPad, Android phone, etc).
You may also want to research the presence or absence of key behaviors, needs and triggers. As such, behavioral information could include things like purchase decisions (would they or wouldn’t they buy) or usage rate (how much time do they spend).
This is all stuff you have to investigate and define, before you even begin planning your research tasks.
But there is one easy way to get the ‘right’ people through the door (physically or virtually)…
Writing a good screener question
Before running any kind of usability test, you’ll want to find people to participate. People who will carry out the tasks you sent them.
A screener (or screener question) is an opportunity for you to have bit more control over who carries out your test before they begin. This will also help you filter out anybody who wouldn’t necessarily be right for it.
Here’s an example from our own guide to writing screener questions (where you can find lots more information on the topic):
If you’re testing the navigation of a website that has a sole audience of civil engineers, and you want to make sure all the relevant categories are represented and sit in the right menu, there’s little point in recruiting a load of people who literally just this second had to Google the term ‘civil engineers’.
If you’re an existing UserZoom customer, we can help you with this task. You can define exactly who you want to test with, for any and all research methods by creating your own screener questions – ensuring you’re gaining insights from only the most valuable and relevant participants.
And if even that sounds like too much work, screeners can be saved and copied across multiple studies, or you can use pre-approved Screeners from UserZoom’s Screener library.
People are sacrificing a significant part of their day to help you, so they will expect to be paid for their time. Obviously the amount you pay will depend on your budget, but the expected rate can be anywhere between $25 – $60 per hour. The more niche your product or market, the more expensive it will be to recruit the users you’re looking for.
If (and that’s a strong emphasis on the word ‘if’) your participants are confident that their input will affect change, have a positive impact on something they care about, or is dedicated to a worthy cause, you may not need to use an incentive.
If however your study is lengthy or complex, you will likely need to incentivize it – and therefore this is something you’ll have to budget for.
You could go the route of using a recruitment service to do your recruiting for you, and as part of the service charge, they will be responsible for paying participants for their time. This still costs money, but removes many organizational headaches.
For more info, check out our guide to how much should I pay user research participants.
You’ll probably only have a limited pool of potential participants. Most of which will be collected via your own email address book, your existing customers or whoever is sat in your local coffee shop with nothing better to do.
And who hasn’t roped in family or friends at the last minute to make up the numbers? This isn’t necessarily a bad thing – they may fit your demographic – but will their insight be truly honest?
Your own personal biases can come into play the more involved you are in the recruitment process – you may dismiss a perfectly good candidate, for reasons that only your subconscious can determine. However that candidate’s observations and behaviors could still be perfectly valid.
Using a recruitment service
So far, we’ve talked pretty generally about participant recruitment. Whether you’re doing online remote testing (where participants can be spread far and wide across the globe) or in-lab testing (where people are coming to your office and you can watch them test your product with you in the room) – the principles of relevancy and volume are still the same.
However, there is one major thing that you should keep in mind when deciding whether to go remote or in-person – if you’re going the in-person route, you’ll likely be solely responsible for all the recruiting.
There are certainly benefits to this, You control EXACTLY who comes through your door and you can build a relationship with the people you’re testing with – this potentially enables them to open up more, and make them more likely to return for ongoing testing.
However it can be time-consuming. As I said earlier, you could be concentrating on equally important things, and this can be especially detrimental if you are a UX team of one. It also requires you to have very strong project management and organization skills. You’re probably already spinning an awful lot of plates as it is.
You could avoid all of the above challenges of recruitment by using a participant sourcing company.
Let’s look at the pros and cons of letting someone else do the work for you…
Pros of UX research services
- They will save you time
- Will have a much wider-net to search for relevant participants. Most platforms should have a global reach, and therefore be able to source much larger segments of people
- Thanks to the sheer number of participants at their disposal, you’re more likely to get the ‘right’ participants, especially if your needs are a bit more ‘niche’
- They should have expertise – for instance finding people that will provide value even when not exactly matching your profile
- Simplicity – they will handle incentives for you
Cons of UX research services
- Can potentially be more expensive than self-recruiting
- Requires coordination with the external recruitment service (unless you find one with automated participant sourcing *cough cough*)
- Requires you to have your study design planned in advance, so the recruitment service can provide accurate pricing and timing estimates
As I hinted at earlier, there is a way you can tackle all these recruitment challenges head-on…
Let us do the work for you
UserZoom customers can access automated recruiting for all of our unmoderated UX research methodologies – including Basic Qualitative Usability Testing, Advanced UX Research, Tree Tests, Card Sorts, Click Tests or Surveys (or a combination of all). This means you can automatically launch studies to the right participants and get rapid results, without having to contact anybody directly.
We are the industry’s first platform that offers self-service task building AND automated participant recruitment for all your UX research needs. This ultimately means you can build, recruit and launch an advanced UX test completely on your own. No fuss, no hassle, just UX insights delivered with scale and speed. If you’re interested, please get in touch!
Want to learn even more about getting the right participants for your study?
Here you’ll learn:
- How to get started with screeners
- How to navigate to Advanced Screeners
- How to set up groups and quotas in a project
- How to add screener questions to your library for repeat use
Christopher is the Content Marketing Manager, which basically means the skipper of the good ship ‘UserZoom blog’. So far his requests for changing its name to the ‘USS-erzoom Blog’ have been rightfully denied. In his spare time, Christopher is a filmmaker and the editor of wayward pop culture site Methods Unsound. He used to be the deputy editor of Econsultancy, editor of Search Engine Watch, staff writer for ClickZ and features editor of CMO.com.