And how to avoid them!
The whole point of UX research is to understand who your customers are, what is important to them, and how they actually use your products.
Gaining real observations from the real people using your product is one of the better methods of UX testing, rather than just indirectly analyzing data about them. HOWEVER… finding actual, real-life people to test your products can be more challenging than you might assume.
For a start, people just don’t like doing stuff. Especially stuff you’re not paying them to do. But how else could you do to get people to come and do some testing? Beg them? Trick them? Lay traps?
Even if you managed to gather together a bunch of angry, humiliated and potentially wounded people in one room, you’ll probably realise you’ve wasted a whole bunch of resources and time, when you should have been doing other important stuff – like writing the tests themselves.
And who’s to say these people are even the ‘right people’. Basically you’ve got yourself a group of testers whose sole connection to one another is that they’re all easily fooled by a rudimentary bear trap.
Which brings me neatly to the first couple of things we need to discuss…
When deciding which participants will potentially give you the ‘best’ data, you’ll probably want to find people who use (or will likely use) your product reasonably often.
The term ‘best’ here means relevant, useful and objective. These participants should also be able tell you honestly what they think about using your product. Sometimes this can be tough to hear, so prepare yourself. If you’re doing UX testing as an exercise in fishing for compliments, you’re doing UX testing wrong.
For an in-depth and entertaining guide to all the different elements that go into recruiting the right participants, download our comprehensive new ebook.
Kuldeep Kelkar, our SVP Global UX Research Services, answered this question in his article on estimating sample sizes for UX research.
If your objective is just to identify ‘low hanging fruits’, a sample size of five is probably enough. However, if you want to go beyond that – bearing in mind there’s tremendous business value in fixing and optimizing several workflows at once – you’ll want to go with a larger group of participants. The higher the group size the more issues you will find.
In many cases, especially where design has matured and lots of optimization is needed, it’s recommended to test with groups of 20, 30 or even more participants.
Depending on your intentions, here are our recommendations about sample size….
The bold and blue rows are the most common set of numbers we see across hundreds and hundreds of studies here at UserZoom.
So immediately, these are the major challenges in participant recruitment: finding the most relevant participants in a volume that makes their input statistically significant.
But before we tackle these issues head-on, let’s discuss other problems that UX researchers come across when recruiting participants…
Tying into the above point on finding the ‘right’ audience – based on your research project, you may want to focus on clarifying just who is your target demographic.
And finding your target demographic can be tricky.As we stated in our previous guide to recruiting the right participants, demographic information can be related to age, location, employment, education and household income.
You may also want to explore issues people are currently experiencing while they visit your website via a specific device (iPad, Android phone, etc).
You may also want to research the presence or absence of key behaviors, needs and triggers. As such, behavioral information could include things like purchase decisions (would they or wouldn’t they buy) or usage rate (how much time do they spend).
This is all stuff you have to investigate and define, before you even begin planning your research tasks.
But there is one easy way to get the ‘right’ people through the door (physically or virtually)…
Before running any kind of usability test, you’ll want to find people to participate. People who will carry out the tasks you sent them.
A screener (or screener question) is an opportunity for you to have bit more control over who carries out your test before they begin. This will also help you filter out anybody who wouldn’t necessarily be right for it.
Here’s an example from our own guide to writing screener questions (where you can find lots more information on the topic):
If you’re testing the navigation of a website that has a sole audience of civil engineers, and you want to make sure all the relevant categories are represented and sit in the right menu, there’s little point in recruiting a load of people who literally just this second had to Google the term ‘civil engineers’.
If you’re an existing UserZoom customer, we can help you with this task. You can define exactly who you want to test with, for any and all research methods by creating your own screener questions – ensuring you’re gaining insights from only the most valuable and relevant participants.
And if even that sounds like too much work, screeners can be saved and copied across multiple studies, or you can use pre-approved Screeners from UserZoom’s Screener library.
People are sacrificing a significant part of their day to help you, so they will expect to be paid for their time. Obviously the amount you pay will depend on your budget, but the expected rate can be anywhere between $25 – $60 per hour. The more niche your product or market, the more expensive it will be to recruit the users you’re looking for.
If (and that’s a strong emphasis on the word ‘if’) your participants are confident that their input will affect change, have a positive impact on something they care about, or is dedicated to a worthy cause, you may not need to use an incentive.
If however your study is lengthy or complex, you will likely need to incentivize it – and therefore this is something you’ll have to budget for.
You could go the route of using a recruitment service to do your recruiting for you, and as part of the service charge, they will be responsible for paying participants for their time. This still costs money, but removes many organizational headaches.
You’ll probably only have a limited pool of potential participants. Most of which will be collected via your own email address book, your existing customers or whoever is sat in your local coffee shop with nothing better to do.
And who hasn’t roped in family or friends at the last minute to make up the numbers? This isn’t necessarily a bad thing – they may fit your demographic – but will their insight be truly honest?
Your own personal biases can come into play the more involved you are in the recruitment process – you may dismiss a perfectly good candidate, for reasons that only your subconscious can determine. However that candidate’s observations and behaviors could still be perfectly valid.
So far, we’ve talked pretty generally about participant recruitment. Whether you’re doing online remote testing (where participants can be spread far and wide across the globe) or in-lab testing (where people are coming to your office and you can watch them test your product with you in the room) – the principles of relevancy and volume are still the same.
However, there is one major thing that you should keep in mind when deciding whether to go remote or in-person – if you’re going the in-person route, you’ll likely be solely responsible for all the recruiting.
There are certainly benefits to this, You control EXACTLY who comes through your door and you can build a relationship with the people you’re testing with – this potentially enables them to open up more, and make them more likely to return for ongoing testing.
However it can be time-consuming. As I said earlier, you could be concentrating on equally important things, and this can be especially detrimental if you are a UX team of one. It also requires you to have very strong project management and organization skills. You’re probably already spinning an awful lot of plates as it is.
You could avoid all of the above challenges of recruitment by using a participant sourcing company.
Let’s look at the pros and cons of letting someone else do the work for you…
As I hinted at earlier, there is a way you can tackle all these recruitment challenges head-on…
UserZoom customers can access automated recruiting for all of our unmoderated UX research methodologies – including Basic Qualitative Usability Testing, Advanced UX Research, Tree Tests, Card Sorts, Click Tests or Surveys (or a combination of all). This means you can automatically launch studies to the right participants and get rapid results, without having to contact anybody directly.
We are the industry’s first solution that offers self-service task building AND automated participant recruitment for all your UX research needs. This ultimately means you can build, recruit and launch an advanced UX test completely on your own. No fuss, no hassle, just UX insights delivered with scale and speed.