The Importance of Screeners for User Research Recruitment
The first defense against bad data is a good screener.
We mentioned last time that finding the right participants for your research studies is important in order to get the actionable insights you’re after. What do we actually mean by the “right” participants? Every product, every solution and every service has a target audience. This is why clarifying who your target audience is in order to know who to recruit is critical.
The question then is after you’ve invested all the time and effort in finding the right audience for your user research, how do you ensure that they’re the ones taking part in your study? The most common method is to use screening questions.
A screener is essentially a script that determines if a potential participant matches the characteristics of your target audience that you defined in your research criteria. They also serve as a way to eliminate any outlying candidates who might make their way into your study but who don’t necessarily match up with who you’re looking to get feedback from. In this way, having a good screener can reduce costs, ease data analysis and reduce the level of response bias.
Screening questions are typically placed at the beginning of a study and can include questions that look like this:
In a week, how many times do you conduct searches using the web?
- Less than once a week
- 2-9 times a week
- 10-20 times a week
- More than 20 times a week
By this point, you should already have your target clarified and should have an answer sheet (or cheat sheet) outlined, which has information on which participants should proceed with the study and which should be eliminated from the study.
Don’t Lead the Wrong People into Choosing the Right Answers
One thing to consider is to not make it obvious as to what you are looking for. For instance, if you are looking for people who plan to buy a washer within the next year, you may not want to phrase the question as:
Are you considering purchasing a washer within a year?
This is obviously a leading question, and a savvy participant (or even a well meaning participant) can take the bait. So the best thing to do is to keep the purpose of your study ambiguous, making it more difficult for potential candidates to guess which answer will prevent elimination.
Here’s an example:
Which of the following items do you plan on purchasing within the next year?
Don’t Make the Screener Do Too Much
Think of screeners as a security guard to a building. They let the right people in and keep the wrong people out. This usually means they aren’t also the chef, front desk or valet. If you try and have your screeners do too much you might find out they haven’t done a good job.
Here are a few pointers on how to successfully use your screeners:
- Ask the elimination questions first: This way you’re being respectful of their time and not asking them to do too much before screening them out
- Don’t use the screener to gather information: This is inline with the above point. You can always ask follow up questions once you know they’re your target participant
- Eliminate conflicts of interest: Someone who hates washing machines might not be the best person to talk to if that’s what your product is
- Screen for experience: Depending on the kind of feedback you’re after, you might want beginners or you might want experts. Be sure to screen for experience if it’s important
- Eliminate the usual suspects: Be sure to screen out the folks whose feedback won’t be pushing forward your research goals
Between your research into who exactly your product/site/service is targeting and finding the right way to invite them into your study, recruiting the right participants can be a lot of work. This is why once you’ve invested the time and energy into inviting participants into your study your screener is so important. It’s there to ensure all your hard work doesn’t go to waste by acting as a quality check on the feedback you’re receiving.
Phil got his degree in creative writing, where they told him he most likely wouldn’t be able to use his degree for his career. He obviously won that round. When not working with UX researchers he can be found teaching martial arts and working on his fiction novels.