Recruiting the Right Participants for Your Remote Usability Studies
Recruiting the Right Participants: Which Method to Use When
Recently, remote user testing has become one of the more renowned ways to conduct user experience research. In comparison to traditional in-lab testing, it offers added opportunities to reach out to more users, collect more qualitative and quantitative data, and can be more cost-effective and time-efficient. Remote User Testing is best used for testing fully functional websites, software applications, mobile applications or devices, clickable prototypes, limited functionality wireframes, screenshots, mock-ups and even product concepts prior to development.
Additional benefits of remote research include its ability to reach people across a broader geographic range, target those niche audiences (sports car owners with manual transmissions, for example), and allow users to participate in their natural environments, like their home or office.
However, despite the vast benefits of remote research, there still remains the challenge of recruiting participants.
Finding the right participants and choosing the right recruitment method for your project is a critical component in achieving your research objectives. So, without further ado, let’s get started on how you can find these key participants and determine which method is right for you.
Find the Right Participants by Targeting Typical Users
First and foremost, what do we actually mean by the right participants? Every product, every solution and every service has a target audience. Your target should include individuals who truly are or represent your actual users. Individuals who fit into your target are going to be the right participants, because they are individuals who are the most likely to provide you with the type of valid feedback that can aid you in making meaningful improvements to a design.
So naturally, a significant step towards discovering the right participants is clarifying your target audience.
Based on your research project, you may want to first focus on clarifying specific elements of data, such a target’s demographic. Demographic related information might include physical characteristics (age, gender, ethnicity, etc.), employment, education and household income information. Likewise, you may also want to explore elements of your target’s channel usage. Let’s say your research is focused on gathering information about what users are currently experiencing on your website while using a tablet device. Of course, you want to have a pretty good idea of which tablet device they are using, be it an iPad, Kindle, Android-based, due to the fact that their device type may be a variable in their experience. And lastly, you may want to research the presence or absence of key behaviors, needs and triggers. As such, behavioral information could include things like purchase decisions (would they or wouldn’t they buy) or usage rate (how much time do they spend).
The point is, when you identify your target audience, you can develop more accurate hypotheses about their reactions to new designs, thus using that research to investigate how your product design could support their needs.
The Importance of a Good Screener
Once you have clarified your audience, how do you ensure that they are the ones taking part in your study? The most common method is to use screening questions.
A screener is essentially a script that determines if a potential participant matches the characteristics defined in your research criteria and serves to eliminate any outlying candidates. Having a good screener can reduce costs, ease data analysis and reduce the level of response bias.
Screening questions are typically placed at the beginning of a study and can include questions that look like this:
In a week, how many times do you conduct searches using the web?
- a) Less than once a week
- b) 2-9 times a week
- c) 10-20 times a week
- d) More than 20 times a week
By this point, you should already have your target clarified and should have an answer sheet (or cheat sheet) outlined, which has information on which participants should proceed with the study and which should be eliminated from the study.
One thing to consider is to not make it obvious as to what you are looking for. For instance, if you are looking for people who plan to buy a washer within the next year, you may not want to phrase the question as:
Are you considering purchasing a washer within a year?
But instead, keep the purpose of your study ambiguous, making it more difficult for potential candidates to guess which answer will prevent elimination. Here’s an example:
Which of the following items do you plan on purchasing within the next year?
- a) Television
- b) Refrigerator
- c) Washer
- d) Dryer
- e) Computer
- f) Car
Here are a few pointers:
- Don’t use the screener to gather information
- Ask the elimination questions first
- Eliminate conflicts of interest
- Recruit based on behavior and attitudes
- Screen for computer and web experience
- Eliminate the usual suspects
Methods for Recruiting Participants
Now that we’ve talked about how you can clarify your target and ensure that those are the ones taking your study, let’s discuss what methods you can use to invite them. There are several methods for recruiting participants and they really break down into two categories:
1. Do it yourself, otherwise known as self-recruiting
2. Have someone else do it for you, like an external recruiter or research panel
Now, depending on your research needs, you first want to examine which of the different methods are more time-efficient and more cost-effective for you. Let’s take a closer look at each method and their specific pros and cons.
1. The Self-Recruiting Option
Self-recruiting is a great way to find participants. It can save you time and money, as well as give you ultimate control of the project.
Practical Ways to Self-Recruit
If you have a good amount of traffic on your website or mobile site, you can recruit participants and explore their experiences, understanding, and level of satisfaction while catching them in the act of things. Using an intercept allows you to create and launch branded studies, getting you data quickly.
Sometimes test participants are right under your nose. You can test participants within your organization, but be careful of this method. Before you consider internal testing, make sure that participants have had no involvement in the design or development of the site or product you are testing and most importantly that they represent a target audience.
Use an Email List
Email lists allow researchers to recruit potential participants from your company’s opt-in contact list. You can also rent or purchase email lists from a marketing research company, however, it is advised to proceed with caution because the degree or quality can be a tad questionable. Also remember, renting or purchasing a list could factor into your costs for the overall study. Here’s a quick tip: Be sure to personalize your emails. Though this may be an arduous task, doing so can increase your response rate by at least 5% or sometimes higher.
Via Social Media
You can collect data through popular social media sites such as Facebook, Twitter, LinkedIn or even Craigslist. With Facebook you can recruit through paid advertisements (ads), fan pages, personal messages, and post in groups. You can use sites like LinkedIn, specifically LinkedIn groups, where you can advertise to potential study participants who are in your target.
Another way to recruit is through classifieds sites, like Craigslist. Craigslist allows you to post on a weekly basis, if your project timeline allows for it. You can place your links on 3rd party newsletters or blogs and you can even use search engine marketing, through Google or other providers.
Utilize Online Crowdsourcing
You can also recruit participants through online crowdsourcing. Crowdsourcing allows you to obtain ideas or services from an online community that comprised of mostly volunteers and part-time workers. One of the more popular crowdsourcing techniques is to utilize Internet marketplaces such as Amazon’s Mechanical Turk. As mentioned in a Measuring Usability blog post Mechanical Turk is fueled by participants, who are willing to take online studies, at relatively low rates. It is best used to engage a diversity of participants, who are found all over the world, in a short period of time. However, with this method you must proceed with caution—the quality of participants can be debatable.
In a nutshell, here’s a breakdown of the major pros and cons of self-recruiting:
Pros of Self-Recruiting
- Cost: Can be less expensive than using an external resource
- Control: Allows researchers to control the process and or quality of participants
Cons of Self-Recruiting
- Time: Can be time-consuming
- Management: Requires strong project management/organization skills from the researcher
2. The External Recruiting Option
If you do not have the time or resources to conduct self-recruiting, but perhaps have a bit of a wider budget, then going with an external recruiter may be the most efficient option for your project.
Research panels generally include a database of people who have signed up and have agreed to be contacted in the future if they match a specific criterion; these individuals are asked to complete a profile with questions that includes various details about themselves, as discussed earlier.
Here’s a breakdown of the major pros and cons of external recruiting:
Pros of External Recruiting
- Time: Frees up time for the researcher, so they can focus exclusively on the design of the study
- Reach: Can potentially leverage multiple sources to access your target audience
- Administration: May help with a screener development
Handle incentives and quota fulfillment
Cons of External Recruiting
- Cost: Can incur costs and can potentially be more expensive than self-recruiting (see cost section)
- Coordination: Requires coordination with the external recruitment agency
- Planning: Requires you to have your study design well-planned, so the recruitment agency can provide an accurate estimate of incident rate to calculate pricing.
- Quality: Quality of panelists data differ across panels and companies
More on the Cost of Recruitment (Including Incentives)
Something important to note when it comes to expense and using an external recruiter is that if you have a particular niche user, such as left-handed dentists who performed a root-canal in the last 3 months, it can be very expensive for the research panel to find that particular niche audience. Why? Because generally those higher costs are determined based on how likely the agency is to find your audience out of the population of participants they have access to or who are currently in their database.
Recruiters usually charge a fee for each participant who is successfully recruited. This typically ranges anywhere from $10.00-$25.00 per participant.
Successful participants are defined as those who meet the criteria, actually take part in the study and are able to complete the study. A good recruiter screens accurately and reminds participants about the study to assure all of their recruits are successful.
Regardless of the method you choose, it is recommended to compensate your participants. Remember that participants are more likely to take a study seriously if they believe their input will affect change and have a positive impact on something they care about. If your study is dedicated to a higher cause or does not take a lot of time to complete, you probably do not need to use an incentive.
If however your study is lengthy or complex, you will likely need to incentivize it. Incentives could include compensating with cash, digital currency, points or even sweepstakes. Incentives can be determined by study length, interest, complexity and the topical expertise of respondents, as well. There will most likely be a fee for additional services, such as creating a screener, so it’s best to discuss any additional services needed during your initial discussions with the recruiter.
Key Takeaways for a Successful Recruitment Process
As you may have picked up, a well-managed recruitment process can be pivotal to the success of achieving your research objectives. Here are a few key takeaways:
- Clarify your target audience
- Ensure you have a good screener
- Do your research on the different recruiting methods
- Weigh the pros and cons of each method and choose the one that best fits your time and budget constraints
- Incentivize participants relative to your study’s length and complexity