Which participant recruitment method should I use for user research?

1) In-house recruiting

In-house recruiting is when you control most, if not all, of the recruiting process internally at your organization.

This is for you hardy DIYers reading this who see something on Etsy and say, “These unicorn mason jars are adorable and perfect for my child’s upcoming birthday party” followed immediately by, “I can make that myself.” You have my respect because I would totally overpay for those unicorn mason jars simply to avoid any and all contact with glitter, or as I like to call it, the Devil’s Dust.

There are several ways that you can conduct your recruiting in-house, and we’ll cover the three most common that we see here at UserZoom:

A.) Emailing your database

B.) Intercepts

C.) Social Media

A.) Emailing your database

The obvious requirement here is that your organization has a database of users that you can contact (and if you don’t, remember that it’s never too late to start thinking about creating one).

Email lists allow researchers to recruit potential participants from your company’s opt-in contact list. This is great because you know they are targeted users who, depending on their level of engagement, are most likely invested in your product and would be willing to help optimize something they are already using.

However, it is advised to proceed with caution because sometimes response quality can be a tad questionable when you’re blasting out emails to relative strangers (remember – screeners are your friends).

Here’s a quick tip: be sure to personalize your emails. Think about it – how do you feel when you see an email that’s an obvious copy and paste job? Though this may be seen as an arduous task, doing so can increase your response rate by at least 5% or sometimes higher.

Pros:

• Your own users are the very definition of ‘target audience’
• Writing an email and personalizing it is still faster and easier than finding participants from scratch can help build rapport with users by showing them their opinion matters
• Depending on the size of your database, this can be done at scale

Cons:

• Have to have a database of customers that have opted into being contacted
• Depending on the size of your database, this can be extremely hard to do at scale
• Run the risk over over-engaging with your users, which can lead to lower levels of involvement as well as lower quality responses
• Without security measures, in place (a.k.a. screeners) you don’t really know who you’re getting feedback from.

B.) Site intercepts

Sometimes known on the street by their more nefarious name – ‘pop-ups‘ – site intercepts are study invites that are coded into your site in order to automatically ask live users if they are willing to join. If you have a good amount of traffic on your website or mobile site, you can recruit participants and explore their experiences, understanding, and level of satisfaction while catching them in the act of things.

The main trick is to try to place the intercept in the place that makes the most sense to recruit. If you want to improve your search or search results, for example, have the invite on the search results page. “Tell us what you think about this page and get a gift card” or “Got 5 minutes? Answer 3 quick questions and get a 25% off promo code.” Also consider a banner in the place you want to recruit and don’t be afraid to appeal to their vanity. “We’re re-doing this page and need expert opinions – click here to be part of the redesign.”

Here’s another tip: always try to make your intercepts branded. Visitors are more likely to say yes when it looks like it’s supposed to be a part of your site, instead of something that makes them want to run their anti-virus.

Pros:

• Passive recruiting in the sense that you can set the incidence rate (10% of all people who land on the page, e.g.) and let it run in the background
• High likelihood that it will be someone you want to hear from – either a user or a potential user
• Depending on your traffic this can be a fast way to get insights
• Can use intercepts as a way to recruit people into your opt-in email list for future studies

Cons:

• Without the use of screeners it’s hard to know who is coming into your studies
• If you don’t have a high volume of site traffic it can be unreliable and hard to scale
• When overused it can potentially cause people to avoid your site or have a negative perception of your site as one loaded with “pop-ups,” particularly if your intercepts aren’t branded

C.) Social media

#researcherlife #usability #tidepodchallenge #crossfit. You know, social media! Look, when your grandparents are sharing political memes on Facebook it’s safe to say that social media has a broad reach (also, yikes meema and papa) and when you want to recruit participants it pays to be where participants are.

You can recruit participants through popular social media sites such as Facebook, Twitter, Instagram, and LinkedIn, just to name a few. With Facebook and LinkedIn you can recruit through paid advertisements (ads), fan pages, personal messages, and post in groups. You can also specifically use LinkedIn groups to advertise to potential study participants who are in your target audience. Need dentists? Search LinkedIn groups for dentists and you may just find who you’re looking for.

Pro tip: It helps to know when people are most active on certain kinds of social media. If you post too early or too late it will most likely get buried and go unseen. Facebook, Twitter and Instagram tend to be used more before work, after work, and around lunch hours whereas sites like LinkedIn are mostly checked during work hours. Also, know your crowd and search for relevant hashtags to help you find them.

Pros:

• Can be relatively fast and simple to create
• Can reach a wide swath of the population both domestically and internationally
• There are myriad social media platforms to choose from, ranging from the broad to the niche

Cons:

• Requires some working knowledge of each social media platform, as well as the types of users and when they are engaging with it
• Hard to control who sees it and therefore might require time and effort ensuring the quality of your data
• Hard to vet people are who they say they are
• It’s possible for your competition to see and participate in your studies

2) External recruiting

We mentioned previously that recruiting can be hard. If someone said, “Prove it” I’d say “Okay, geez” and then do a Google search for recruitment help and point to the entire industry that exists to specifically help with it. External recruiting is when you reach out to one of these third party vendors to recruit participants for you.

To use our unicorn mason jar metaphor from before – it doesn’t mean that you can’t make them, it just means you might not be willing or able to make the unicorn mason jars and the unicorn cake all at the same time and that one mom who always has Pinterest perfect decorations everyone won’t shut up about is going to be there so you are willing to pay for help.

That’s the beauty of allowing others to recruit for you – it frees up your bandwidth to do other important things, such as analyzing insights, creating reports and presenting them to the team or preparing for the next sprint. Keep in mind, though, that you will be paying for that breathing room.

We’ll cover two different aspects of sourcing vendors – general and specialized panel vendors.

A.) General panel vendors

The good news is that this industry has quite a lot of players in it, which means there is a very good chance that you can find the people you’re looking for.

One thing to keep in mind is that some panel vendors focus on a specific market or on a specific geographic location. For example, there are vendors who focus on mobile users while others focus on users who are located in Canada. More on that below.

Once you find a panel vendor the usual process is they will ask you to provide a demographic screener for the type(s) of participants you’re looking for. They’ll run that through their database to see how many fit the description and provide you with an estimated CPP (cost per participant) which will then be used to supply a quote based on the N, or sample size, you’re looking to get feedback from.

Keep in mind that this cost also covers all the work that the panel provider will be doing. They will be doing the inviting, they will be the ones handling compensating the participants, they will be the one who do all the manual work and interact with participants through emails. If you have the budget for it this can be a lifesaver.

But what if you’re looking for something very specific, and not just a little specific? That’s where specialized panels come in.

B.) Specialized panel vendors

What do we mean by specialized? Let’s use accessibility testing as an example. If you want to test your site with participants that have visual or motor impairments, you might find that other panels would have a very hard time providing them. Instead of saying it would be too difficult or costly to take into consideration (and it’s our belief that the internet should be accessible to all) take some time to research on whether or not there is a panel out there that focuses on it.

The good news is that, for this example, there are panel providers like Knowbility. Knowbility Inc. maintains the AccessWorks panel, which includes people who are blind or have low vision, physical/motor impairments, hearing impairments, and/or cognitive impairments.

Other examples of specialized panel providers would be Reckner, who focus on doctors, nurses, and other healthcare professionals, as well as Schlesinger Group, who focus on getting hard to reach audiences for qualitative testing.

The main takeaway here is that if you do find yourself wanting or needing to run a test with a very specific, harder than usual demographic do a little digging to see if a panel exists that caters to this specific need.

3) Guerrilla testing

Guerrilla testing, or guerilla research, is basically reaching out to readily available people around you. Sometimes referred to as “down and dirty” testing in that it’s typically cheap, fast, and fairly casual. It is also sometimes referred to as the “coffee shop method” as it isn’t uncommon to simply step into a coffee shop and offer to buy people a coffee if they give you some quick feedback.

Sometimes you may want to test with participants that are right under your nose, such as coworkers and/or friends and family. If you decide to conduct internal testing with coworkers make sure that they have had no involvement in the design or development of the site or product you are testing, and most importantly, that they represent a target audience.

The same with your family – if your mom recognizes this is the thing you’ve been talking about at family BBQ’s she will most likely have a biased opinion, but your cousin Reggie who never shows up to family gatherings might be okay.

Is this method extremely targeted? No (unless you’re working on a coffee related product or your family are customers). But it is feedback and it is cheap and relatively easy to gather.

Pros:

• Cheap from an overhead and compensation perspective
• Easy to pick a place and walk in with a laptop
• Surplus of coffee shops with wi-fi
• You know where your coworkers sit
• Gives you a chance to catch up with Reggie

Cons:

• Most likely not testing with your target audience
• Not necessarily a ‘natural’ setting so may not be capturing natural user behavior
• Very difficult to do at scale
• Friends, family and coworkers can carry a bias
• Tests the patience of coffee shop staff?

Related reading: to help you get started, check out our user testing participant recruitment resources!