The complexities of international usability testing
Set your expectations appropriately for international testing.
As companies expand into international markets they are realizing that thinking globally and acting locally means ensuring that their product is optimized for all customers or users, wherever they may be. Sometimes businesses accidentally offend users from a different culture, which can harm their brand. This is why international usability testing is extremely important; it helps companies build confidence that their products will be adopted and enjoyed by users from various geographies or cultures.
This might mean repositioning your brand or your brand message for different markets based on the feedback you receive from your global users, but getting this kind of actionable feedback from users abroad can be difficult. Given the expenses companies incur while traveling and renting usability lab facilities, many have instead opted for remote international usability testing which allows them to collect feedback from users around the world without ever leaving their chairs.
In this article we’ll take a look at some of the challenges researchers face while conducting remote international usability testing.
Recruiting – Finding your target users
This is a common research challenge regardless of where your users live, but when you need users from specific countries and cultures your participant pool becomes significantly smaller. Recruiting participants from specific regions, countries, or even states/provinces/cities can be challenging and expensive. And even more so as you as you filter the populace for specific demographics and profiles. However, there are several ways to recruit the right participants for your international studies:
- Having a relationship with a panel vendor that focuses on international testing can be a huge boon when it comes to recruiting users. You should always ask about their testing pool and whether they have historically had success providing participants from a country or culture.
- If you already have a localized site or app you should consider opt-in recruitment methods such as invitation layers or feedback tabs that invite organic users to participate in your testing.
- Consider recruiting from your own customer database – people who are already users of your product or service within the countries you want to perform your research.
- Recruiting on your own through social media or other online outlets where your target users are likely to be found. This can be especially useful for hard-to-find profiles (for example, Rocket Scientists in Norway) and when you only need a small number of users.
Regardless of how you recruit your target audience you have to ensure that your usability test speaks to your audience while staying focused on your objective to discover what you are hoping to learn, which brings us to our next challenge.
Language – Getting your point across
Language is a beautiful, fickle thing filled with subtleties that are easy to miss if you’re not a native speaker. There are tons of examples of company slogans having unintended (and sometimes hilarious, sometimes horrifying) translations when taken abroad. Obviously, testing slogans with native speakers is important to make sure the intended message gets across but this also underlies another important aspect of international testing. Language can be a bit tricky.
This is why we would always recommend using a native speaker or professional translation service for test scripts. If you are doing research in a particular country there may be native speakers within your own company that you can reach out to for help. Most usability test scripts will only have a few pages of text to translate.
Analysis and reporting, however, are much bigger challenges to translate than test scripts. This is where you will definitely need a partner (again, either a native speaker or a translation service) who can help with not only translating large amounts of written and verbal responses, but who can also help you with the overall analysis of the results in terms of interpreting cultural factors and biases. You don’t want to miss out on a critical cultural insight due to the inability to fully understand the user’s written or spoken feedback.
Speaking of Speaking…
If you’re looking for talk-aloud or think-out-loud feedback for your usability study, getting quality results can be challenging in any country. Unless you are using a panel of coached usability testers, think-aloud protocol is still relatively new for the general population, even in the U.S. So set your expectations appropriately. It’s likely to be difficult to get good verbal feedback from international audiences. Difficult but not impossible as long as you allow for more planning, time and (potentially) cost.
Of particular note is China. Though many companies are setting their sights on large expansions into China, the world of market research (& panel providers) has not yet evolved to keep up with the demand from researchers to conduct usability studies or other research with the Chinese population.
We’ve found that the best approach to recruit participants and achieve quality feedback is by combining the above points:
- Evaluate panel vendors closely for their ability to successfully recruit in your desired countries
- Establish a close relationship with your chosen panel vendor as you may need to be flexible & creative to achieve the desired number or profile of participants
- Partner with a translation vendor or native speaker who will stay with you for the duration of the project to translate your usability test script and responses, to help with your analysis of the findings. For a remote unmoderated usability test, your translation partner should be able to guide participants in how to provide feedback in both written and verbal (think-aloud) form. You might also consider sharing video examples of speak-aloud sessions in the participants’ local language so they better understand what is expected of them. For live moderated sessions, your translation partner can also assist the moderator to coach participants to speak aloud during the session.
If your target audience is global there are many cultural subtleties that must be taken into account. In terms of truly “localizing” your site or app, you will want to keep an eye out for any international nuances that may not be appropriate for a local audience. For instance, is your site’s messaging appropriate for an international audience? What do the colors on your app represent? Are they perceived offensively? Be proactive – research and test early and often to avoid re-work later.
Performing international testing and research can help you ensure that your sites and apps will be both valuable to, and adopted by, your target users here and abroad. While there are challenges presented by the logistics of international testing, they are not insurmountable. With the right amount of planning, preparation and patience your international testing can reap great rewards.
Phil got his degree in creative writing, where they told him he most likely wouldn’t be able to use his degree for his career. He obviously won that round. When not working with UX researchers he can be found teaching martial arts and working on his fiction novels.
Toni has over 20 years of experience in roles focused on UX, User Research, Product Management and Marketing and has helped hundreds of companies realize the benefits of using remote testing platforms to gain powerful insights quickly and cost-effectively.