How PayPal increased credibility and confidence with multiple research methods

Deborah Torres, Sr. UX Research and Ops Manager at Paypal, shares how UX research is done at PayPal.

With 113.2 million active accounts, every UX related decision has a significant impact. Deborah and her team, like many other UX researchers, face two main challenges:

1) Delivering quality credible results to stakeholders and decision-makers;
2) Getting it all done quickly and efficiently.

Using data to tell a complete story

PayPal’s research team uses multiple research methods to provide a complete story about the customer experience and to gain credibility with their decision-makers.

“Sometimes I go into a room and present to stakeholders the results of a usability study with 10 participants. However, those executives really want to know what is beyond those 10 participants. So triangulation, whether it’s through remote usability testing or through weblogs, allows me to confidently say that this problem holds true in the entire world with a sample of 200, 400 or even 1000 users,” says Deborah.

However, where do you get time, money, and resources to do all this research?

Deborah says that she gets requests for research that needs to be done quickly and with limited resources. On top of that, PayPal has a small team of 6 UX researchers who are expected to deliver 42 research studies, talk to 413 participants, and field 271,244 touchpoint surveys – all within 6 months.

"Complaining about constraints wouldn’t help. You need to figure out how you can possibly scale your research. This is the time when you should consider using tools to maximize the time and money that you do have."

Deborah Torres, PayPal

Sr. UX Research and Ops Manager

Problem: reducing the number of support calls

PayPal needed to reduce the number of customer support calls from people who wanted to find out how they could service their account better. Most of the calls involved users wanting to update their account information such as email, password, or credit card information. The goal for the research team was to make it easier for these folks to self-serve themselves online.

Initially, this was the page that these tasks could be completed. It looked more like a link farm, according to Deborah.

PayPal’s UX Research team needed to improve the findability of this page and to ensure that its users could use this page to complete the intended tasks.

Creating a research plan

As a first step, PayPal’s research team wanted to make sure that they truly identified the top tasks that users came to perform upon hitting the profile page.

Secondly, they needed to understand what the users’ mental model for servicing their account was.

And lastly, upon using the inputs from the steps above to create an alternative design, they wanted to gather qualitative feedback on the new design.

Tackling the methodology part

PayPal uses a Tracker Survey (similar to True Intent Studies) to identify the top tasks that users are engaging in and whether they are successful. They also look at web traffic and weblogs to confirm what these top tasks are (SiteCatalyst data).

For mental models and organizing concepts, they use card sorts to see how people group items. Finally, for gathering qualitative feedback, they run a usability test.

Executing the research

Since Paypal had a tracker survey + web traffic data, additional time and cost were not needed for identifying the top tasks. Qualitative feedback is very important to Paypal. As such, they didn’t mind spending money on a traditional in-lab usability test.

However, when it came to card sorting, they had to figure out how to be more efficient with their resources. As such, they wanted to study how a remote unmoderated card sort compared with a traditional in-lab one.

“With remote unmoderated card sort, we can have more participants and spend less time traveling, setting up the lab, and recruiting users. We also save time on the analysis of the gathered data. Online card sorting tools do a really good job of outputting a nice tree diagram or some other type of quantitative measures to help make the decision quicker. In the lab, you may have to do some manual work. Also, in-lab testing can get rather expensive, whereas remote unmoderated tools can save a lot of money,” says Deborah.

The UX team thought it was important to get quantitative feedback since they were going to use the card sort input into the design that would be tested with actual users in step 3 shown above. More feedback and larger sample sizes would give them more confidence to take the findings back to the design team to iterate and to design on.

In the end, they chose to use UserZoom’s remote unmoderated card sorting solution for this project. As such, they were able to deliver a research study that was cost and time-efficient. They were able to get the design that they needed and to push it out of the door on time.

At the end of the webinar, Deborah left everyone with the following parting words: “Here is the 10% that you should go away remembering – triangulation. Triangulation and using multiple research methods tell a more comprehensive story. It increases the credibility of the research and increases the confidence of the decision-makers. And secondly, be sure to utilize tools to execute efficiently.

There are a number of research tools out there and you should definitely consider using them appropriately to save time, money, and your heartache.”

Find out how UserZoom can help your team