The Task-Based UX Benchmark Study
UserZoom ran a quick task-based UX benchmark study between REI and Patagonia, two well-known outdoors and sports gear retailers, to compare the experience of finding products on their sites as well as the checkout process.
- We ran the unmoderated remote task-based benchmark study with 60 in-the-wild users on their own devices over the course of a single day
- We split participants equally between each website so that 30 went to REI and 30 went to Patagonia
- The Online Task: From the homepage, find a climbing pack and add it to your cart. Then proceed through the checkout process as a guest – we will automatically redirect you at the shipping information page.
- We validated the tasks via URL validation
Let’s Get Ready to Rumble!
We showed all participants an image of REI’s and Patagonia’s homepage and asked them to choose which site they associated with several UX attributes.
As our battles continuously show us, user first impressions tend to be more favorable towards the homepage with minimal scrolling and text (Patagonia) while the homepage that requires more scrolling and has more text (REI) is usually considered both Informational and Overwhelming.
In our experience, it’s rare that these attributes aren’t associated together which is a consideration companies should take into account – information overload can be overwhelming to your first time visitors, and isn’t seen as being more Helpful than a minimal homepage (as long as visitors can intuitively find what they’re looking for on the minimal page).
Screenshot Click Test
We split the participants evenly and asked where they would click if they wanted to register for an account online.
57% of participants clicked on Account in the header. In the footer, 13% clicked on Your Online Account while 10% clicked on Membership.
47% of participants clicked on the Account image, but since it’s not labeled, we can see more spread of clicks in other areas. Interestingly, 27% of participants clicked on Sign Up For Patagonia Emails despite being clearly labeled for emails and not an account.
We split the participants evenly and asked where they’d be able to find gift cards.
67% of participants correctly selected Gift Cards under “More.” Everyone who correctly selected this area did so on their first try. The next highest area participants selected was Deals, with 23% of participants incorrectly clicking here.
An impressive 100% of participants were able to correctly find and select Giftcards. Out of all participants, 87% found it on their first try. The remaining 13% looked at Store under “Inside Patagonia” before going back to Shop.
Online Task Outcomes
We asked participants to find a climbing pack for their upcoming summer trip and place it in their cart, and then proceed through the checkout process as a guest until the shipping information page . In order to validate whether users were successful at the task we validated via the URL of the shipping page.
If users got to this page they were labeled as Success.
Non Success meant that a user either Abandoned the task due to difficulties with the website or said they had finished the checkout process before reaching the correct page, which we labeled as Error.
Users on both sites had a majority of users successfully complete the checkout process, but users on REI’s site were 13% more likely to successfully complete the task than Patagonia’s users.
Despite having more users successfully complete the task on REI’s site, Patagonia’s successful users were able to complete the task more quickly than REI’s. What’s interesting is that both sites had practically the same amount of page views and clicks required, but users on Patagonia’s site were faster.
REI user session
Patagonia user session
Note that the participant tries to add the climbing pack to their cart without choosing a size (0:32) and doesn’t seem to notice the error message, or understand right off the bat what the error message is for.
Participants were more likely to err than abandon the task on both sites, meaning that they thought they had completed the checkout process when they hadn’t. No participants abandoned the task on REI’s site while 2 abandoned the task on Patagonia’s site.
When we asked them a follow up question asking why they had abandoned, this is what they said:
- I wasn’t able to find the item requested
- When I tried to checkout there was a page error twice in a row (see the Patagonia video below)
REI user session
Patagonia user session
Ease of Use
After the tasks we asked all the users to rate how easy or difficult it was to accomplish, with 1 = Very Difficult, 4 = Neutral and 7 = Very Easy.
Kudos to both brands – participants on average felt the task was easy to accomplish, but REI had slightly more people rate them easier to use than Patagonia.
Problems & Frustrations
We asked the users which, if any, of the following problems they encountered while on the site.
Participants who selected “Other” were given an open-ended follow up question to explain. The users who chose with for REI said:
- On the page for gift cards the site didn’t load all the way
- Couldn’t find any packs specifically for climbing
- Homepage was cluttered
We asked participants to rate their perception of the brand before and after their experience with the site.
For the rating scale: 1 = Very Negative, 4 = Neutral, and 7 = Very Positive. We also included the option for participants to say they weren’t familiar with the brand before the task.
REI saw a tiny dip in brand perception after participants interacted with the site, ending with a positive 5.4 out of 7. Patagonia saw a nice bump in brand perception after participants interacted with the site, bringing their final brand perception rating up to 5.9 out of 7.
Net Promoter Score (NPS)
After participants interacted with the sites we asked them to rate how likely it was they would recommend them to friends, family or colleagues.
Participants were less likely to recommend REI’s site to friends and family than Patagonia’s participants were.
For retailers, the customer’s ability to find the products they’re after and completing the purchase process is paramount. In this regard, both sites performed very well. The winner, however, in this week’s usability battle is REI for having a higher overall success rate on the online tasks with fewer abandons and a slightly higher overall ease of use rating. Patagonia put up a tough fight, and should be proud of their brand perception and NPS ratings, their task efficiency, and their taxonomy.