The Task-Based UX Benchmark Study

UserZoom ran a quick task-based UX benchmark study between Wayfair and Overstock, two well-known online retailers, to compare the experience of applying for a store credit card.

  • We ran the unmoderated remote task-based benchmark study with 60 in-the-wild users on their own devices over the course of a single day
  • We split participants equally between each website so that 30 went to Wayfair and 30 went to Overstock
  • The Online Task: Find the page where you would apply for a store credit card and begin the application process
  • We validated the task via URL validation when participants got to the first page of the application process

Opinion Poll: Who Will Win?

Last week we asked our readers who they felt would win this UX Battle of the Week.

UX BOTW: Wayfair v Overstock poll

Well, feeling lucky? Read on to see if you were right (or skip ahead if you simply can’t contain your excitement).

Let’s Get Ready to Rumble!

We showed all 60 participants an image of Wayfair’s and Overstock’s homepage and asked them to choose which site they associated with several UX attributes.

UX Battle of the Week: Wayfair vs Overstock homepage

UX Battle of the Week: Wayfair vs Overstock UX attributes

This week’s participants preferred Wayfair’s homepage over Overstock’s in almost every category, particularly when it came to being seen as Easy, Welcoming, Informational and Helpful. The votes were much more close when it came to being seen as Trustworthy and Lively. A vast majority of participants felt that Overstock’s homepage was Overwhelming.

Screenshot Click Test

We split the participants evenly and asked where they would click if they wanted to track an order.

Wayfair

UX Battle of the Week: Wayfair heatmap

80% of participants correctly clicked on Account in either the top navigation bar or on the footer of the page. Of the 80%, 73% clicked on Account at the top of the page.

Overstock

UX Battle of the Week: Overstock heatmap

86% of participants correctly clicked on My Account in either the top of the page or in the footer. Interestingly, these were split evenly between top and bottom with 43% each. Online Help Center was also a correct place to click but no one chose that as the first place they would click.

Tree Test

We asked participants where they felt they would be able to find a new washing machine.

Wayfair

UX Battle of the Week: Wayfair tree test

Wayfair had a 67% success rate with an average time of 27 seconds to find the correct area. The next highest area users incorrectly selected was Kitchen appliances with 10%.

Overstock

UX Battle of the Week: Overstock tree test

Overstock had a 33% success rate with an average time of 37 seconds to find the right area. 40% thought it would be under housewares while 13% felt it would be under bathroom.

Online Task Outcomes

In order to validate whether users were successful at starting the process for a store credit card, we did a URL validation on the first page of the process. If users managed to get to the page from the homepage they were labeled as Success.

Non Success meant that a user either Abandoned the task due to difficulties with the website, or said they had completed the task without reaching the correct page, which we labeled as Error.

UX Battle of the Week: Wayfair vs Overstock success rates

Success

Users were 13% more successful at starting the credit card application process on Wayfair’s site than on Overstock’s.

UX Battle of the Week: Wayfair vs Overstock time on task

Despite having more users successfully complete the task on Wayfair’s site, Overtstock’s participants were able to complete the task about 16 seconds faster than Wayfair’s participants. Both sites had similar amounts of page views and clicks required to complete the task.

Wayfair user session

Overstock user session

Non Success

UX Battle of the Week: Wayfair vs Overstock non-success times

Users on both sites were more likely to err than abandon, stopping at the page where they saw the button or link to begin the process.

Of the users who abandoned it was evident that they either felt they had to make an account first, which led to them abandoning the site before even finding the page to begin the credit card application process, or they balked at clicking the apply now buttons.

Wayfair user session

Overstock user session

Brand Perception

We split the participants equally between brands and asked them to rate their perception of the brand before and after their experience with the site.

For the rating scale: 1 = Very Negative, 4 = Neutral, and 7 = Very Positive. We also included the option for participants to say they’re not familiar with the brand.

Brand Perception Pre-Task

UX BOTW: wayfair v Overstock brand perception pre-task

Before participants interacted with the site, Wayfair had an average brand perception rating of 5.7 out of 7 with 33% of users say they were unfamiliar with the brand.

Overstock’s brand perception was 5.5 out of 7 with only 13% of users saying they were unfamiliar with the brand.

Brand Perception Post-Task

UX BOTW: Wayfair v Overstock brand perception post-task

After participants interacted with Wayfair’s site the overall brand perception went up to 5.9 out of 7 with zero negative ratings.

Overstock’s overall brand perception from its users dropped down to 5.1 out of 7 with 10% negative ratings.

Ease of Use

After the task we asked all the users to rate how easy or difficult it was to accomplish, with 1 = Very Difficult, 4 = Neutral and 7 = Very Easy.

UX BOTW: Wayfair v Overstock Ease of Use

Participants on Wayfair’s site felt that the task was overall fairly easy to accomplish, giving it an average rating of 5.4 out of 7. 77% gave them the highest ratings possible.

Overstock scored slightly lower, ending with a more neutral overall ease of use rating of 4.8 out of 7. 40% of participants gave them the highest ratings possible.

Problems & Frustrations

We asked the users which, if any, of the following problems they encountered while on the site.

Wayfair

UX Battle of the Week: Wayfair problems and frustrations

Overstock

UX BOTW: Overstock problems and frustrations

If users chose Other they were given an open-ended question to describe their problem or frustration. The users who chose this for Overstock said:

  • “There was a debit and credit card offering. The debit (Mastercard) was easy to find on the homepage but the credit card was buried and hard to find.”
  • “Couldn’t find a proper category for the washing machine earlier”

Net Promoter Score (NPS)

After participants interacted with the sites we asked them to rate how likely it was they would recommend them to friends, family or colleagues.

UX BOTW: Wayfair v Overstock NPS

Both brands did well but in the end Wayfair’s participants were more likely to recommend the site than Overstock’s participants.

Conclusion

UX BOTW: winner-wayfair

The winner this week is Wayfair for having a higher percentage of users successfully complete the task of starting the process of applying for a store credit card, as well as for having a higher overall ease of use, NPS and brand perception score.


Want to learn more about competitive
UX Benchmarking with UserZoom?

personalized demo