The Task-Based UX Benchmark Study

UserZoom ran a quick task-based UX benchmark study between Pandora and SiriusXM, two well-known online music providers, to compare the experience of registering a new account, finding information about advertising online as well as purchasing gift cards from their site.

  • We ran the unmoderated remote task-based benchmark study with 60 in-the-wild users on their own devices over the course of a single day
  • We split participants equally between each website so that 30 went to Pandora and 30 went to SiriusXM
  • The Online Task: Find the page where you would be able to purchase a gift card
  • We validated the task via URL validation

conclusion anchor button

Let’s Get Ready to Rumble!

We showed all 60 participants an image of Pandora’s and SiriusXM’s homepage and asked them to choose which site they associated with several UX attributes.

pandora desktops

UX Battle of the Week - Pandora v SiriusXM brand attributes

Pandora’s minimalist approach to their homepage won overwhelmingly in regards to being seen as Easy. SiriusXM’s homepage swept the remaining positive, and negative, categories – Informational in particular. Interestingly, both websites were tied when it came to being seen as Trustworthy.

Screenshot Click Test

We split the participants evenly and asked where they would click if they wanted to register a new account.

Pandora

Pandora heatmap - UX Battle of the Week

83% of participants successfully clicked on an area where they would be able to register a new account. The vast majority, 73%, clicked on Sign Up in the middle of the page while 10% clicked on Sign Up in the header.

SiriusXM Radio

UX Battle of the Week - SiriusXM heatmap

17% of participants successfully clicked on an area where they would be able to register a new account. 10% clicked on My Account in the header while 7% clicked on the Get Started tab. Of the outside areas participants erroneously clicked, 37% chose Sign Up For a Trial.

Tree Test

We asked participants where they felt they would be able to find information about advertising their business to listeners.

Pandora

Pandora Tree Test - UX Battle of the Week

80% of participants successfully chose Advertising, with 73% of participants getting it right on their first choice. Of the outside areas participants selected, a majority (13%) chose Businesses.

SiriusXM

SiriusXM Tree Test - UX Battle of the Week14% of participants successfully chose Advertise With Us, with 3 out of the 4 successful participants getting it right on their first choice. A vast portion of participants, 46%, felt that How Can We Help? would be the correct area.

Online Task Outcomes

In order to validate whether users were successful at finding the gift cards we validated the task via a URL validation. If users got to the correct page, they were labeled as Success.

Non Success meant that a user either Abandoned the task due to difficulties with the website or said they were done before reaching the correct page, which we labeled as Error.

Pandora vs SiriusXM Success Ratios - UX Battle of the Week

Success

Users were 20% more successful on Pandora’s site than on SiriusXM’s.

Pandora vs SiriusXM Success & Efficiency Ratios - UX Battle of the Week

Not only were users more likely to be successful on Pandora’s site, they were also able to complete the task in far less time, less page views and less clicks.

Pandora user session

SiriusXM user session

Non Success

Pandora vs SiriusXM Non Success Ratios - UX Battle of the Week

Non successful users on both sites were far more likely to abandon the site than to make an error. For both sites, users tended to abandon after not being able to navigate to the correct section and not finding the right page after using the search function.

Pandora user session

SiriusXM user session

Brand Perception

We split the participants equally between brands and asked them to rate their perception of the brand before and after their experience with the site.

For the rating scale: 1 = Very Negative, 4 = Neutral, and 7 = Very Positive. We also included the option for participants to say they’re not familiar with the brand.

Brand Perception Pre-Task

Pandora vs SiriusXM Pre-Task Brand Perception - UX Battle of the Week

Both sites started off with positive ratings – 5.7 out of 7 for Pandora and 5.2 out of 7 for SiriusXM.

Brand Perception Post-Task

Pandora vs SiriusXM Post-Task Brand Perception - UX Battle of the Week

After participants interacted with Pandora, the overall brand perception dropped down a fraction of a point to 5.6 out of 7. 60% of participants gave them the highest scores possible.

SiriusXM saw a greater drop in their score, down to a 4.3 out of 7. 20% of users gave them the highest scores.

Ease of Use

After the task we asked all the users to rate how easy or difficult it was to accomplish, with 1 = Very Difficult, 4 = Neutral and 7 = Very Easy.

Pandora vs SiriusXM Ease of Use - UX Battle of the Week

Pandora’s users felt the website was easy to use, with 80% of participants giving it the highest ease of use scores for an overall rating of 6 out of 7.

SiriusXM’s users felt the task was difficult to accomplish on the site, ending with an average ease of use rating of 2.8 out of 7. 10% of users gave SiriusXM’s site the highest ease of use scores possible.

Problems & Frustrations

We asked the users which, if any, of the following problems they encountered while on the site.

Pandora

Pandora Problems & Frustrations - UX Battle of the Week

SiriusXM

Sirius Problems & Frustrations - UX Battle of the Week

Net Promoter Score (NPS)

After participants interacted with the sites we asked them to rate how likely it was they would recommend them to friends, family or colleagues.

Pandora vs SiriusXM NPS - UX Battle of the Week

Pandora’s users more likely to recommend the site to friends and family than SiriusXM’s users.

Conclusion

UX Battle of the Week winner - Pandora vs SiriusXM

The winner this week between the battle of online music providers is Pandora for having higher success rates in regards to registering a new account, finding out how to advertise to listeners, and our task on finding the page for gift cards. Pandora also had higher brand perception, ease of use, and net promoter scores.


Want to learn more about competitive
UX Benchmarking with UserZoom?

personalized demo