UX Battle of the Week: T-Mobile vs Verizon
The Task-Based UX Benchmark Study
UserZoom ran a quick task-based UX benchmark study between T-Mobile and Verizon, two well-known wireless service providers, to compare the experience of finding information about setting up automatic payments online.
- We ran the unmoderated remote task-based benchmark study with 60 in-the-wild users on their own devices over the course of a single day
- We split participants equally between each website so that 30 went to T-Mobile and 30 went to Verizon
- The Online Task: Find the page that tells you how to set up automatic payments for your bill
- We validated the task via URL validation when participants got to the correct page
Let’s Get Ready to Rumble!
We showed all 60 participants an image of T-Mobile’s and Verizon’s homepage and asked them to choose which site they associated with several UX attributes.
This week’s participants preferred T-Mobile’s homepage over Verizon’s in every category, particularly when it came to being seen as Easy, Welcoming, Lively and Helpful. The votes were much more close when it came to being seen as Organized, Trustworthy, and interestingly enough, Overwhelming.
Screenshot Click Test
We split the participants evenly and asked where they would click if they wanted to learn more about the carrier’s business solutions.
60% of participants correctly clicked on Business in the top navigation bar. Of the outside areas, Plans and Why T-Mobile were the highest clicked areas for a combined 30% of the clicks.
77% of participants correctly clicked on Business in the top navigation bar. Smartphones, Plans, Tablets and Why Verizon areas had a combined 20% of the clicks.
We asked participants where they felt they would be able to learn more about international coverage.
T-Mobile had a 93% success rate, with participants finding the correct selection in under 10 seconds. Verizon had a 33% success rate with an average time of 23 seconds.
Online Task Outcomes
In order to validate whether users were successful at finding the page for automatic bill payments, we did a URL validation on the page. If users managed to get to the page from the homepage they were labeled as Success.
Non Success meant that a user either Abandoned the task due to difficulties with the website, or said they had completed the task without reaching the correct page, which we labeled as Error.
Users were 20% more successful at finding information for setting up automatic bill payments on Verizon’s site than on T-Mobile’s.
Successful users on Verizon’s site were able to find the page in about 47 seconds less time than T-Mobile’s participants and with less clicks. Both sites had similar amounts of average page views.
T-Mobile user session
Verizon user session
T-Mobile users were slightly more likely to abandon the task entirely than to make an error whereas Verizon users were more likely to err than abandon. What’s interesting is that participants on Verizon’s site tended to stay on the website far longer before abandoning or erring compared to participants on T-Mobile’s site.
T-Mobile user session
Verizon user session
We split the participants equally between brands and asked them to rate their perception of the brand before and after their experience with the site.
Brand Perception Pre-Task
For the rating scale: 1 = Very Negative, 4 = Neutral, and 7 = Very Positive. We also included the option for participants to say they’re not familiar with the brand.
T-Mobile started with a neutral brand perception of 4.6, while Verizon started with a slightly higher score of 5.3 out of 7.
Brand Perception Post-Task
After participants interacted with T-Mobile’s site their brand perception rose slightly to 4.8 out of 7. 44% of users giving them the highest score, which was up from 37% pre-task.
Verizon saw a slight decrease in brand perception after users interacted with the site, ending with 4.6 out of 7. 37% of users still gave them the highest score, but that dropped down from 53% pre-task.
Ease of Use
After the task we asked all the users to rate how easy or difficult it was to accomplish, with 1 = Very Difficult, 4 = Neutral and 7 = Very Easy.
Given the success rates this is probably not very surprising, but participants on the T-Mobile site felt that the task of finding information about automatic bill payments on the site was rather difficult – the average ease of use score was a 2.8 out of 7.
Verizon fared slightly better and ended with a neutral ease of use rating of 4.2 out of 7.
After the task we asked all the users to rate how satisfied they were with the site, with 1 = Very Unsatisfied, 4 = Neutral and 7 = Very Satisfied.
Both brands had user satisfaction ratings that were fairly evenly spread out across the spectrum. T-Mobile’s users ended up giving an average satisfaction rating of 3.9 out of 7, while Verizon’s users average rating was 4.2.
Problems & Frustrations
We asked the users which, if any, of the following problems they encountered while on the site.
If users chose Other they were given an open-ended question to describe their problem or frustration. The users who chose this for Verizon said:
- “The video is a great idea but it was a little buried and hard to find”
- “Should have a ‘videos’ section that’s easy to find in the main menu”
Net Promoter Score (NPS)
After participants interacted with the sites we asked them to rate how likely it was they would recommend them to friends, family or colleagues.
Both brands were incredibly close in regards to NPS, but in the end T-Mobile had one more Promoter than Verizon, which edged them in for a narrow victory.
The winner this week is Verizon for having a higher percentage of users successfully find the automatic payment information, as well as for having a higher overall ease of use and user satisfaction score.
Want to learn more about competitive
UX Benchmarking with UserZoom?
Thank you for signing up!
You should receive an email from us shortly