The Task-Based UX Benchmark Study

UserZoom ran a quick task-based UX benchmark study between Vanguard and T. Rowe Price, two well-known financial and retirement planning companies, to compare the experience of applying for a Roth IRA online.

  • We ran the unmoderated remote task-based benchmark study with 60 in-the-wild users on their own devices over the course of a single day
  • We split participants equally between each website so that 30 went to Vanguard and 30 went to T. Rowe Price
  • The Online Task: You want to open a personal Roth IRA. From the homepage, navigate to where you would start a personal Roth IRA. When you have reached the correct page we will automatically redirect you and ask a few questions.
  • We validated the tasks via URL validation

conclusion anchor button

Let’s Get Ready to Rumble!

We showed all participants an image of Vanguard’s and T. Rowe Price’s homepage and asked them to choose which site they associated with several UX attributes.

Screen Shot 2016-07-13 at 10.59.03 AM

Participants this week preferred T. Rowe Price’s homepage to Vanguard’s in most categories. Where Vanguard did pull ahead was in the Lively and Overwhelming categories.

Screenshot Click Test

We split the participants evenly and asked where they would click on the personal investment page if they were looking for a help center.

Vanguard

vanguard heatmap

30% of participants correctly clicked on Client Support in the header, while 10% correctly clicked on Technical Support in the footer. The most clicked incorrect place that participants felt would lead them to a help center was Advice & Guidance.

T. Rowe Price

T. Rowe Price heatmap

93% of participants correctly clicked on Help in the footer.

Tree Test

We split the participants evenly and asked where they’d be able to learn more information about advisory planning services.

Vanguard

vanguard TT

90% of participants correctly clicked on Partner with an Advisor. A majority of them, 73%, got it right on their first selection.

T. Rowe Price

t. rowe price TT

50% of participants correctly clicked on Advisory Planning Services. Just under half of the successful participants got it right on their first selection.

Online Task Outcomes

We asked participants to navigate to where they would start an application for a personal Roth IRA. In order to validate whether users were successful at the task we validated via the URL of the beginning of the application process for a Roth IRA.

If users got to this page they were labeled as Success.

Non Success meant that a user either Abandoned the task due to difficulties with the website or said they had started to apply for a Roth IRA before reaching the correct page, which we labeled as Error.

Screen Shot 2016-07-11 at 3.09.24 PM

Success

Both sites had a majority of users successfully complete the online task, but users on T. Rowe Price’s site were 4% more likely to successfully complete the task than Vanguard’s users. This was an incredibly close battle, with the difference coming down to a single participant difference.

Screen Shot 2016-07-11 at 3.13.53 PM

Both sites were also extremely close in regards to how efficiently participants were able to complete the online task. However, Vanguard’s participants were slightly more efficient overall than T. Rowe Price’s participants in completing the task.

Vanguard user session

T. Rowe Price user session

Non Success

Screen Shot 2016-07-11 at 3.23.18 PM

Once more we see a startling similarity among the sites but this time for their non successful participants. Not only were the participants evenly spread between Error and Abandon, their time on task, page views and clicks are eerily similar as well.

Overall the users on both sites who abandoned had the same two main issues: a few were worried about their personal information (the study wouldn’t have let them get that far) and abandoned early, but most simply couldn’t find the correct section of the website for a Roth IRA, such as in the videos below where the participants go to the retirement plans through an employer sections of the websites.

These two comments sum up the abandon comments we received:

  • It wasn’t anywhere near as clear as it should have been. I didn’t even know where to start, let alone what I was looking for
  • Wasn’t sure exactly where to go, seemed like there was a lot of information on the page and it was overwhelming without being too clear

Vanguard user session

T. Rowe Price user session

Ease of Use

After the tasks we asked all the users to rate how easy or difficult it was to accomplish, with 1 = Very Difficult, 4 = Neutral and 7 = Very Easy.

Screen Shot 2016-07-11 at 3.47.52 PM

We’re seeing a pattern of similarity here. This group of participants responses were spread fairly evenly on both sites, resulting in the neutral average ease of use scores.

Problems & Frustrations

We asked the users which, if any, of the following problems they encountered while on the site.

Vanguard

Screen Shot 2016-07-12 at 3.13.08 PM

The participant who chose other said:

  • I got lost among all the information and was overwhelmed

T. Rowe Price

Screen Shot 2016-07-12 at 3.15.58 PM

Similar difficulties were shared for both sites. Namely, that participants were unsure on where to begin the task and that confusing navigation was causing participants to spend too much time clicking around the site.

Brand Perception

We asked participants to rate their perception of the brand before and after their experience with the site.

For the rating scale: 1 = Very Negative, 4 = Neutral, and 7 = Very Positive. We also included the option for participants to say they weren’t familiar with the brand before the task.

Screen Shot 2016-07-12 at 3.31.17 PM

Both brand’s scores dropped down after participants interacted with the site, which is fairly typical. It should be noted that 40% of T. Rowe Price’s participants gave them the highest ratings possible, while only 24% of Vanguard’s participants gave them the highest ratings.

Net Promoter Score (NPS)

After participants interacted with the sites we asked them to rate how likely it was they would recommend them to friends, family or colleagues.

Screen Shot 2016-07-12 at 5.10.42 PM

In what shouldn’t be a surprise to anyone that has read through the previous results – both brands ended up with the same NPS. Despite the same overall NPS, however, Vanguard had more people say they would promote the company than T. Rowe Price.

Conclusion

Screen Shot 2016-07-13 at 11.03.58 AM

This week’s matchup was probably the most neck and neck we’ve seen in our battles of the week, but in the end when the smoke cleared and the participants finished, one brand emerged as the victor. The winner of this week’s fierce usability battle is T. Rowe Price! 

T. Rowe Price ended up having a narrow lead in overall successes, less problems and frustrations reported, and a higher final brand perception. Vanguard was right there the entire battle, however, and had slightly more efficient success times and a greater percentage of success in the tree test. Both sites tied in regards to ease of use and Net Promoter Scores.


Want to learn more about competitive
UX Benchmarking with UserZoom?

personalized demo