UserZoom’s UX Battle of the Week turns one!

It was exactly one year ago today that we launched our very first UX Battle of the Week between Cigna and Humana. In order to mark the occasion we ran the same benchmark that we ran a year ago in order to see how much has changed for the brands.

From all of us here at UserZoom, we want to quickly say thank you to our readers for following our battles and we look forward to even more of your favorite brands battling it out in the future. Now, without further adieu, let’s get ready to rumble!

The Task-Based UX Benchmark Study

UserZoom ran a quick task-based UX benchmark study between Cigna and Humana, two well-known healthcare providers, to compare the experience of getting a quote for dental insurance online.

  • We ran the unmoderated remote task-based benchmark study with 60 in-the-wild users on their own devices over the course of a single day
  • We split participants equally between each website so that 30 went to Cigna and 30 went to Humana
  • The Online Task: You’re shopping for a new preventative dental plan and want to know how much your monthly premium will be. From the homepage, get a monthly premium quote for the Preventative dental plan in the 94111 area code.
  • We validated the tasks via a validation question about the monthly cost

conclusion anchor button

Let’s Get Ready to Rumble!

We showed all participants an image of Cigna’s and Humana’s homepage and asked them to choose which site they associated with several UX attributes.

Cigna v Humana homepage

Cigna v Humana II - homepage results

What’s interesting is that in this case both are fairly similar in regards to scroll length, and their footers and headers are almost identical in terms of organization of information. So why the difference in opinions? What appears to be the main differentiator for participants are the hero images. Cigna’s, which has several smaller images and 3 different sections, was seen by participants as being more Welcoming and Lively, but also more Overwhelming.

Screenshot Click Test

We split the participants evenly and asked where would they click to learn more about Cigna’s/Humana’s mobile app.

Cigna

Cigna heatmap II

76% of users correctly clicked on Mobile Apps just above the footer.

Humana

humana heatmap II

There was far less consensus on Humana’s homepage where they should click to learn more about the mobile app. There was a combined 36% of users who correctly clicked on either the Android or Apple direct download link or the “learn more” link.

Online Task Outcomes

We asked participants to find the monthly cost of preventative dental insurance in the 94111 area code. In order to validate whether users were successful at the task we validated via a question about the monthly cost.

If users chose the correct monthly cost they were labeled as Success.

Non Success meant that a user either Abandoned the task due to difficulties with the website or chose an incorrect amount, which we labeled as Error.

2015

Screen Shot 2016-06-28 at 5.05.23 PM

The first time we ran the study both brands were in a dead heat.

2016

humana v cigna success II

For the task this time around, users were 22% more likely to be successful on Humana’s site.

Success

Since the first time we ran the study we can already see a difference in both sites on overall success rates, but have there been any changes in user efficiency?

humana v cigna efficiency 15 v 16

The good news is that it seems both sites have become slightly more efficient since 2015, with overall times dropping down for successful users on both sites. Cigna saw a more significant decrease in the amount of time users required to successfully complete the task.

Cigna user session

Humana user session

Non Success

Cigna v Humana II non success

We tested with slightly more users this year (60 overall compared to the 52 we tested with originally) but relative to the overall amount of non-successful users, abandons on both sites have gone down since 2015. The users who abandoned on Cigna’s site this year shared the common thread of searching in the wrong areas, or trying to use the search function to no avail (see the Cigna video below). The user who abandoned on Humana’s site this year got sidetracked looking at the different dental plans and abandoned before receiving a quote.

What’s interesting is that for almost all of the errors on both sites participants saw the spouse and dependent fields and immediately added them, despite the task only calling for a single person. This resulted in them receiving a different quote (see the Humana video below).

Cigna user session

Humana user session

Ease of Use

After the tasks we asked all the users to rate how easy or difficult it was to accomplish, with 1 = Very Difficult, 4 = Neutral and 7 = Very Easy.

Screen Shot 2016-06-29 at 1.06.14 PM

Problems & Frustrations

We asked the users which, if any, of the following problems they encountered while on the site.

Cigna

Screen Shot 2016-06-29 at 1.08.26 PM

Users who chose “Other” were asked a follow up question to explain. This is what Cigna’s users said:

  • I couldn’t locate the area code given in the list
  • The purple and yellow colors were not my favorite choice
  • Didn’t like how it included medical as a check box when I was searching in “dental plans”

Humana

Screen Shot 2016-06-29 at 1.12.50 PM

Users who chose “Other” were asked a follow up question to explain. This is what Humana’s users said:

  • The text was slightly larger which made the page load slowly
  • Was nice, but like all medical/healthcare sites, was confusing
  • I just wish that there were less things on the screen, always seems crowded

Brand Perception

We asked participants to rate their perception of the brand before and after their experience with the site.

For the rating scale: 1 = Very Negative, 4 = Neutral, and 7 = Very Positive. We also included the option for participants to say they weren’t familiar with the brand before the task.

Pre-Task Brand Perception

Screen Shot 2016-06-29 at 1.31.02 PM

We didn’t ask participants about their perception of the brand before the task last year, so this is for 2016 only. As you can see, a good portion of participants weren’t familiar with the brands but those who were had an overall positive perception.

Post-Task Brand Perception

How did participants feel about the brands after their interaction with the site? Take a look:

Screen Shot 2016-06-29 at 1.36.18 PM

Cigna’s users were more positive about the brand last year than this year. Humana’s users on the other hand were slightly more positive about the brand this year than they were last year.

Net Promoter Score (NPS)

After participants interacted with the sites we asked them to rate how likely it was they would recommend them to friends, family or colleagues.

Screen Shot 2016-06-29 at 1.50.51 PM

Both sites actually saw a decrease in their overall NPS in 2016 amongst these particular users.

Conclusion

Winner - Humana

The winner in this week’s usability battle, for the second year running, is Humana! Not only did they have a higher task success rate with fewer abandons, they also had fewer reported problems & frustrations, and higher scores in terms of ease of use and net promoter score. 

This was a great benchmark to run, and to be able to look back at last year’s results and see how the results have changed (and how much these battles have grown) in the interim was insightful. We once more want to thank our readers for following our battles, and we’ll see you on the digital battlefield soon.


Want to learn more about competitive
UX Benchmarking with UserZoom?

personalized demo