Great user experience matters for digital experiences across all industries yet many websites catering to the health insurance market fall short.

In this UX benchmarking report we’re comparing four such healthcare websites across multiple US states, based on their usability, trust & credibility, loyalty, appearance and task success.

For a deeper dive into the results, you can join Kuldeep Kelkar, Vice President, User Experience Consulting & Professional Services, and Dana Bishop, Senior User Experience Researcher, as they discuss the complete UX Benchmarking Study via our on-demand webinar: Competitive UX Benchmarking: How Four Healthcare Insurance Sites Scored Across the Board.

In the meantime, here are four of the key highlights from the benchmark.


First of all, let’s answer why you might run a UX benchmarking study. With this kind of project you can look at:

  • How your site performs relative to your competitors
  • Interpret your usability standing, feature set and more within your industry
  • Learning from their success and failures, what works or does not for your competitors
  • Industry best-in-class examples to emulate
  • Benchmark with other industries

In order to score the sites, we combine various measurements, collecting both behavioural data (such as task success, task time, page views) and attitudinal data (such as ease of use, trust and appearance) to create a single benchmarking score for each website.

We then conducted two studies, one quantitative, one qualitative. Our quantitative study had a sample size of 200 for the screen recording behavioural test, while the qualitative study with a sample size of 20 for a think-out-loud test.

For more in-depth information on the methodology and how we create the benchmark score, check out the webinar.

Four Blue Cross Blue Shield (BCBS) websites were tested: Texas, New Jersey, Massachusetts and California. Most insurance companies are regulated at state-level, which means that there are certain intricacies dependant on which state the insurance is being sold. Even though the four selected sites are BCBS sites, as you can see they look quite different.

They have different features, functions, aesthetic design and how a user finds a doctor, or other healthcare information is different on each site too. So we wanted to find out which design works the best for the user…

The winners and losers

BCBS Texas was the overall winner, scoring 76%. It offered the best overall experience based on our benchmark score, beating California (63%) New Jersey (56%) and Massachusetts (48%).

Texas is also notable for being the only website of the four tested to be granted ‘good’ status, the others ranged from average to poor. However Texas only just scrapped above average, so it’s worth noting there’s room for improvement.

Pre and post brand perception

We also looked at shifts in overall perception of each website, pre and post study.

BCBS Massachusetts saw the biggest drop in brand perception after three tasks on site (roughly 15-20 minutes into the study), dropping from 75% to 40%. New Jersey and California both dropped 18 and 16 percentage points respectively, while Texas was the only site to improve slightly (68% to 70%)/

It’s important to measure perception as it really shows you how usability issues with a website can alter the overall feeling a user has towards a brand. In the case of Massachusetts – very negatively.

Now that we can see there’s a deficit between the promise of a site and the realities of using it – now we need to find out why that’s happening.

The need for multiple KPIs

One of our tasks asked users to compare and select a healthcare plan based on some parameters we gave them. In this case our stud asked…

“Assume that you lost your job and need to find healthcare insurance for you and your spouse. You’re both generally healthy but do get sick a few times a year and will occasionally visit specialists as needed. Compare healthcare plans and select one that will meet your needs.”

Digging into the data, California stands out as an interesting case of not taking the first KPI as a win. As you can see from the table, California had a good Task Success rate (how many people completed the task) which almost gave our winners Texas a run for its money.

However if you look at the next two KPIs, the users took twice as long and had to navigate double the amount of pages to get to task completion than they did with Texas.

That’s not to say that Texas is the golden child. Looking at the attitudinal data, we asked our users what problems or frustrations they encountered during the task and for Texas, 62% of our users encountered one or more frustrations. 28% weren’t sure which plan was best for them, 24% found it difficult to find plan details and 20% found it difficult to differentiate between plans.

As for Massachusetts, we saw similar issues but in much greater numbers. 40% found it difficult to compare plans, 36% weren’t sure which type of plan to choose, 34% weren’t sure which plan was best for them.

One of our participants stated, “I’m not sure what all these things are. There’s a lot of things packed into a small space.”

For a website where the primary function is to present its visitors with clear information about it’s different plans, this isn’t a good revelation.


Massachusetts may have scored the lowest, but it doesn’t mean other sites don’t have issues.

Compared to the other sites, the New Jersey Horizon Blue site required more personal data to find plan information. 20% didn’t understand why so much personal information was required.

Comments from the think-out-loud study included:

  • “The process was too lengthy”
  • “The compare plans page gives NO information about what the plans actually entail. In order to have more comprehensive service, I had to fill out an actual application including email address and personal data…”
  • “Entering information can be overwhelming. I wish the process in getting a quote walked me through the information was a bit simpler”
  • “No way to get other information without entering a lot of it first”

For more detail on the above studies, and for the findings from the further studies we ran on the sites, check out our webinar, available to watch on demand: Competitive UX Benchmarking: How Four Healthcare Insurance Sites Scored Across the Board.