Understanding how your site compares to your competitors’ is critical to winning customers, increasing conversion rates, and providing an impactful user experience that boosts your site’s credibility and loyalty to your brand. Today, customers are much more likely to buy from companies who provide better online user experiences than their competitors. This is true of consumers as well as for B2B. And it’s also now true of both desktop AND mobile experiences. Your site’s user experience is a powerful representation how your company values its customers. It’s no secret to the UX world that customers who feel confident while engaging with a site are more likely to purchase and/or engage with your company both at their initial site visit and over time.

Bottom line – whether your company is already leading-edge or just starting out, it’s crucial to study how users interact with your own site as well as your competitors’ in order to obtain and retain a competitive advantage.

Screen Shot 2015-03-13 at 1.46.23 PM

Why conduct competitor testing

  • Gain market intelligence and domain knowledge

Every company should be making an effort to gather key insights about competitors’ site and product experiences. This is often not a high priority in many organizations, and it can seem like a daunting and time-consuming effort. But remote research tools such as UserZoom can make it easy and cost-efficient.

  • Identify strengths and weaknesses of your user experience

By testing your site against competitor sites, you can gain deeper insights into your own competitive advantages, as well as your weaknesses. You can identify functional areas on your site where you exceed your user’s expectations, as well as areas that need improvement.

  • Develop new insights, inspiration and best practices

By learning how users interact with your competitors, you’ll discover best practices within competitors’ user experiences that are key to driving their success. You can stay on top of new or enhanced site features that have the potential to increase conversion rates or revenue, and incorporate them into your own future releases. 

What type of competitive data to collect and measure

Of course, actually collecting and evaluating information from testing your competitors’ experiences is the key to success. You could opt for a one-time competitive evaluation exercise, but benchmarking competitive information over time will yield the most beneficial results, and it will keep your organization engaged in monitoring the competitive space.

What is benchmarking?

Establishing competitive performance metrics based on usability Key Performance Indicators (KPIs)

  • Your team should determine the keys (or KPIs) to a successful user experience on your site. This can include data automatically collected from user interactions with the site such as efficiency – the amount of clicks, page-views and/or time for a user to complete a task. Measuring efficiency against your competitors is a key step in beginning your competitive analysis.


  • You can also gather effectiveness metrics, which indicate how successful users are in completing the goal they set out to accomplish on your site. Then measure against the same user goals on competitor sites. Effectiveness can be gathered by way of data tracking a users’ path and their arrival on specific page(s) on a site. Success could also be self-reported via questions posed to users.


  • You may already be using ease-of-use and satisfaction metrics for your site via surveys or SUS (Satisfaction Usability Scale). By collecting these same metrics from a study where users engage with competitor sites, they become valuable benchmark metrics.
  • NPS (Net Promoter Score) measures the likelihood of users to recommend a website to a friend or colleague. It’s a popular metric with many companies.   When you conduct user research on competitive sites, consider measuring NPS for your site as well as competitor sites.

Screen Shot 2015-03-12 at 12.32.36 PM

Screen Shot 2015-03-12 at 12.32.45 PM

  • Another interesting metric for usability studies is to measure brand perception before and after users engage in certain tasks on a site. You’ll be able to understand how positively or negatively you or your competitors’ brands are perceived after interacting with a site.   This gets to the heart of how user experience impacts brand credibility and loyalty.


When to collect and report competitive benchmarking data

No matter how many of the above competitive metrics you decide to collect, the essence of benchmarking is to measure data over time. The frequency at which you collect and report competitive benchmarking data can be based on any or all of the following:

  • The speed at which your industry moves. However frequent change is introduced to customers. For fast-moving industries this may be monthly, for others it may be much longer term.
  • Typical site or product release cadences. You will want to repeat your benchmark studies as often as major changes typically occur on sites and/or products that users engage with (either within your company or as a general industry practice).
  • Internal metrics reporting timelines. If your company has regular timeframes for reporting key business metrics (for example, quarterly) you may want to align your collection and reporting of competitive benchmark data.
  • User research roadmaps. If your company has a roadmap and a regular cadence for user research (usability tests, surveys, customer interviews, etc.), CONGRATULATIONS! This should make it fairly easy to layer competitive benchmark studies onto your existing roadmap.

How to collect competitive benchmarking metrics

You may be wondering exactly how to set up usability studies or other user research that will facilitate collecting competitive data and KPIs. Here are a few tactics to consider:

  • A popular and recommended approach is to conduct regular usability studies on both your site and competitors’ sites and gather feedback from users in-the-moment as they are interacting with each site.
  • Consider the number of metrics you want to collect, and the types of corresponding tasks you want users to attempt during a usability study. Then consider how long it may take a single user to perform those tasks on multiple websites (yours and competitors’). Asking a single user to perform multiple tasks on multiple sites during a single usability test session may be overwhelming and cause users to become fatigued, confused or ambivalent. You may consider designing studies where each user tests only one site, and you then compare the results across users (in the research world, this is known as between-subjects design).
  • Although by definition benchmarking focuses on quantitative data, you’ll want to supplement your benchmark metrics with qualitative data as well. This is easily accomplished by using tools such as UserZoom that allow you to record usability test sessions including screen recordings and audio & video feedback from users as they complete tasks. Use video footage to show your stakeholders examples of pain points on your site or competitors’ sites.
  • It’s worth highlighting that UserZoom allows you to facilitate remote usability testing with large numbers of users (e.g. in the hundreds) in order to obtain statistically relevant benchmarking data. Studies utilizing remote users can be conducted in a matter of days, allowing you to collect and share data quickly and regularly.
  • Finally, by conducting studies in automated remote usability testing platforms, you are easily able to copy & repeat studies as frequently as needed. A perfect match for benchmarking purposes.

Competitive benchmarking is about letting real users help you reveal pain points on your own site and competitors’ sites. When you benchmark KPIs on your site vs. competitors over time, you keep the competition in view and benefit from data and inspirations that can drive future enhancements to your site – ultimately leading to a stellar user experience.

Interested in seeing a few examples of Competitive UX Benchmarking with UserZoom?
Check out some studies here:

Retail UX Benchmark Study: Measuring the Online Experience of Retailers on Tablet Devices

Auto Insurance Benchmark Study: Measuring the Online Experience of Auto Insurance Companies

About the Author

Toni Allen, UX Research Manager at UserZoom


Toni has over 20 years of experience in roles focused on UX, User Research, Product Management and Marketing. She was formerly a Sr. Product Manager for consumer experiences at HP. In recent years she has specialized in unmoderated remote usability testing and user research for both desktop and mobile, and has helped hundreds of companies realize the benefits of using remote testing platforms to gain powerful insights quickly and cost-effectively.