What is UX benchmarking and why is it so valuable?
To celebrate the launch of our brand new UX benchmarking 101 ebook, here’s our guide to why you should care about this awesome, highly persuasive methodology.
Anyone who works in user research knows their work can uncover an array of user insights which can inform business and design decisions, and ultimately help create a better experience for users. Hooray! You’re the best. Pat on the back for you and your significant UX spirit animal. Ours is a great-horned owl.
However, it can be difficult to quantify anecdotal experience, especially if you’re mainly running moderated tests and getting qualitative insight (i.e. people telling you why your product is rubbish or great).
But how can you communicate this data effectively and efficiently to stakeholders? How do you tie an ‘improved user experience’ to improvements in your company’s overall key performance indicators (KPIs)? You worked hard for those insights, dammit, and you should be rewarded!
And even more importantly, how can you measure user experience improvements over time and therefore justify more testing to make your product even more awesome and user-friendly?
One way is through UX benchmarking!!!
(You probably guessed this already, because you looked at the headline.)
Quantifying the impact of UX research through benchmarking can help prove the value of research, tie user experience to business objectives and engage stakeholders.
BUT WAIT!!! How can you benchmark experience? Aren’t experiences subjective? How can you place a value on the subjective? This is the stuff that made Descartes suffer from ‘exploding head syndrome’.
Don’t worry – yes, subjectivity is a challenge, and as a result so is identifying and tracking meaningful metrics. But if you implement a valid benchmarking strategy, and select the right metrics to track, you’ll be well on your way to benchmarking glory and your head will remain relatively intact.
What to know how it all works? Step right this way…
The following article is adapted from our brand new ebook, UX Benchmarking 101, a free-to-download, 42 page guide to running both longitudinal and competitive benchmarking.
What is UX benchmarking?
Benchmarking creates a baseline for understanding the current user experience on your website, app or any digital product. And the sooner you get started, the sooner you’ll be able to measure and improve the user experience. You may also hear this referred to as longitudinal benchmarking – which is basically measuring over time.
When you start benchmarking you need to create a baseline. A starting point. Otherwise you’ll have nothing to measure against.
You won’t know how fast you can run 100 metres until you actually put on a pair of running shoes and persuade a confused friend to measure the distance and time for you. The same is true for your website, app or other digital product.
Just pick a feature or tool you want to measure and improve on your website or app – let’s say it’s the checkout section on your retail site. You then run some testing to get some baseline usability metrics.
We discuss methods of testing and metrics in the UX benchmarking ebook, but for this example let’s just say we ran a basic usability test where we timed how long it took a group of 20 test participants to find a specific item and purchase it through checkout. We’ll also look at average number of pages they visited, and whether the task was successful.
You’d then input these figures in a table and then you will have your baseline data to begin tracking this particular user experience over time.
Perhaps within this initial test you got some in-the-moment feedback from your participants where they mention feeling “overwhelmed by so many options during the checkout.” You could then make some changes to the user experience based on this research (for example, simplifying or removing
some options), then you run another round of testing and see how you perform against your baseline.
REMEMBER: keep all the details of the test study the same for each round of testing otherwise it won’t be an accurate benchmark. For instance – the item being purchased, the device/browser being used, the number of test participants, the type of task and any questions you might ask.
You then start to build a picture like this (which is essentially a scorecard, and you can learn all about UserZoom’s UX scorecard program here).
As you make changes and run each round of testing, you’ll begin to see problem areas for improvement. You’ll then see exactly how your improvements are affecting these metrics, and if you tie these metrics to wider company KPIs (improve revenue, improve ease of use, improve customer satisfaction) your user research can be seen as an invaluable tool in the organization.
What can you UX benchmark?
So you know you want to benchmark something, but what exactly can you benchmark?
- Your website, app or any other digital product
- Any of the above versus those of your competitors’
- Different iterations of prototypes through development
- Different versions of your live website or app
- A prototype of a feature versus a live feature of a competitor
- User journeys from one point on your website to another
Or a combination of the above – basically anywhere your users and customers are accessing your site or product online can and should be UX benchmarked.
This applies to external sites as well as internal sites or intranets, because you want to ensure your online properties are as streamlined and optimized for your internal users as they are for your customers. Just imagine if Amazon had the same UX as your employee expenses portal? *Shudder*
When and how often should you benchmark?
Really you can can begin any time from concept, to live working product, to every iteration that follows. But basically… start NOW!
As for how often?
We work with many different customers who have different intervals in which they run their benchmarking. Sometimes it’s annually, sometime’s biannually and sometimes quarterly or even monthly. It really depends on your development cycle and how often you’re rolling out design changes.
We would also recommend testing before or after launching a major update or redesign that may not fall within your regular intervals, as you don’t want to find that something isn’t working six months after launch, because that’s when your timetable says you’re due another round of tests.
Why is UX benchmarking so valuable?
Why would you undertake UX benchmarking in the first place? Here’s our executive-friendly guide to why UX benchmarking is so rad.
Engage stakeholders and secure buy-in
While video-based qualitative insights are compelling, you may struggle to get buy-in from more data-driven stakeholders. Therefore you may get increased stakeholder buy-in when you run studies specifically with benchmarking in mind, because the higher sample sizes increase statistical significance, and this kind of robust data brings much more confidence.
Identify product strengths and weaknesses
Benchmarking across your key customer and user journeys on your digital products enables you to understand specifically where your strengths or weaknesses lie. This can help you to work on reinforcing your competitive advantages with further developments or changes to your messaging.
Inform better design decisions
Making changes to a digital product without measuring the outcomes quantitatively (i.e. numerically) means that you are operating in a knowledge void. If changes are made without the context of knowing if they’re making an experience better or worse, changes become reactive and based on gut instinct. Dirty Harry operated purely on gut instinct, and although he got results, you’re not Dirty Harry. I’m sorry you had to hear it from this UX blog first.
Validate designs against business objectives
Benchmarking key user journeys on your digital products prior to and after a significant change will enable you demonstrate the impact of user research on the most important business KPIs, such as conversion rates, basket abandonment or even the bottom line.
Demonstrate the positive impact of your efforts
Adding quantitative, measurable metrics to your UX research not only helps support your findings in the moment, it will also help you understand trends and provide context to design and business decisions in the longer term.
– – – – –
Of course if you REALLY want to knock your stakeholders’ tiny weird trainer socks off, you could appeal to their more competitive nature and show them EXACTLY how you compare to your fiercest rivals! You can find out everything you need to run competitive UX benchmarking by downloading our complete 42 page UX benchmarking 101 ebook…
Christopher is the Content Marketing Manager for EMEA, which basically means the skipper of the good ship ‘UserZoom blog’. So far his requests for changing its name to the ‘USS-erzoom Blog’ have been rightfully denied. In his spare time, Christopher is a filmmaker and the editor of wayward pop culture site Methods Unsound. He used to be the deputy editor of Econsultancy, editor of Search Engine Watch, staff writer for ClickZ and features editor of CMO.com.