The Task-Based UX Benchmark Study

UserZoom ran a quick competitive UX benchmark study between Magento and Shopify, two software companies that enable retailers to easily create an ecommerce platform, to see how users experienced navigational and core tasks, as well as how they rated their overall experience on these websites.

  • We ran an unmoderated remote task-based benchmark study with 60 in-the-wild users on their own devices over the course of a single day
  • Participants were equally divided between the sites so that 30 went to each
  • They completed several tasks while on the site: a screenshot click test, a screenshot timeout test, a tree test, and a task-based test where we asked them to find the page where they could learn about how other companies found success using their service
  • We also measured ease of use, brand perception, NPS and how users rated their overall experience

Here is a sample of the UX benchmark.

First impressions are lasting impressions

We showed all 60 participants static images of the homepages of Magento and Shopify and asked them to choose which site they associated with several UX attributes.

Right off the bat you can see that Shopify got more votes in every positive category. It’s worth noting that both companies are fairly close to one another in their ratings except for Informational and Overwhelming, in which case there is a more clearly defined “winner” and “loser.”

It’s also one of the few times in our UX Battles of the Week history where Informational and Overwhelming weren’t directly correlated.

Want to see the rest? Download the battle to find out who won!


Show me who won!