The Task-Based UX Benchmark Study

UserZoom ran a quick competitive UX benchmark study between Coach New York and Kate Spade New York, two well known American fashion design companies, to see how users experienced navigational and core tasks as well as how they rate their overall experience on these websites.

  • We ran an unmoderated remote task-based benchmark study with 40 in-the-wild users on their own devices over the course of a single day
  • Participants were equally divided between the companies’ US sites so that 20 went to www.coach.com and 20 went to www.katespade.com
  • They completed several tasks while on the site: a screenshot click test, a tree test on their main menu and a task-based test, where we asked them to complete a task on their sizing information
  • We also measured their brand perception and how they rated their overall experience

conclusion anchor button

First impressions are lasting impressions

Contenders

For starters we wanted to know the users’ first impression to both of the contenders’ sites. We showed all 40 participants images of the Coach and Kate Spade homepages and asked them to choose which site they associated with several UX attributes.

UX Attributes

 

There is no doubt that this round goes to Kate Spade. Kate Spade NY was the most related to the terms Organized, Easy, Welcoming, Trustworthy, Lively, Informational and Helpful. Coach’s homepage only got identified by most with the term Overwhelming.

Round one goes to Kate Spade!

Screenshot Click Test

To understand if visitors were able to easily find important information, we showed them each an image of the assigned contender’s homepage. Based on that image, we asked them to complete a simple task: “Where would you click if you were looking to get email updates from the brand?“.

Heatmap

85% of the Coach homepage visitors succeeded on this test by clicking on the one option the site offers for registering to the newsletter in the bottom of the page. The rest of the participants clicked on other parts of the homepage: 10% on the ‘Contact Us” option of the “Customer Care” section and 5% clicked somewhere else.

Heatmap

The Kate Spade site also provided visitors only one option to join their newsletter found on the bottom of the homepage and all of its participants clicked there!

With a 100% success, Round 2 goes to Kate Spade!

Tree Test

Good information architecture is necessary for providing a good search experience. To evaluate the findability, labeling and organization of the contending sites’ structures we placed participants in a plausible scenario: “Find this season’s new arrivals for handbags.

Participants were shown the site’s menu structure as a list of topics and asked to select the area where they think they would find the latest handbags for the fall season. Once finished they would mark the task as completed and if they reached the requested info the task would be rated as Success.

Tree test

Both sites have dedicated menu options where handbags can be found.  The Coach site keeps its new arrivals under each item section, while the Kate Spade site has a separated category called “New” where all the recent items are found under their own category.

On the Coach site 90% of the participants chose the correct option: “Bags > Women > New Arrivals” while the rest of the participants either chose “World of Coach” or “Bags > Men > New Arrivals” which do not include the new arrivals on handbags.

Tree test

On the Kate Spade site 55% of the participants chose the correct options: “New > Handbags” the rest of the participants, almost half of them, chose the wrong section “Handbags > View All” option.

Round 3 goes to Coach!

Online Task-Based Test

To help their customers find the perfect garment size they need, fashion brands provide their very own sizing guides where body measurements and measuring tips are shown so that customers can correctly make their choices.

For this task we asked participants to pretend they were purchasing a top for a woman with a bust size of 35.5 inches.

To evaluate the Success of the task, a validation questionnaire was used. There was only one right answer, size 6, so participants would only succeed if they did correctly reach the sizing chart and review the measurements and sizes.

A Non-Success result meant that the participant either Abandoned the task or did not correctly answer the validation question. In which case, that result would be labeled as an ‘Error’.

So, how did the two challengers do?

screen-shot-2016-10-27-at-3-19-08-pm

Results show that a lot more participants were able to find the requested item on the Kate Spade site.

Some participants had a hard time finding out what size they should choose. 67% of the participants chose the wrong size while 33% decided to abandon the task, giving the following reasons:

  • I searched for over 15 minutes and still could not get anywhere that displayed sizes for tops. I was able to get colors, materials but I just could not get Tops sizes and after 15 minutes searching, am pretty much convinced this information is not available at this website. I checked site map, search area, filter area, men’s tops area, women’s tops area, man, if this information is there, well, it’s not as obvious to see ass other information. As a last resort I would have just contacted one of the stores for support.” (Coach)

 

  • “It was too difficult…sizes weren’t shown in this type of measurement.” (Kate Spade)

 

Ease of Use

After the task we asked all the users who were able to complete the task successfully to rate how easy or difficult it was to accomplish, with 1 = Very Difficult, 4 = Neutral and 7 = Very Easy. This is what they said:

ease of use

Results show that more participants were able to complete the task on the Kate Spade site, and found it easier to complete there too, which is why round 4 goes to Kate Spade!

Problems & Frustrations

After the study, we asked all participants if they encountered any problems or frustrations and their responses are here:

The analysis shows that more visitors to the Kate Spade site seem to be happy with the experience they had. 60% of the Kate Spade participants selected the “I did not encounter any problems and frustrations”, while only 25% of the participants on the Coach site chose that same option.

In general, Coach visitors complained the most about having to make too many clicks to find information and the navigation being confusing, while the main complaint on the Kate Spade site was about having to make too many clicks to find information.

Brand Perception

Now, let’s talk about perceptions and find out what participants were really thinking about their experience. We asked participants to rate the perception they had of the challenging brands before and after their experience on sites.

For the rating scale: 1 = Very Negative, 4 = Neutral, and 7 = Very Positive. In the pre-test survey we also included the option for participants to say they’re not familiar with the brand.

Before participants interacted with the websites, Coach had a brand perception rating of 5.2 and Kate Spade a 6.1 out of 7. Coach had 10% of users saying they were not familiar with the brand while 15% of the Kate Spade participants said they were unfamiliar with the brand.

After the participants completed the different tasks, we asked if their brand perception changed. For Coach, brand perception fell 0.8 points while the Kate Spade brand perception only decreased 0.3 points.

Net Promoter Score (NPS)

As the final part of the study, once the participants have interacted with the sites, we asked everyone to rate how likely it was they would recommend the brands’ sites to friends, family or colleagues.

The results show that even though both brands have a negative NPS, more people would be comfortable recommending Kate Spade based on their experience with their homepage.

And the winner is…

Winner

This week’s battle was a tough one! Coach had okay results but Kate Spade managed to keep on top in almost every task. With better results and with more positive visitors’ opinions, there is no doubt Kate Spade is providing a better digital experience to its visitors.