The value of eye-tracking in research

Andrew Schall is the Senior Director of Experience Design Research at Mayo Clinic, where he’s working with his team to create the next generation of healthcare technology. He has over 20 years of professional experience in user experience with an expertise in eye tracking. This episode we hear how and when a UX Researcher could use eye tracking technology.

Transcript

Alfonso de la Nuez: Welcome to UXpeditious! A show that brings you quick, insightful interviews with design, product, and UX leaders.

Dana Bishop: In each interview we dive into how UX research impacts user insight; shaping the design and business strategy of some of our favorite tech tools and products.

Alfonso de la Nuez: I’m Alfonso de la Nuez, Co-CEO and Co-Founder of UserZoom.

Dana Bishop: and I’m Dana Bishop, VP of Strategic Research Partners at UserZoom.

Alfonso de la Nuez: And we are your hosts. Joining us today is Andrew Schall, Senior Director of Experience Design Research at Mayo Clinic and a leader in the UX field. He’s also a faculty member at the Maryland Institute College of Art, teaching Human-Computer Interaction to graduate students.

Dana Bishop: In this episode, we’re talking about eye tracking. Andrew is an expert on the topic. He’s Co-author of the book “Eye Tracking in User Experience Design” and more recently he contributed a chapter on the topic in the 3rd edition of 'Measuring the User Experience.'

Alfonso de la Nuez: Andrew. So thank you so much for joining us today. I know that you've written extensively on the UX research method of eye tracking. Can you give us a quick explanation of what eye tracking is.

Andrew Schall: Eye tracking is a really exciting technology that really allows us to see things from the user's perspective quite literally. I mean, we're able to actually track very accurately where a person is looking, and that can be looking at something that's on their screen, on their mobile phone, or actually these days, even in their physical environment.

It's a really powerful tool for those who are trying to understand things like visual hierarchy or visual weight or areas where as designers who really want to make sure that our designs are actually usable. And so it really aids in helping make sure that the designs are actually helping people focus on the things that they need to focus on to get things done.

Lots of different applications that hopefully we can get into some of them today, but really exciting technology that's actually been around for quite a while, and I think has been gaining traction steadily over the years in user experience.

Dana Bishop: That's what I was wondering. So in terms of UX researchers, when might a UX researcher employ this method and why?

Andrew Schall: Yeah. I would say it's not for use in every situation. I think a lot of researchers who haven't done eye tracking are not really sure when is the best time to apply it. So I would say it is something that you want to apply much later in the design process. So where we do a lot of upfront research and quite a lot of iterative research during wire framing or early prototypes where the design is still really forming, it's not really the best time to do eye tracking then because your eyes are... It is so amazingly easy to influence where someone's eyes look.

So if your designer changes the color of a button at some point or you make something bigger or you move something from here to there in your interface design, it really can dramatically change where they're looking. And so it really gets the most value when we have a design that is fairly well baked. We're really using to evaluate where the hierarchy is with that design.

Dana Bishop: Interesting. Also curious about if I can follow up with the cost of doing eye tracking and the ROI of that process.

Andrew Schall: Yeah. I think that's an important one because eye tracking is not cheap. It has come down a lot though over the years. I would also say that when we talk about eye tracking, there's what I call the traditional eye tracker, which is a physical hardware device, and in some cases that's something that fits onto a computer monitor. In some cases it's actually eyewear. So it's actually glasses that are used. And some of those can be expensive, but there's really a whole new generation of eye tracking that's even going beyond hardware devices where you can actually use a webcam to do eye tracking now. And so that technology is getting better every day.

And I think the ROI though really is getting data and insights that we just can't get through traditional usability testing or other research means. You're getting a much deeper understanding of users behavior that there's really no equivalent, in my experience. So you're getting that extra value through these deeper insights that can't get otherwise.

Alfonso de la Nuez: So let's talk a little bit about the three types of visualizations, gaze plots, which is qualitative, gaze replays, also qualitative, and heat maps, more quantitative. Can you tell us about these three and maybe explain to people what each is?

Andrew Schall: Yeah. Well, they're all basically based on the same information, which is basically a coordinate system of where a person looked at any given time, which is basically a fixation is a dot, if you will, on the screen. A lot of these things are qualitative in terms of how we use it.

We're trying to understand the general patterns of eye movement behaviors that help us understand what people pay attention to, what was that path that their eyes took. And we look at that both on the individual basis, so each person's individual scan path or gaze plot diagram, and usually that's contextual. So it's in the context of what were they trying to find, what were they trying to do.

And then we have aggregated, more quantitative things like heat maps that we might combine 50 participants' eye gazes together and see where the areas of attention were, where those hotter areas were, where there's a lot of people that looked.

Honestly, I find the most valuable is actually the eye gaze replay, basically the playback of the eyes, because I can see what the person was trying to do as they're doing it. I can see where their eyes are going. And so it brings everything together of understanding with eye tracking being kind of a layer, if you will, on top of all the other data we typically collect. It gives me those extra understanding of what those behaviors are.

Dana Bishop: I hear you talking about it, it sounds very technical, but I also know that it's also very relatable. How do you feel that is when you're reporting back to stakeholders and executives about your findings? What is that experience like for someone who's maybe not an eye tracking expert?

Andrew Schall: That's a really good point. So yeah, I mean, eye tracking has a lot of values beyond just the data itself. I think it's very visual and it's fascinating. And I think that it ends up bringing a lot of the stakeholders that we want engaged in user research to observe. Certainly it gets their attention. It gets them engaged. And so obviously there's huge benefit to that alone.

And I think also what they really enjoy is there's a real-time aspect to this. So usually they're waiting to get the results from a usability test. Here I can watch live and I can literally see where the eyeballs are going. And especially if you are a designer and you just created that page, there's nothing like seeing a designer's reaction to seeing literally where a person is looking in their design. I think it helps connect them a lot more to the research by literally seeing it happen.

Dana Bishop: It also settles a lot of arguments quickly. Right? So button placement, button color, it becomes pretty apparent which one's more effective.

Andrew Schall: Yeah. I mean, it's very objective. And honestly, though, a lot of the times, the interest is not just where the eyes look. It's actually where they didn't look a lot of the times. So it's like, "Why didn't they look at that? Why didn't they notice that?" I want to point out though, that eye tracking is powerful, but I always say it's not a mind-reading device. I don't know why they didn't look at that. Or why did they spend so much time looking at this particular element? So we still need the traditional, old-fashioned, think aloud interviews. You still need the why behind the what, and eye tracking is mostly the what.

Alfonso de la Nuez: And the point that we were making earlier, they're both very visual. So a lot of people out there are familiar with the heat maps coming from analytics versus the heat maps that come from eye tracking, which are completely different, obviously.

Andrew Schall: I think there is value in click data, for sure. There's no question. But it really is a tiny percentage of what's actually happening because there's thousands of eye movements before you decided ultimately where to click with your mouse. So you're really missing a lot of the picture. So but they can work well together, though. It's very complementary. Where did they look which ultimately drove them to what they did?

Dana Bishop: Can you talk a little bit about the difference between eye tracking for desktop computers versus smartphones? These days we've always got our phone in our hand. We're walking, we're driving, we're shopping, we're doing this and that when we're on our mobile device. I'd imagine there's some differences as far as how people interact, behaviors. Do you use a different method? Do you look at the data differently?

Andrew Schall: Mobile eye tracking is definitely a different animal from desktop or laptop testing, for sure. It really relies on different technology. So when we do mobile-based studies or tablet-based studies, we're really using the eye tracking glasses for that. And the glasses are really more of a go-to tool these days, honestly, because we do so much mobile research. They are much better though for qualitative studies, I will say.

So for those who are really interested in understanding this generally, what are people looking at in the context of ordering a latte or... Actually, my team was doing research last week for a wayfinding project.

So Mayo Clinic has developed a wayfinding app to help you navigate the very complex hospital campus they have. So it's basically a combination of using the glasses to understand how are they referencing the app? What are they looking at? And then also what's happening in their 3D world, their physical world as they're navigating? So the glasses are basically capturing everything regardless of whether they're looking at the phone, looking in their environment. And it's also capturing, do they notice the signage?

And so the glasses kind of free us, if you will, from being fixed to a computer monitor. But we still do a lot of the desktop testing. Again, we were also doing content testing, actually that same week. We used a traditional eye tracker that's attached to the monitor so we can actually get really accurate understanding of how people's reading behavior is happening on the computer. So two different eye trackers, two different methods for that.

Dana Bishop: Really wide-ranging. That's interesting. It hadn't occurred to me that you're using the same technology to capture the phone and looking around your environment.

Alfonso de la Nuez: I'm thinking of today's environment, but we always talk about doing different methods depending on the goal and depending on the stage that you're at in the product design and development life cycle or process, and eye tracking may come in not every single week or every sprint, but for some specific, deeper studies on something specific.

Might happen once a month or a couple of times a quarter, whatever. It came to me as it is consuming, but it is also really valuable if done correctly.

Andrew Schall: Right. If done correctly and for the right use. I mean, it's cool technology, and sure, it'd be cool to do eye tracking for everything. But again, going back to that content example. So Mayo Clinic is revising all the content on our website.

So we really, really want to understand fundamentally, "Well, what are people paying attention to with our current content?" We really don't know. So it provides that foundational understanding and a baseline really for us as we basically create a whole new content strategy for the site. So in that case it was worth the time to do the analysis because that data's going to keep being utilized over time.

Dana Bishop: Any quick tips you want to leave on eye tracking, want to leave us with?

Andrew Schall: Oh. Well, I would say, I think of the common questions I get. Sample size is always a big question. So my recommendations would be also again, knowing what you want to get out of the eye tracking. Are you focusing more on the qualitative metrics? You might be able to get away with a smaller sample size.

If you're focusing on quantitative metrics, we're going to be aggregating large numbers of participants, you want a much larger sample. I mean, it could be 40 or 50 participants. So for those who are accustomed to running very small, quick, agile usability tests of 5 to 10 people, you might often need more participants than you typically are used to.

Dana Bishop: Right.

Alfonso de la Nuez: This is just another one of those methodologies that you can use to again, just get to know the user or the customer better. So greatly appreciate it. I learned a lot.

Andrew Schall: My pleasure.

Dana Bishop: Yeah. Fascinating. Really appreciate your time today.

Alfonso de la Nuez: That's Andrew Schall, Senior Director of Experience Design Research at Mayo Clinic.

Dana Bishop: Thanks for listening to UXpeditious. Make sure to continue listening to our new episodes each week for quality insights from UX industry leaders. If you like what you heard, help us out by rating and reviewing the show on your favorite podcast platform.

Alfonso de la Nuez: UXpeditious is produced by UserZoom in partnership with Pod People. Special thanks to our production team: Christopher Ratcliff from UserZoom; and the team at Pod People: Rachael King, Matt Sav, Aimee Machado, Hannah Pedersen, Colleen Pellissier and Jason Mack.