Making rapid research a reality

Devin Harold is the Director of UX Research for Financial Services at Capital One. In this episode, we chat with Devin about UX research processes, specifically rapid research. Devin is an experienced UX leader who has leaned into the rapid research process to scale user insights and drive some big initiatives.

Transcript

ALFONSO DE LA NUEZ:

Welcome to UXpeditious! A show that brings you quick, insightful interviews with design, product, and UX leaders.

DANA BISHOP:

In each interview we dive into how UX research impacts user insights; shaping the design and business strategy of some of our favorite tech tools and products.

ALFONSO:

I’m Alfonso de la Nuez, Chief Visionary Officer and Co-Founder of UserZoom.

DANA:

And I’m Dan Bishop, VP of Strategic Research Partners at UserZoom.

ALFONSO:

And we are your hosts!

Most companies these days are demanding high impact research and user insights. But the truth is most research teams are pretty small and can only focus on the highest of priorities. So, how are researchers scaling their insights and making an impact on the business?

DANA:

One way to scale research impact is by implementing a rapid research program. Today we’re talking with Devin Harold who’s done some deep thinking and action on how, why, and when to start a rapid research program.

ALFONSO:

Hey Devin. How are you doing? Thank you so much for joining us. Can you please start by introducing yourself?

DEVIN HAROLD:

My name is Devin Harold. I'm a UX research director at Capital One. I focus on delivering insightful research for all of our product and design teams within our financial services space.

ALFONSO:

Awesome. Let's talk about rapid research today. Can you please give us a short explanation of what rapid research is to you?

DEVIN:

Yeah, that's a great question. For me, rapid research means delivering an expected, consistent and frequent cadence of insights for teams on a level or maybe even again, that cadence that they're not really used to. So a number of different research methods and approaches take time, takes time to recruit participants, takes time to develop a quality moderation guide, but how do we systematize those things in a way that delivers consistent value over time and starts to be really expected? So the question becomes, what do we systematize and put guardrails around and against. So that way we can do the same thing over and over again. So that's what rapid research means to me.

DANA:

Could you give us additionally a little context around what is in insights index and how are these related to rapid research?

DEVIN:

An insight index. I'm sure people have a lot of different terms for it, whether it be repository of insights an insight index. I see them as kind of one and the same, a knowledge management platform that builds off of the things that you are learning over time to limit insight waste, improve your planning processes, and really just serve as a sounding board for what do we already know before we go and learn something new. How it ties in with rapid research in my mind is because you're on this regular recurring cadence of always-on research, always learning.

There's a lot to be said in terms of the power of codifying the individual insight nuggets that you learn from every rapid study on a week by week or biweekly cadence, plugging that into an insight index and then being able to reference that over a period of time. Let's say every month or every quarter to pull up deeper level of insight and understanding of your customers.

Insights are the input. And then that allows you to meta analyze, scrub insights that you have. If you are lucky enough to have an archivist like Amazon is, you can actually have someone who's full-time looking into the insights that you're delivering across the business to deliver really insightful outputs in terms of here's what we know about our customers.

DANA:

I love the phrase you used, reducing insights waste. That's a great concept or goal in that, so I love that. What are the business and strategic values of implementing a rapid research program in your view?

DEVIN:

Well, I think speed and efficiency are the softer values, but that actually leads to a whole lot more downstream. When I built the rapid research program team at Verizon, we saw an almost double amount of research that we were doing as a team. Our bandwidth had drastically increased. What that enabled us to do was not only to increase bandwidth of research that we're running as an organization, which gets ahead of these recurring themes. Let's talk about insight waste, like you mentioned.

One of the things they say is it cost 10 times as much to fix a bug in development and a hundred times as much to fix it in production. Well, think about all the bugs that we fixed by doubling our bandwidth as a team. Now also think about how much more we learned because we've freed up time from our lead researchers' time.

If I'm a principal researcher and I've been in the game for 15 years, I have a lot more to offer than just always doing small usability tests. Now, maybe that's where I really love to shine. But, from my experience in working with larger research teams, they really love to spend most of the time on the grayest areas, the funkiest problems, mixed-methods, and to say, "Hey, you don't have to worry about some of the evaluative or iterative work anymore. We got to program that can accommodate that."

It allows us to get past the local maximum. We are shooting for one mountain in front of us, but if we free up our time, we can actually see on a broader horizon and we can be able to reach new heights that we never thought possible before because we freed the bandwidth from our lead researchers to do more strategic and foundational work. So those are two areas of business impact, I could see a rapid research team really helping to drive

ALFONSO:

So in creating a rapid research initiative, what do you consider to make sure that it has that ROI that management typically is asking for?

DEVIN:

Yeah. I think if you can take that idea of the bugs that we talked about getting ahead of what usability issues you might be able to get ahead of with a rapid research program and put that into a business case, by saying, we need a rapid research program that requires this amount of cost, but this is how much savings that we will get. So for specific lines of business, maybe they care about driving revenue. Maybe they care about decreasing the call ins, call-in rate reduction. There's a cost associated with that. Usually that's not too difficult to get a hand of. Average basket size or maybe the average time it takes for a customer to be on a phone call with a technician.

If you could get a hold of those numbers and just apply it to that same 10 and hundred rule in terms of the usability issues you could fix in advance, you can roughly size based on how much research my team is doing right now. If we could double our bandwidth in the research we're doing right now, here's how much more we can make the business. Here's how much less we would be costing and fixing a lot of the usability issues that we found. That's an approach that I would take to really determining solid ROI for a rapid research program.

DANA:

Yeah. It sounds like great advice for someone who's trying to get that buy in to implement a program like that. On the flip side, what might some of the pitfalls to avoid be, when you're implementing a rapid research program?

DEVIN:

That's a great question. I think for me, the pitfalls would be, you have to be careful what it is that you determine to be in scope for rapid research, because you don't want to accidentally cannibalize the work that your team does want to do because I could see rapid research in two ways. I actually talked about this earlier this year in a conference with UserZoom as well. We talked about two ways of implementing rapid research.

One is if you just plan it in your sprint cadence. This next sprint cycle, or for the next two sprint cycles, we're going to do really foundational research. Then the two sprint cycles after that, we're going to really just do agile, rapid research to get through our research backlog and really do a lot of usability testing and evaluative work. Or you can do what we did at Verizon, which was we hired two associate researchers who are relatively new to the field that could tag team and deliver really great value together as a team.

Both options have pros and cons and trade-offs. You just need to be really intentional with where your company and where your organization is in terms of what would be most valuable for you. Because if you just jump right into rapid research, without thinking critically around the guardrails, you could get yourself in a really hot mess. For example, if you're trying to make longitudinal work rapid, that could be really tough. Maybe it sounds great in theory, but maybe you won't really get quality results on the other end.

Here at Capital One, we have a lot of compliance-based requirements that we have. Making sure that legal signs off and approves a lot of the research that we do, making sure that we are accommodating any risk to data management.

When we're running research, that's going to be a question. How do you make that rapid, not to cut corners in the legal and compliance processes that are required of researchers at regulated organizations like Capital One. You don't want to cut those corners because you're going to create a whole lot of risk and you could actually jeopardize your entire research practice by breaking the trust with some trusted legal partners. So again, I think intentionality is the name of the game with how and where you decide to do rapid research.

DANA:

What about processes within the research realm? So guidelines and templates and things like that. What's some of your advice around that in terms of making it a really efficient, rapid research program?

DEVIN:

For what I've found success with in terms of a rapid research program, that's where that balance of intention and programitization comes into play. Having very rigid guidelines in terms of recruitment, where you're not going to really provide too many flexible recruitment profiles because otherwise, how do you know you can recruit them in time. You only got maybe a week or two weeks in order to recruit specific types of participants. So you want to be very mindful of who you can reach in that amount of time. But then in terms of templatization, I think outside of recruitment, I would recommend templatizing some of your process.

ALFONSO:

So you've been talking about things that are related to research ops, right and to organization and making sure that you can operate efficiently. How would you talk about the balance between quality and speed?

DEVIN:

I would say that's where, and I may sound like a little bit of a broken record, but the templatization is really important. Because when you think about systematizing something and you want it to be repeatable, a lot of the quality comes in the upfront effort to make sure that the systems and programs you're creating to deliver insights at scale are themselves quality systems and processes and programs, because then when you repeat them and you kind of feel like you're on autopilot, you know that you are not short cutting the quality of the insights itself because you've done the effort already. You've already made the investment. So when you're creating those templates, be very intentional on what is the most type of research that you would do in a rapid research program? What are the right ways of asking the right questions and what are the things we should templatize and what are the things we should not templatize. Again, bringing in that intentionality really would help to make sure that you sustain quality while being rapid.

DANA:

For somebody new who's just getting into rapid research and trying to build a program and having done it successfully yourself, what's a realistic timeframe for how long it takes to get in your cadence and your rhythm and sort of get it to that point. I mean, is it on day one or does it take a year?

DEVIN:

That's a really great question. One that we actually unpacked a little bit earlier this year in a conference that we were at. The one thing I'll say is at Verizon, we got done, we got to a sort of break even point relatively quickly; it was about four to five months. That was from the moment that we identified the need to the moment that we were able to secure a cadence. That was because again, to sort of Alfonso's point, we were really intentional upfront about what are the partnerships we have and the tools that we have to make this work. And of course, in partnership with UserZoom, it was a whole lot easier to get that off the ground as we start to think about, we had a coding tool in place, we had an easy way to get ahead of customers.

We were able to look at the panel and see what's possible to recruit in that quick amount of time. All of those conversations were relatively easy, but in a company like Capital One where we are a little bit more compliance based, we have a little bit more in terms of process that we need to make sure that we uphold, it could take anywhere from that five months to a year. But I think that it's not as daunting or as difficult as people think.

That's the one thing I really want to make sure that is clear that really the toughest part is that upfront investment of what are the things that we need to templatize. What's the cadence that we can realistically do. Then the rest is just being consistent. For example, if you're building a rapid research, maybe not program, but cadence leveraging sprints to do rapid agile testing. Rather than building a whole separate program with separate tools or anything like that. Then those are things that just require a lot more planning. So you may not need brand new tools. You may not need brand new processes. You can fit that into existing tools and existing processes. You just need to be a little bit more forthright with your planning and fitting in rapid research in the cadence of your team that makes sense.

DANA:

I think it's great. I think it's a great insight. There's a little bit of planning and a little time involved in getting to that goal, which is being a well oiled machine. It doesn't happen immediately and there's a lot that goes into that, but the payoff is huge.

DEVIN:

Absolutely. One of the terms that I've used instead of a well oiled machine is one I've been saying a lot more recently is an insights ecosystem. How can we create an ecosystem of systems, programs, processes that enable us to do the micro, tiny democratized studies that everybody can do to the large foundational, complex meaty projects that will take months on end. But we all are in it together and we're all working with one another and it all comes back to the same team or the same process. So yeah, insights ecosystem.

DANA:

I love it. It's very big picture.

ALFONSO:

Thank you so much, Devin, for joining us today. Rapid research is a super hot topic and a lot of companies are getting there.

DEVIN:

Absolutely. Thank you both so much for having me. It was a pleasure as always.

ALFONSO:

That’s Devin Harold, Director of UX Research, Financial Services at Capital One

DANA:

Thanks for listening to UXpeditious. Make sure to continue listening to our new episodes each week for quality insights from UX industry leaders. If you like what you heard, help us out by rating and reviewing the show on your favorite podcast platform.

ALFONSO:

UXpeditious is produced by UserZoom in partnership with Pod People. Special thanks to our production team: Christopher Ratcliff from UserZoom; and the team at Pod People: Rachael King, Matt Sav, Aimee Machado, Hannah Pedersen, Colleen Pellissier, and Michael Aquino.