Comprehensive guide to incorporating Agile UX research
Incorporating research into Agile remains one of the biggest issues for UX teams. And not just incorporating the research, either, but incorporating it well.
I’ve seen lots of people share their tips to coordinate design and development efforts and create ever leaner and faster design documents, but user research remains an area of struggle for most teams. Even if teams are able to figure out how to fit research into sprints, making sure the data collected gets appropriately analyzed and then actually used can still be troublesome.
In this comprehensive guide, we’ll discuss challenges of integrating UX research and some strategies to overcome those struggles.
Agile: a quick recap
Agile teams are supposed to focus on launching small chunks of working code often, foster collaboration between and among users and cross-functional teams, and have an openness to flex plans and processes as new information is learned. There aren’t distinct periods for discovery and definition, design, development, and testing. Rather, the team is to be constantly learning, adapting, and building.
A lot of Agile teams used a framework called Scrum with well-defined, timeboxed chunks of time called Sprints to plan and do increments of work.
In theory, this lends itself perfectly to performing and incorporating UX research. Iterative cycles of building and learning? Focus on responding to items learned over time? Heck yes!
But in practice, Agile teams often struggle with keeping up with the pace of research. How do you make sure you have something to research every single cycle? How do you analyze all the data you collect quickly? And how on earth do you keep dev teams on track with a constant influx of information?
The truth is, there is no magic answer. As with much of the rest of UX, the best solution depends on context. How big is your team? Who’s on it? What are you working on?
Read on to hear about ways to shift practices to make the most of UX research in an Agile setting.
Agile team makeup
There are tons of ongoing debates about both “ideal” Agile team makeups and the extents to which a UXer’s skills should go. Should Agile teams have embedded UXers or should there be a separate team of cross functional UX professionals? If there’s a UXer on a team with developers, should that person do research, low and high fidelity design, maybe even code?
I’m not here to rehash that debate or pick sides, but the scope of a UXer’s role on an Agile team matters when it comes to incorporating research for two major reasons: UX is really hard on your own, and you shouldn’t be the one testing your own designs.
The best solution is probably going to differ from place to place, but there are a few scenarios I’ve seen work quite well:
1) Embed a full time designer and researcher on each cross-functional team
The two can crossover skills to some degree and collaborate on sketches, prototypes, and actually running research, but there is some separation of who owns visuals and who owns research. This setup is really awesome, but I’ve only been in a few organizations that had the budget to pull this off.
2) Have a full time designer on each Agile team and a research unit that floats between Agile teams
Researchers can and still should collaborate with designers, they will just have to divide their attention between teams. Usual pros and cons of splitting attention apply.
3) Have one generalist UXer on each cross-functional team
I’m not generally in favor of attempting to hire UX Unicorns, but I have seen a few generalist set ups work quite well in Agile. The key for success in this case is to have one generalist UXer on each cross functional team, and then have the generalists partner up to research each other’s work, especially when it comes to usability testing.
This helps make sure you don’t introduce the unavoidable bias of testing your own work (e.g. how much do you love option A that I lovingly crafted and how much do you hate option B my PO forced on me?) Sometimes you just can’t afford a research specialist who is trained to remain neutral, but partnering will certainly work better than testing one’s own work.
There are many more set ups that can work, but given the option, I’d go with number one – embed a full time designer and researcher on each team.
In traditional development processes, there’s an entire phase dedicated to figuring out what you need to figure out. You might get weeks or months for discovery, with plenty of time to design just the right study and include a lot of participants. All that’s great, but it’s not realistic in Agile. Instead, you are constantly learning, designing, building, and launching.
Most teams use some version of a Sprint to timebox efforts, and these Sprints are usually 2-3 weeks long, which means that UX research efforts are now supposed to be planned, executed, and analyzed in that time. Even without the scrum framework, there is the push to get feedback faster and faster and the dedicated time to research is gone.
So how to figure out what questions to ask and how to answer them? The good news is that all the same rules of UX research considerations apply in Agile. Once you figure out your research goal, the same methods you’d traditionally use will still apply. Christian Rohrer and Jeff Sauro both have a good cheat sheets if you need help with that.
However, you’ll probably have to adapt the methods to fit the time frame. My first suggestion is to break down your research goals into the smallest possible questions and create specific hypotheses to explore. Investigating a single question at a time will allow you to narrow the scope of your study, which in turn makes planning, running and analyzing all faster.
You might even be able to combine two narrow goals into a single session. For instance, maybe you’ll follow up on a benchmark for an existing interaction and then do a single task tree test. The key is to maximize the time you have with participants to answer the most pressing questions at the time.
You’ll also want to consider if you can relax some of your traditional standards for research. No, you don’t want to ask the developers on your team to be usability test participants, but maybe you can find representative-enough users in your sales department or pay for close-enough users from a survey service.
Maybe you can get away with talking to 4 people instead of the 8 or 10 you’d like. The key is to get enough information to be useful without losing the integrity of your research efforts.
You may also want to look into relying more on remote unmoderated, and automatic methods. There are things that are better suited for in person or moderated, but you can still get great insights and the turnaround time can often be very quick. It’s especially helpful if you have a spread-out user base that operates in different time zones.
Remember – the beauty of constantly doing research is that you always have another opportunity to get clarify or ask more questions.
For a free, in-depth guide to running user research in Agile Sprints, read our comprehensive e-book:
Logistics: How to streamline user research to keep pace with Agile?
There are lots of methods of creating something to test quickly, here are a few tips that have worked for me:
1) Set a research time every sprint and hold it sacred
No meetings, no dentist appointments, no ifs ands or buts. You want everyone on the whole team to know this time is important, and it gives everyone a chance to come see the sessions. Once you’ve set aside the time for yourselves, it also means you can start recruiting people regularly. Even if you haven’t yet decided what you’re working on in a sprint, you may know you want to target a specific persona and start contacting them to get something in their calendars.
2) Build up your own database of participants
Identifying, reaching out to, and then screening participants can be a cumbersome process. Have a group of willing participants can really speed up the process. If you have existing users, you can set up a screener survey and invite users to sign up via normal communication channels like emails, post it on your website or social media outlets, or invite customers who contact support. You could also recruit participants who aren’t yet customers with social media posts. Continue to recruit people constantly and build up a pool you can contact over time.
For an in-depth and entertaining guide to successfully recruiting the ‘right’ participants for user research, download our comprehensive new ebook:
3) Allow self-scheduling
One of the biggest time-sucks in recruiting moderated sessions can be the back and forth of finding a mutually agreeable time to meet. Since you have a set block for you to conduct sessions, you can use one of the many scheduling tools to let participants book their own block of time. I’m a fan of pow wow, but Calendly is also popular and there are bunches of others. The key is to look for something that will let you send customized confirmations and reminders with contact information.
4) Use an unmoderated tool with a built-in panel
Then you don’t have to deal with any of this! You just set up the test, set the screening questions for respondents, and let the results flow in.
Learning as you go
Once you have the logistics in place to run sessions consistently, the biggest challenge will be figuring out what to do with all the information collected and what the team should do next.
As a reminder, it helps to set up the research properly with a specific research goal and hypothesis. If you know exactly what you are looking for, it’s much easier to decipher the story that the data you’ve collected tells.
Let’s say you’re working on incorporating a new item into an ecommerce platform and want to determine where it should live. Maybe you have a hunch that it should live in a specific category but some of your teammates aren’t sure. Your hypothesis could then be something like, “I believe users will be able to find product A in category B.” Then you keep track of how many people are and are not able to find the item during your test.
This is a very straightforward example, but it’s amazing how much easier it is to synthesize research when you have a clear hypothesis.
Involving the team
You should also take advantage of Agile’s emphasis on cross-functional, collaborative teams and get as many members of your team as possible involved in watching research sessions, whether that means observing interviews or reviewing remote session videos.
You can still take on the brunt of planning, but you want the team involved in viewing sessions. This is especially helpful for the researcher because someone else can take notes and you can really focus on running the sessions. I like to have the team take notes in a simple shared spreadsheet, with each participant listed in a row and the key research goals listed across the top. That way, anyone watching can add notes in a relevant place.
Having the whole team involved in sessions helps with more than just session logistics. A lot of non-UX researchers are abstracted from users. Developers and business representatives might read reports or hear stories about frustrations, but there is nothing quite as illuminating as watching someone grimace their way through an interface. You want the whole team to be learning the frustration points and areas of opportunity together.
That not only makes a bigger impression on the rest of your team and builds empathy with your users, but you spend less time documenting what happened and more time working on solutions together. Everyone learning together means there can be less opinion-based debate and more data-based decisions, which lets you all move faster.
You can have a debrief session immediately after the final session where everyone shares their key takeaways, discusses priority of findings, and starts brainstorming solutions that address experience issues and account for technical feasibility and business perspectives.
You won’t have to worry about convincing people to buy-in to your recommendations or figure out whether they can be built, because the whole team will be there to come up with the next steps. Rather than ending the research sessions with a report of what you found, you can end with the team having a shared understanding of findings and the beginnings of a plan to solve issues or shift things.
You might want to incorporate the research debrief as part of the planning session for the next iteration of work. Since there is already time set aside for teams to plan in most variations of Agile, you won’t be asking for additional meeting time from your team. Just be careful to keep the focus on understanding the data you just collected and setting the frame for the work that needs to be done next, rather than trying to completely solve issues then and there.
You could even schedule debrief sessions after each participant and see if there is anything minor you could change between sessions to improve the experience. For instance, if two users in a row can’t find an item, a developer or designer might be able to quickly update a prototype or test version that you can show the next participant. That way, you don’t spend a whole chunk of time confirming that something is broken. Rather, you can get some rapid feedback on different versions and pick a direction based on the data you collect.
It can take an adjustment if you’re transitioning from a traditional Waterfall culture, but there are definitely some logistical and team adjustments you can make to incorporate UX research in an Agile environment. I’ve found taking advantage of Agile’s focus on collaboration between cross functional team members can actually maximize the impact UX research is able to have across the board.
Maintaining the UX and Agile relationship
All things, even the best built machines, need maintenance. The same is true for UX teams; especially when it comes to incorporating research findings into an agile UX process. The following tips will help you keep up the much needed maintenance to avoid your hard won research getting lost in the flow, and to keep the process flowing smoothly.
As mentioned in the previous piece, relying on team collaboration can allow you to shift your focus away from creating lengthy summary documents. You will still need some way to keep track of everything you’ve researched, however, in case team members shift, priorities change, or you simply don’t have time to address all your findings. Plus, you need ways to document what you’ve done for posterity’s sake or in case there is a stakeholder or a team member that couldn’t be involved.
Keep everyone in the loop
As soon as the research debriefs are over, I like to send out a summary email to the team that highlights what the key research question was, what we found, and what we’re planning to do next. Simple, to the point, and the same format every time so people know what to look for. If there was any sketching or other visual brainstorming from the debrief session, include photos and brief explanations. The key here is to be brief, both for readers and for yourself; aim for the summary to take no more than 15 minutes to write and 5 minutes to read. If the team has a wiki or other shared note resource for things like retrospective notes, keep an ongoing summary and just copy and paste the email.
Mind the backlog
Next, add your suggestions to whatever mechanism you use to keep track of future work. Most Agile methodologies (e.g. Scrum, XP, Kanban, etc.) call for the use of some version of a backlog, or repository of user stories that describe needed features, changes, or other work to be done. Rather than creating a separate UX backlog for things that improve the experience – add those stories to the overall backlog to ensure that they are visible to the whole team and get prioritized along with other types of work. This will reinforce the importance of focusing on people’s experience and the importance of including the whole team in those efforts.
If it becomes clear that you have a large amount of work to be done to figure out UX solutions, you can consider having a UX spike. A spike is just a focused iteration where you the whole team focuses on only one type of story, whether it’s tackling bugs, understanding a technology, or brainstorming design solutions. The key difference between a spike and a normal sprint is that you end the spike being better informed but not necessarily with working or releasable code.
You can also utilize the Agile practice of having team retrospectives to keep UX research findings top of mind. In a retrospective, the team meets to discuss the work cycle that was just done and what could be done better from a process and team perspective moving forward. You can take a small amount of this time to review and discuss the research that was just done or to update the team on UX metrics you monitor over time. That way, you ensure that users’ goals and needs stay top of mind and the team can shift as it needs to in order to keep improving. If your team has a wiki or other tool that they use to keep track of retrospective outcomes, you can add a line or two for a UX summary so things aren’t lost.
Agile UX Success
In summary, the whole Agile team will need to adjust to successfully incorporate UX research well into the process. There is no one solution that will fit for every team, but try the above tips to get yourself in the right direction.
If you’d like to know more about how UserZoom can help test, measure and improve your own site’s UX, please get in touch!
Amanda is President of Stockwell Strategy, a UX research practice focused on lean research methods and integrating user knowledge with business goals to create holistic product strategies for businesses large and small. She has focused most of the last decade focused on finding innovative ways to understand end users and embed that knowledge into overall process. She’s lead teams that provide research, design, and UX strategy services and frequently writes and speaks about her experience. She has a human factors background and an engineering degree from Tufts University.