A quick start guide to running user research in Agile

Get UX research up and running in Agile, fast!

This is a guide for UX and research practitioners who have either newly joined an organization that’s using the Agile methodology, or their current product development is adopting Agile for the first time and they’ve been told to “make UX research happen in this new fangled process immediately please!” 

In this article, we’ll discuss:

  • What is Agile?
  • What are the benefits of Agile?
  • Why user research has traditionally struggled in the Agile environment
  • How to make user research happen in Agile and make everyone think you’re super awesome

And if we have time at the end, we’ll wrap up with a Q&A/rapturous applause/speed-cupcake-eating-contest.

What is Agile?

Agile is an iterative approach to developing software, websites, apps or any other kind of digital experience.

Within this method, a multi-disciplinary team formed of relevant people from across the entire organization work in short 2-3 week ‘Sprints’ to deliver a specific part of a product rather than working on one massive, finished project. This might be a part of a prototype, maybe some updated piece of code, or a new page design.

Basically, something small gets delivered that improves the overall product, and everybody in the Sprint works towards it.

What are the benefits of working in Agile?

The major benefit of working in Agile is that you can get a working product or feature in front of users in a very short amount of time.

In the traditional, and increasingly outmoded, ‘Waterfall’ methodology, development is an eight-step process (conception, initiation, analysis, design, construction, testing, implementation, and maintenance) that can take 12 months. During that expanse of time, anything can go wrong.

Budgets can be slashed, competitors can get the jump on you, your stakeholders are suddenly replaced and you have to get buy-in all over again. A lot can happen in a year, and all that time, resource and investment can be scrapped in a heartbreaking moment.

But because of the speed and efficiency of Agile, you don’t have to worry about all of the above. Minor incremental changes developed during a very small window of time by a tight-knit, completely focused group can deliver results, which over time can build into much bigger gains and improvements for the entire product.

Being Agile also means you can more easily change direction in the face of evidence, without having to scrap an entire 12 months worth of development. Hey, you’re fleet-footed, you’re scrappy, you’re AGILE, you can do that kind of thing!

Why user research has traditionally struggled in the Agile environment

Although the Agile manifesto says: “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software” often the feedback you get from customers only occurs once the finished ‘thing’ is live and being used by the customer.

This is a false economy. As UXDI states in their Introduction to UX design training coursethe only thing you learn from putting badly designed software in front of users is that your users don’t like badly designed software.

We’re firm believers in embedding user research into product development as early as possible – it can save so much wasted time and money, and save you from releasing a product that users don’t actually want or need.

The problem is that within a two week Sprint, user research isn’t done because Agile adopters don’t think they can get users into a two-week sprint fast enough to do the tests.

They think that it’s going to slow them down and they won’t have their assets ready in time, or that it might be too low fidelity to gather any useful insights. As a result of that, user research and testing is abandoned and the finished product is a sub-optimal experience.

As this 2018 quote from Forrester states, “Agile product teams complete substantial work without user experience testing, either leading to lower quality products or rework that increased expenses and delayed launch.”

In a traditional development process, you get weeks or months for discovery, with plenty of time to design just the right study and include tonnes of participants. But this isn’t realistic in Agile. How can you possibly figure out what questions to ask and how best to answer them in a two-three week Sprint?

How to make user research happen in Agile and make everyone think you’re super awesome

We talk to a lot of designers, product managers, developers and UXers who work in Agile (or pseudo-Agile) teams, and we find many similar issues preventing them from running user research within Sprints:

These barriers include:

  1. Assets are too ‘early-stage’ for gathering useful insights
  2. Participant recruitment takes far too long 
  3. The team lacks the necessary research skills
  4. There are just too many research questions to answer

The good news is, all of the above can be solved, you just have to learn to adapt your methods to be more… uh… agile. Once you figure out your research goal, the same UX research methods you traditionally use still apply, you just have to adapt the methods to fit the time-frame.

Let’s take a deeper look into the above issues and provide some Agile solutions…(Please note: some of the following insight is taken from our own Agile ebook, as well as articles by Lee Duddell and Amanda Stockwell).

1) Run early-stage design tests

You don’t need to present users with finished, or nearly-finished assets to gather useful feedback. You can start getting actionable insights on very early designs – even hand-drawn sketches, as well as wireframes, flat visuals or prototypes.

This type of early-stage design research is known as directional testing, and can often be as simple as checking where people would click on a series of flat images, or a think-out-loud study of a more complete journey in a tool like Axure.By carefully setting a scenario for users explaining they’ll be using a work-in-progress site, you can ensure feedback is useful even when the test asset is low fidelity.

2) Use a recruitment service that can source participants for you within hours or create your own ‘standby’ pool of participants

With traditional recruting approaches, it can be a struggle to recruit enough participants within a Sprint to conduct testing. Even finding one or two participants that fit your profile can be a challenge.

However if you use an automated participant sourcing engine, such as our own Intellizoom, where you can choose from a global pool of more than 120 million users, you can source in advance and be able to view results within hours of launching a study. Giving you something better to do with your time than looking for users.

If budget is an issue, you could create your own small, reliable pool of participants to have at the ready before your Sprint. These people could be gathered from your mailing list or via a pop-up on your site/app.

Just a word of warning: in a pinch, you may want to ask random people from your team or organization to be test participants, but they won’t be suitable as they’re too close to the product and probably far too ‘tech savvy’ to represent your own average user. Remember: You are not your user!

If you must ask people you know, try friends and family as they may be closer to your representative audience. However they may give biased feedback as they know you and may* not want to hurt your feelings. (*relationship dependant)

Remember to maximize the time you have with participants by asking only the most pressing questions. Maybe you can get away with talking to four people instead of the 10+ you’d normally interview.

The key is to get enough information to be useful without losing the integrity of your research efforts.

3) Democratize UX research with templates that don’t require a Human-Computer Interaction degree to use

Just because there’s no researcher in your Sprint team, it doesn’t mean your team has to skip user research. In fact you can speed up and scale the entire organization’s research capabilities without hiring hundreds of researchers by ‘democratizing research’.

This simply means enabling non-practitioners to run their own tests within Sprints.

You can do this by using a UX platform, which automates time-consuming activities (like recruitment) and is already pre-loaded with test templates – to enable designers, product owners and anyone who is not a full time researcher to run their own testing, quickly.

Or if you have a minimal budget, you could set up your own repeatable study templates to answer the most common questions that product teams need answers to.

Here are a couple of examples from our Senior UX Director, Lee Duddell:

Common question #1: “Before we start designing, what’s most important to customers?”

You could answer this question with an online survey, which allows you to ask a wide variety of probing questions to get inside the heads of potential customers to understand their attitudes and habits. To understand what’s most and least important to customers, the survey should include a ranking question, like this:

Surveys are fast to set up and can be even faster to analyse. You can set up a survey in less than an hour, get results the next day and then start making insight-driven decisions about designs.

It’s also really easy to get rapid feedback on design elements, like icons, as you can upload them straight into the survey.

Common question #2: “What do users think of this concept?”

This can be answered with a think-out-loud study where users speak their instinctive, immediate thoughts to a written concept that you present. It is a great way, before you’ve actually started designing, to gauge how users respond to your ideas and what elements/benefits are most and least appealing to them.

In the screenshot below, users are presented with a very basic idea (as a Google Doc) and asked to give verbal feedback:

Concept testing can also help you start to bring your customers’ language into the team by (literally) listening to the words they use to describe our concept and the problem you are aiming to solve for them.

For more ideas, we’ve gathered together five of the most common questions asked by product teams and the associated tests our clients use to solve them: Five UX tests you should be running in Agile.

4) Answer the most impactful questions first and consider implementing a research backlog

You need to be clear that it’s simply not possible to test everything (or even most things) while running research in Agile. However, working in a two-week sprint should focus your mind on answering questions that will be most impactful to the business.

Rather than measuring your output (just finishing the product), focus on the outcomes (delivering revenue, driving conversion, making $$$) by answering fewer, but more business KPI related questions, faster.

Fresh research is soon eaten up and product and design are often left wanting more. For the questions that are simply too large for Agile to handle, take a different approach.

In contrast to directional testing, you can carry out foundational research (exploratory research on a topic that hasn’t been clearly defined) on a separate track over weeks and months.

This ensures the big questions are given the time they deserve so you can reach the right, user-centered conclusions. This basically means that you’re avoiding ‘research FOMO’ while directional testing ensures you are impacting your business in every single Sprint.