Every successful user study starts with a clear hypothesis. Here’s how to get started.
A good hypothesis is critical to creating a measurable study with successful outcomes. Without one, you’re stumbling through the fog and merely guessing which direction to travel in. It’s an especially critical step in A/B and Multivariate testing.
Every user research study needs clear goals and objectives. Writing a good hypothesis stands in the middle of that process, which looks like this:
1: Problem: Think about the problem you’re trying to solve and what you know about it
2: Question: Consider which questions you want to answer (and we do not mean the study questions to your user)
3: Hypothesis [the focus of this guide]: Write your research hypothesis
4: Goal: State one or two SMART goals for your project—specific, practical, and actionable
5: Objective: Draft a measurable objective that aligns directly with each goal
In this article, we will focus on writing your hypothesis.
According to the dictionary definition, a hypothesis is “a supposition or proposed explanation made on the basis of limited evidence as a starting point for further investigation.”
1: A hypothesis is your best guess about what will happen
This change will result in this outcome.
The change is a variation on an element—a label, color, text, etc.
The outcome is the measure of success, the metric—click-through, conversion, etc.
2: Your hypothesis may be right or wrong, rather than ‘what you want’—just learn from it
The statement might be quite bold, such as “Variation B will result in 40% conversion over variation A”. If the conversion uptick is only 35% then your hypothesis is false. But you can still learn from it.
3: It must be specific
Stated values are important. Be bold while not being ridiculous. Believe that what you suggest is indeed possible. Don’t use ‘...will be better’ or ‘...higher’, use ‘xx%’ or ‘nn’.
4: It must be measurable
The hypothesis must lead to concrete success metrics for the key measure. If click through, then measure clicks, if conversion, then measure conversion, even if on a subsequent page. If measuring both, also state in the study design which is more important, click through or conversion - and you might state why.
5: It should be repeatable
You should be able to run 3-4-5 different experiments testing different variants, and then go back later to re-test #3 and get similar/same results. That’s the science. If you find that your conversion went down, then back up to a prior version and try a different direction.
Any good hypothesis has two key parts, the variant and the result.
First state which variant. Just one; A-B-C… or the recipe if multivariate (See AB-MVT).
“Variant B will …” Also be sure that you’ve included screenshots of each version in your testing documentation for clarity, or detailed descriptions or flows of processes.
Next, state the expected outcome. “Variant B will result in a higher rate of course completion.” After the hypothesis, be sure to specifically document the metric that will measure the result - in this case, completion. Leave no ambiguity in your metric.
A good hypothesis has its birth in data, whether the data is from web analytics, user research, competitive analysis, or your gut (which we’ll call experience).
It should make sense, be easy to read without ambiguity, and be based on reality rather than pie-in-the-sky thinking or simply shooting for a company KPI (key performance indicator) or OKR (objectives and key results). The data that result is incremental and yields small insights to be built over time.
The images below are for A, B, and C variants. The ‘control’ is the orange box, while green and grey are variants B and C (Always state a control, which is generally the current design in use).
Hypothesis: Variant B will result in the highest click rate.
Read the examples below and think about how this hypothesis could be improved.
Background: It has been noted through web analytics that…
Background: It has been noted through web analytics that:
NOTE: In your background, it’s best to link to actual studies that show your insights.
Ultimately, creating a solid hypothesis is about following a process. By thinking about the problem, your prior data, your experience, plus the design options you’ve created, you already have everything you need to write a great hypothesis.
Take your user research skills to the next level. Discover the UserZoom Academy today.
UserZoom Academy is designed to equip UX practitioners, from beginner to expert, with the skills and knowledge needed to create better digital experiences, boost their professional development, and promote UX awareness across their organizations.