When it comes to usability testing, many are unaware that there are two types of evaluations that can be conducted during the product design and development lifecycle. These are formative and summative usability tests.
The main reason why usability tests are important is to find out the effectiveness of the design and to evaluate the ease-of-use of a product. Even though both types of testing are performed to understand and improve the usability of a design, they are performed at different stages of the development process.
So which tests do we use during the product design and development lifecycle? The only way to know is to understand the differences between the two.
Let’s take a look at what formative and summative usability testing means…
Formative usability testing is done early in the product development to help form the product’s shape and design.
Formative usability testing answers the why and how questions of the design’s usability. It answers why something is not working and involves iteratively evaluating a product during design and development.
The goal is to detect any issues and eliminate usability problems before a product is fully developed. Formative usability testing takes the role of a support tool for decision making during the beginning stages of the design process, which helps to discover insights and shape the product’s direction.
Formative usability testing tends to be more qualitative in nature. We can see how users actually experience the design and see where and why they get stuck, and hear what they say using the think-out-loud method.
It is crucial to observe and understand the users’ thought processes and their actions resulting from them. The data collected during formative usability testing is observational in nature, hence the name qualitative usability testing.
For an in-depth guide to measuring UX and proving the value of your research, check out our free-to-download, 42 page guide to running both longitudinal and competitive benchmarking.
Formative usability testing heavily influences the design decisions and is considered to be an excellent tool to figure out which design features are useful and which are not.
It’s recommended to conduct at least two formative usability tests.
If we don’t conduct formative usability testing during the design and development lifecycle, we might end up designing something that will simply never be a usable solution.
Formative usability testing is carried out:
Summative usability testing is usually performed later in the product development process when a product is fully developed.
It is often conducted when a design is reasonably complete and involves evaluating the design against quantitative goals or competitor’s products.
It is an evaluation of a product with representative users and tasks designed to measure usability (defined as effectiveness, efficiency, and satisfaction) of the complete product.
The main purpose of summative testing is to evaluate a product through defined measures. It uses UX metrics of users’ success to assess whether the product meets defined usability success metrics.
So, summative usability tests are quantitative in nature, and they act as a final validation where usability issues have been identified and addressed.
Summative evaluation tells us how usable an interface or a product is. If we don’t do summative usability testing before a product is released, then we won’t find out if an interface or a product has any problems or not.
Summative usability testing is used to obtain measures to establish a usability benchmark or to compare results with usability requirements. The metrics recorded in summative usability testing reflect what actually happens during the test and not user perceptions or feelings.
The usability requirements should be task-based, including: task completion rate, time on task, error rates, and overall user satisfaction.
It’s about measuring effectiveness and ease of use. These metrics include: pass/fail of user tasks, the average time it takes to complete tasks, counting the number of clicks, users’ errors or system errors.
In summative testing, the volume and speed of tasks undertaken, and the pass/fail metrics are more important than the quality of the observation or user narrative.
Summative usability testing is carried out:
Both formative and summative are forms of evaluative research. With formative and summative testing during the design and development lifecycle, both qualitative and quantitative data are collected. These tests help to determine whether the design elements are working or not and how they can be improved.
Start formative testing as early and as often as possible in the design and concept phase for a user-centered design approach. Incorporating user feedback through formative and summative usability testing early and often can result in a better and more usable product.