Frowned upon at the movies, beloved in research.
TOL is a study that is used to get verbal feedback from participants. Questions in a TOL test are crafted in a way to elicit verbal and observational insights from participants.
This method can be used from the very beginning with wireframes to the final iteration of the website, and indeed, TOL usability testing is recommended throughout the whole product lifecycle to ensure that your website/app being developed is user-centered.
In particular, TOL testing is important for when you need to know the HOW and WHY behind the WHAT. For example, when new changes have been or will be made to a live site consider conducting a TOL study in order to ascertain user comprehension, usability and preferences.
There are many times when hearing what users have to say as they experience a site, app or prototype can be useful. As such there are many use cases for when a TOL study might be what you’re looking for to complete your research goals, and we certainly see a broad range of use cases from our customers.
Here are just a few of the more common ones that we see:
Just as with most research projects, you first start by identifying a research goal and writing a hypothesis. Then you design a test that will hopefully prove or disprove your hypothesis by writing research questions that will get to the heart of your issue.
Where a TOL study differs the most is that your questions and tasks have to be written and crafted in such a way that you get your participants to channel their thoughts aloud. This can be more challenging than you might think, but with practice and a little sound advice from a company who knows a thing or thirty about TOLs and you’ll be swimming in actionable feedback in no time.
We have quite a bit of experience getting companies up and running with TOL studies. Here are a few of the most practical and helpful pieces of advice we give customers:
Give participants the ‘OK’ to delve into their imagination, and speak from the heart.
Here are a few essential facts to consider while deciding on whether or not a TOL study is the right approach for your research goals:
You get videos with rich audio insights from participants that need to be analyzed. What we usually do in house on the professional services team, and what we recommend customers to do for their stakeholders, is to create a report that is supported using clips/participant quotes that shows the answers to the hypotheses and or research questions and trends identified.
On top of this you will also get task metrics, such as Success, Time on Task, Clicks and Pageviews. And then depending on what type of supporting questions you include in your study you may also get likert scale ratings, NPS, open-ended text boxes, etc.
This is a big one! Trying to keep coherent notes while analyzing videos and trying to put the whole picture together can sometimes be challenging, which is why we recommend including two things when analyzing TOL responses:
Analysis Matrix. In order to prevent your notes from looking messy while analyzing TOL videos we recommend that you create an analysis matrix like the image below. This will help you track significant observations, quotes, video time frames and start to track trends.
Issue severity chart. Using a issue severity chart, such as Low, Medium and High, will help you prioritize and identify the most important issues (if conducting a usability test).