As researchers and designers we often find ourselves limited in how we conduct research. We run countless research studies but can’t quantify our data. We write very solid screeners but still do not hit our target audience. We watch participants struggle through our designs but can’t obtain concrete data on how to improve the navigation. This article will focus on four research goals and how an unmoderated remote testing solution can be used to accomplish them.
There are various software vendors, tools and solutions available in the market. We advise that UX Pros do their homework and identify their own research goals in order to select the right one. For those of you unfamiliar with UserZoom, it is a package of on-demand software products, compiled under the same platform. The software is used to run unmoderated remote (or online) user experience research. This form of research provides pre-defined tasks or an open task study that participants engage with using their own computer in a native environment. With the remote unmoderated method, a researcher doesn’t need to be physically present with a participant. You can cost-effectively conduct high sample-size studies and collect responses from participants who are located across the US or internationally.
Research Goal Number 1: Identify Visitors’ Core Tasks (True Intent Study)
Too often we conduct company-centric testing, using tasks which WE consider important. However, part of embracing a user-centered design/research approach includes understanding what YOUR customers want to accomplish when they visit your site. This understanding gives you a foundation for making design decisions to drive a better user experience. With this knowledge you can define your future research goals and needs and ultimately create a solid usability roadmap.
A major strength of the UserZoom research solution is the ability to determine what the goals are of real visitors who visit your site. With a True-Intent Study, you can intercept your visitors and follow-up with intent and satisfaction questions prior to leaving your site. The data is automatically captured and aggregated into graphs and charts, which you can export into a report or filter right within the UserZoom interface.
True Intent Invitation Screen
Research Goal Number 2: Improve the Navigation Experience (Card Sorting Study)
Many times we see our customers struggle with our menu structure and do not have a basis for how to improve it. When developing or redesigning the navigation structure of a website, it is important to understand the expectations of visitors. If a user’s mental model of information aligns with a website’s design, then the experience of navigating the site is going to be more efficient and effective. This includes speaking the same language as your users and limiting technical jargon. Unfamiliar terminology impedes your visitor’s ability to make navigation decisions, detracting from the overall experience on your site.
UserZoom features a Card Sorting solution that can be used to understand how participants mentally organize the information on your site. During a Card Sort, participants are given a list of menu items they can drag and drop into categories (menu headers). Data from a card sort study can then be used to generate a tree diagram (dendrogram) that groups items based on their relationships. These groupings can be used as a basis for improving the structure of the website. Additionally you have the opportunity to ask follow-up survey questions. These questions allow you to further expand on terminology, confusing items and gather critical feedback to help with your design/redesign.
Drag and Drop Card Sorting Interface
Research Goal Number 3: Quantify Usability and User Experience (Task-Based Study)
When working to improve the user experience of your site, statistically significant data can be valuable to show that design changes result in improvements. With an unmoderated remote testing method you’ll be able to obtain those statistics through the ability to test large samples. Comparing pre and post design metrics or conducting iterative testing and gauging metric improvements can be valuable in a design process. You have the ability to periodically test your website, improve your design, and produce quantitative data that validates design decisions. Some of the usability metrics that can be used to measure performance include effectiveness (i.e. Was the user able to complete the task?) and efficiency (i.e., How much time and how many clicks did the task take to complete?). You can also collect behavioral data, such as clickstreams, and click heatmaps. The combination of these two types of output makes unmoderated remote testing a powerful user experience research solution.
Task-Based studies allow you to measure user performance on any number of website tasks. Every task entered into a study includes a task description, validation, and follow-up questions. You can validate that a user successfully completed a task automatically by page or question. While a user is performing each task, UserZoom is capturing the number of clicks, pages visited, and time spent on task. UserZoom aggregates the participant data and allows you to export to a PowerPoint, Word, or Excel for easy presentation and analysis.
Task Results Summary
Research Goal Number 4: Establishing an Industry Usability Benchmark (Task-Based Comparison Study)
Competitive user experience benchmarking is the process of comparing your website’s performance against competitor websites. When conducting user experience benchmarking, a website’s performance is judged by user experience metrics, including efficiency, effectiveness, and satisfaction. The practice of user experience benchmarking is excellent for determining how your website stacks up against the competition. By utilizing statistically significant quantitative data you can determine the relative quality of your website. Competitive benchmarking studies also can yield qualitative data that reveal the more specific strengths and weaknesses of each website.
The first step of a competitive benchmarking study is defining the specific metrics that you want to analyze across the websites. The next step in the process is creating specific tasks that participants will perform. The task-based study type allows you to implement within or between-subject designs with task randomization. Participants can be randomly exposed to any number of tasks or survey questions on one or multiple websites. The task effectiveness, efficiency, and questionnaire responses are aggregated for each website that is tested and significant differences between the websites can be called out.
Example Benchmark Comparison of Four Companies
User experience design is critical in today’s highly competitive online marketplace. This is driving user experience designers, researchers, and marketers to focus on the customer and their online experience. There is a need to gather more concrete, statistical, and geographic data, which can be obtained through unmoderated research methods.
Over time User Experience has become a complex concept and measuring it can be quite challenging. It requires that you understand customer’s needs, behavior and satisfaction. New cost-effective methods and technologies are there to achieve these goals. This method allows you to gather quantities of data sufficient for statistical testing and gain additional insight, which complements your other methods.
By adding this method to your toolkit you will be able to meet your web design goals by quantifying your data. Integrated with your current methods you will have a complete and accurate data set to define and improve your site. With a defined roadmap integrating True Intent, Card Sort and Benchmarking methods you will not only improve but also ensure the optimization of your site design.
Written by Nathan Colker, User Experience Researcher at UserZoom.