The importance of using data to validate qualitative research findings
20 years ago I had a vision that every company will become a ‘digital experience company’, and it took the global pandemic to accelerate us towards that reality. Today, 70% of companies either have a digital transformation strategy or are working on one. Digital is the primary currency of customer brand and product interactions, and the quality of the digital experience is the primary competitive differentiator.
We’re about to end 2021 and the effects of the pandemic will leave us with a more digital, remote, yet interconnected and collaborative world. It will lead businesses to accelerate their digital transformation and get their act together when it comes to delivering great Digital Experience (DX).
The evidence is in these recent stats:
Bottom line: Investing in great digital experiences is great business.
My personal experience over these two decades, however, has taught me that designing and delivering great DX is hard. It doesn’t just take a village, it takes a change in culture, it takes resources and new skills, a shift in how we do business altogether, at all levels of the organization.
Every day more and more organizations are realizing the business value of great design and great user experience. Organizations are investing in hiring UX talent, UX leaders, designers, researchers, content writers, and more. Design-led businesses are winning in this Digital Era.
Simply hiring a few designers is not enough to win in this digital era. Talented Designers and Researchers need UX Leadership support across process, strategy, tools, budgets, and outcomes.
Success in the digital experience economy requires an integrated and organization-wide approach to understanding and measuring the digital experience. We call this Experience Insights Management (XIM), the combination of processes, strategies, and technologies that allow businesses to manage the gathering, analysis, and sharing of experience insights across all digital touchpoints.
Ultimately it’s about putting the end-user at the very center of the decision-making process. So here are six steps to delivering exceptional digital experiences via Experience Insights Management (XIM) in 2022
Most organizations start with design validation research and aspire to move higher up the UX maturity curve. Building the right foundation is critical. Usually, UX research, at first, is concentrated on design validation in agile sprint development. When UX research establishes itself as a critical activity, it's time to build that right foundation via Research Ops.
We are seeing a growing trend towards scaling processes, workflows, research standards, and templates. I always say: Researchers should do less research and more decision-making. A Research Ops program can help you streamline these UX activities, and allows you to manage budget, centralize customer and user insight and build UX advocacy across the entire organization.
To properly test and measure digital UX and CX, you need to combine qualitative AND quantitative research methods. Surveys conducted in isolation may not always be the right method and companies who simply do a handful of usability test videos and call that ‘user research’ are not seeing the full picture of their experience. You need a multi-method approach to have confidence in major design decisions. Companies should be rigorous about this!
And the good news is that it's now getting easier and cost-effective to execute mixed-method research via software platforms that automate a big part of the process of collecting actionable insights and help scale to fit today’s agile development demands. Be it simple qualitative and quantitative research, such as think out loud usability tests (moderated or unmoderated), click tests, 5-second test, surveys, card sorts, tree tests, discovery interviews, or higher sample UX measurements, the cost per study is much lower and time-to-insights much faster these days vs five years ago.
As NNG states, “The best approach is to use multiple research methods, so the limitations of one method are mitigated by data from another source.”
I get it, it’s a tough job. Participant recruitment alone is a huge part of the whole ResearchOps equation. No matter how great the research team, the methods used and the technology that enables research studies, without the right participants, the project falls apart.
Precisely because of this, I strongly suggest companies be extra careful and work hard to invite the right people to the study. Quoting NNG once again, “While finding the right participants for a user-research study can be challenging, you’ll get more meaningful insights if your study participants have the same behaviors, interests, and knowledge as your actual users.”
I bought the reclining sofa I have in my home office about 10 yrs ago. Today, its functionality remains 100% the same. But one quick look at my iPhone and I have 28 apps that need to be updated (and I do updates every weekend). Digital products are live products, constantly evolving. To be truly user-centric, we must remain close to the end-user throughout both pre-production and post-live stages. And again, measure and monitor the quality of the experience with a mix-method approach.
What we commonly call user testing, which is a qualitative research method, is a great method to identify usability issues, validate designs and get quick feedback with a small sample of users, but it only tells us part of the story. To get statistical significance and conduct benchmark studies, we use continuous measuring programs (and metrics such as our own QXscore), to measure both attitudes and behaviors, enabling business leaders to evaluate performance and for stakeholders to demonstrate the impact of UX and identify opportunities to improve.
Back to the reclining sofa example, the people working on the design and development of the sofa user experience (Product folks) were certainly different than the ones doing the post-experience customer research, or customer experience measurement (Marketing folks). But in digital products, these teams must work together. Today’s UXers are definitely interested in obtaining insights once the product is live and so customer AND user experience research is going to be part of their roadmap.
Marketing teams are usually the ones in charge of this and are users of web analytics tools, C-Sat and NPS. Product teams are where you’ll commonly find Product UX Designers and Researchers. From personal experience, I know these two teams all too often work in silos, sometimes overlapping work, which is obviously inefficient and can lead to confusion. But fortunately, I’m already hearing a lot of Experience leaders talk about ‘Total Experience’ and are moving on from the U or the C. We’re all in it together. That’s why I can see the convergence of CX and UX happening soon, motivated in this case by the pandemic that’s leading companies to go digital.
Rich valuable insights collected via research cannot die in PowerPoint decks. While it's critical to answer immediate design or business questions via research, it's equally critical to ensure that broader trends are captured, made accessible, and shared across the entire company.
Like ResearchOps, insights management is a team sport that requires collaboration across Product, Marketing, Designers, Researchers, and the Engineering team. The real value will be in the time to insight and decision (not just time to data). This means ensuring insights are organized, accessible, shared, and used across the company. AI and Research Ops solutions (like our own EnjoyHQ) will be the biggest differentiators in the future, as you scale your Research team and roadmap.