Malini Leveque, VP of User Insights and Research at SAP, shares how she’s leading product teams throughout the organization using big data, analytics, and live user insights to drive actions that generate real business value.
Malini Leveque is VP of User Insights and Research at SAP. A huge multinational software company, SAP has over 100,000 employees. In this episode, Malini shares how she’s leading product teams throughout the organization using big data, analytics, and live user insights to drive actions that generate real business value.
ALFONSO DE LA NUEZ:
Welcome to UXpeditious! A show that brings you quick, insightful interviews with design, product, and UX leaders.
DANA BISHOP:
In each interview we dive into how UX research impacts user insights; shaping the design and business strategy of some of our favorite tech tools and products.
ALFONSO:
I’m Alfonso de la Nuez, Chief Visionary Officer and Co-Founder of UserZoom.
DANA:
And I’m Dana Bishop, VP of Strategic Research Partners at UserZoom.
ALFONSO:
And we are your hosts.
On today’s episode, we’re talking with Malini Leveque, VP of User Insights and Research at SAP. Malini shares with us how access to lots of data is directly impacting product and business decisions.
DANA:
Malini has tons of experience leading design-thinking transformation, user research, and UX strategy at companies including Hewlett Packard, and Oracle - prior to joining SAP.
ALFONSO:
Hey, thanks so much for joining us, Malini. It's a pleasure to see you and to hear from you today. Can you please introduce yourself?
MALINI LEVEQUE:
Certainly. Firstly, I'm delighted to be here. I'm Malini Leveque, and I lead the Global Product research and Analytics Practice at SAP.
ALFONSO:
Awesome. It's amazing to have somebody who leads research at a company as big as SAP. I know that there's been exponential growth when it comes to experience research. You recently actually wrote something like this, right? With a massive expansion in Cloud-based application performance monitoring, APM, product teams throughout the organization use big data, analytics, and live user insights to drive actions that generate real business value. We would love to hear all about that.
MALINI:
Let me start with, first, what does APM or application performance management really mean? This is really an essential part of DevOps and AIops tooling that helps companies monitor, measure, and optimize the performance of their applications and IT infrastructure. Specifically, APM can assist with app availability, transaction times, and other performance issues that could impact the user and the customer experience.
Now, I'd say that APM really captures various performance metrics from multiple systems, and that's where Cloud comes in, including infrastructure, which is your servers, network storage, and the apps and micro app services to create this holistic view of solving problems and improving user experience. Now the question is, what is the value of application performance management, data analytics, live user insights, big data and all these buzzwords in some sense? How do they really apply to a business or a user?
The bottom line today is that as users and customers, we expect a near-flawless user experience. Luckily, today we've got these APM tools, and I'll just throw out a few examples like Dynatrace or Datadog, AppDynamics that provide the DevOps capabilities to spot these anomalies earlier and predict behavior, and they go ahead and enable that corrective action that is required to mitigate downtime. With these tools, all product stakeholders benefit. For designers, you can get a 360 degree view of user experience.
For example, app use through funnel analytics, session replays, drop-off rates, journey analytics across all the digital touchpoints to measure the impact of customer behavior and business outcomes. For developers, they can get a code level visibility into issues and crashes to speed up that triage and stay ahead of performance issues. Then, when you're looking at, say, product management, they can prioritize their product features and plan roadmaps based on real user sessions and user data. Frankly, today I would say that using big data and data analytics to inform priorities and decisions is just becoming a baseline expectation for all product stakeholders and leaders.
ALFONSO:
It's becoming strategic and it's becoming... what's the word I'm trying to use? You cannot live without it basically to make decisions, or you could but you could be flying half-blind.
MALINI:
Well, not only are you flying half blind, but just your ecosystem around is immediately going to ask for the evidence. Strategy has become very evidence based, and that evidence is based off of all these data points that we are collecting, which the bar is getting lower and lower, and you see these data points are being collected across all the different functions in product development. When we think about user experience, one doesn't really think of DevOps tools, but for an end user, if a site crashes, design is not the first thing they're thinking about or click-throughs, they first want it functioning.
DANA:
The application performance monitoring or APM is interesting. I'm kind of wondering about how UX figures in, so traditionally UX research, there are two big types of research, qualitative and quantitative. How do you think the boundaries between those two are blurring?
MALINI:
That's actually a very good question. When I think about mature research teams, they just understand the salience and the strength of qualitative and quantitative research methodology, and the value of mixed method research to articulate this full story. Both these research methods provide a deeper understanding around customer needs to ultimately drive better, more informed decision making. In our research and analytics teams at SAP, over time, deeper analysis of numeric data, which is your click streams, log on times, durations, are leading to insights into user behavior patterns. Similarly, we found that analyzing vast quantities of qualitative data from our surveys with text intelligence analytics enables our data scientists and research to calculate user sentiments on usability and usefulness.
Ultimately, businesses and development teams benefit with deeper insights from both these methodologies and beyond. I'd say that today analytics from our insights teams at SAP, it's a continuous triangulation of larger and larger growing data points, KPIs, and data sets.
DANA:
I know that you have implemented passive user triggered surveys to capture some information. How does that also sort of work with the larger monitoring and research that you're doing?
MALINI:
The passive... In fact, it's interesting because we have, right now as we speak, have taken those passive user surveys and really moved these surveys that were primarily pool surveys, which clearly are biased because the user is clicking on a button to actually initiate the survey. We moved them into intercept surveys, which are time triggered and conditional surveys. Within a particular flow, we can collect user data without telemetry right now through surveys by intercepting user flows, as well as showing surveys based on time. A lot of our satisfaction scores, the PSAT, the UMUX, those surveys are being collected through intercept surveys.
DANA:
In my career, I've often had people come from the data analytics side and say, "Hey, we know what's happening, but we don't know why," especially when you're trying to identify and solve a problem that might be happening, a drop-off or slow load times or whatever it might be. But, you can see that people are abandoning maybe things in a cart or something, but you don't know why. It sounds like you're sort of filling in that gap and getting some of that why behind the what is happening with these intercepts.
MALINI:
Dana, that is correct, but I would say that the why is really answered through the traditional research. When we are looking at quantitative research, behavior research, that really is putting the finger on the human pulse through your focus groups and the traditional research methods as we know it. It is that balance between doing some traditional research, and yes, the text fields can augment, but I would not rely 100% on it, even though we are delighted to have this focus on telemetry and quantitative data as well as the surveys as a mechanism to collect some of this quant/qual triangulation through the pretext field, but that does not replace the traditional research practice within our team.
DANA:
That's what I was wondering. We obviously have growing access to all this data, collecting data, and analyzing it. How do you think that is impacting the UX research discipline?
MALINI:
In short, I'd say that the definition of the research practice is completely transforming today. At SAP, our research practice, which we've called our team the product research and user insights team, and it is really building a practice that has expanded our expertise from traditional research into a lot of the data analytics telemetry and just dealing with big data, big data insights.
What we've gone ahead and done is that, within our team, built these proficiency pillars that'll give you some insight into do you know how we're expanding the research practice? These proficiency pillars are really built to create a very modern user-centered culture of curiosity. And I keep using the term... it's about building this culture of curiosity that is going to inspire the product innovation and grow customer value, but this has to happen across SAP. Yes, one of the first pillars is our traditional product research practice, which continues to work with product teams and focus on building a user research practice that informs your product planning, design and development with users and market insights.
This pillar, just as we know it traditionally, it works with product teams to deliver that data driven user insights to improve the current end-to-end product experience. Now in addition to that, we have prioritized investments to build an analytics and insights technology practice to provide best-in-class tools, instrumentation, telemetry, and enablement to measure this in-product user experience. Your surveys are one of the many pieces of telemetry. All of this with this common goal to empower product teams to continuously improve their Cloud experience with analytics, best practices and these actionable insights.
In 2022, this year, we have focused a lot of our investment on developers required for the instrumentation and the technical enablement, and our data scientists to build the PX scorecards, the product experience scorecards, and analyze the KPIs to empower the product teams. This is across SAP, empowering them with the data informed user insights.
I would say that the analytics, both on the instrumentation telemetry development as well as the data science and the ML is our biggest skill and tech investment. That is growing this practice of research, and I'd say extending the definition. Another big pillar for us along with ML, AI, and instrumentation has been of the research ethics practice. This team really focuses on establishing those best practices for product inclusion, equitable AI, trust and transparency, GDPR compliance, sustainability principles.
ALFONSO:
Very important.
MALINI:
We are working on setting up these frameworks to guide teams across SAP on these foundational topics. You see, we've got... the traditional research practice expanded it and created another pillar on just analytics and insights technology. Then, the third pillar that is focused on research ethics and ethics practice, looking at... interestingly, legal and GDPR compliance is a very big part of this practice as is the trust and transparency. Then, to scale the research practice across SAP and deliver on our corporate mission, we have built a robust research foundations and operations practice, so that's its own proficiency pillar. Here, the teams operationalize a lot of the research practice with tooling, onboarding, and managing intelligent insight hubs, dashboards, research repositories, and building the customer panels to empower our user research community to share and thrive.
You see, I would say that researchers have always been very comfortable working with big data, but now moreso with all this ML and AI, they need to look for the blind spots, they need to be more inclusive, double click on trust and transparency. I would say that today our researchers need to have more confidence to advocate for the user, the customer, the human voice, and all of humanity in the face of all this machine data.
ALFONSO:
It's been wonderful, wonderful, wonderful to have you, Malini, with us. Such an honor to have a leader and a large technology company like SAP explaining and sharing with everyone the growth story of research and user insights, quantitative, qualitative, just a fascinating story and what an amazing job you have. Thank you so much for being with us today.
MALINI:
Thank you, Alfonso. Thank you, Dana. This has really been a delightful conversation.
DANA:
Thank you.
ALFONSO:
That was Malini Leveque, VP of User Insights and Research at SAP.
You already know at UserZoom we thrive on collecting user insights to improve digital experiences, and the same is true for this podcast. We would love to hear from you, our listeners, on how we can improve the show. Please follow the link in the episode description and take our quick survey. We’d love your feedback, as well as any recommendations for future guests or topics.
DANA:
Thanks for listening to UXpeditious. Make sure to continue listening to our new episodes each week for quality insights from UX industry leaders. If you like what you heard, help us out by rating and reviewing the show on your favorite podcast platform.