Measuring Impact Without a Ruler
Understanding the value of UX research beyond measuring ROI
A few years ago, I transitioned from project management to a new role called the Director of Client Impact. We were very interested at the time in the measurable impacts of our UX research projects, so we decided I’d join kickoff calls and lead a discussion about it. We were surprised to find that our clients almost universally had no measurable impact in mind for the work they were asking us to do.
That is not to say they didn’t have goals. It’s just that, as we quickly learned, it’s often difficult to draw a direct line between UX research projects and measurable outcomes. This is especially true if the product in question is not a mass-market product that allows you to test design changes with large groups of people. But even with products that are used by a lot of people, it can be very difficult to determine if measurable changes like increased sign-ups, CTA clicks, purchases, or whatever the goal might be is a direct result of design changes that were made based on research findings. There are just too many steps and potentially confounding variables in between.
So, why do UX research?
There are a few great reasons to talk to people about using your product or service. For one, the more you do it, the more likely you are to design products that meet people’s actual needs rather than your guess at what their needs might be. In the early days of the Internet, people released products that met their imagined users’ needs all the time. The late 1990s and early aughts were chock full of web applications that were very clearly designed by the engineers that wrote the code for them. As many of you probably know, that period is now a cliché for non-user-focused design. If you went to any UX conferences in the early aughts, you probably saw this cheeky t-shirt more than a few times.
Nowadays we finally understand that in order for people to happily use our products and services, we need to a) understand the problems that they actually want us to solve, and b) understand the way they think about the problems. This is the way to deliver experiences that make sense. Steve Krug said it best when he titled his seminal book on user experience Don’t Make Me Think; the goal is to solve people’s problems in ways that feel intuitive to them. And that’s why it’s worthwhile to do UX research even if you can’t draw a clear line between any particular design change from research findings and a measurable result.
Another related but slightly different reason to do UX research has more to do with the product stakeholders. Everyone from executives to product managers to designers and others involved in the creation of products can get passionate ideas about solutions to the problems they are trying to solve with their products. Some might view their professional reputation as being at stake, or they might have other reasons for being particularly committed to a feature or design element. Whatever the reason stakeholders are so committed to the solutions they’ve helped to design and implement, there is nothing like watching someone trying to use a flawed solution to cure people of their commitment to it! And, as my colleague Kristy Knabe recently wrote, a good UX consultant can help teams turn these less-than-cheery insights into opportunities to improve the product.
There’s no question that well-designed, intuitive products that solve real problems are the ones that grow and meet measurable goals. And it’s hard to argue with the idea that the more UX research teams do, the more likely they are to understand people’s problems and produce well-designed, intuitive products to help solve them. However, focusing on an elusive numerical impact now seems beside the point, and I’ve shifted my focus from trying to draw straight lines between research insights and improved metrics. Instead, I assess the impact of our work in much the same way we do UX research: I ask our clients if we solved the problems they were trying to solve in a way that felt positive and met their expectations.
