From experience to understanding: how in-house UX research helps build better products

In product teams working on digital solutions, current process models often divide the workflow into the problem and solution spaces, or into research and design, or even discovery and delivery. Whatever we call these two phases, the first step is always to understand what a user-centered response to the challenges in people’s lives might look like before we set about implementing it. First, “Build the right thing”; then, “Build the thing right.”

Even when launching a new feature, or even a brand-new product from scratch, the demand for quick, tangible results often outweighs the need to first gain a clear understanding of users and their context. In our experience as consultants, this frequently leads to a tendency to cut corners when defining the problem space. An ambitious desktop research project promises quickly available insights into user groups and work contexts; even ChatGPT supposedly already knows how the sales department of a German DAX-listed company operates. And if necessary, a reasonably recent study is consulted, or perhaps commissioned, so the team can work on the product more quickly and purposefully. Hardly anyone leaves the building to engage users outside their own corporate or project bubble; the externally gathered insights are filed away as an executive summary in Confluence. Check.

In this post, I’d like to share a few reasons why it’s a good idea for a product team to take responsibility for conducting its own research with real people. What’s the difference between firsthand experience and an externally commissioned study? How can a team establish a research process, and how can the effort involved be kept within reasonable limits? Here are a few suggestions.

Reality check

Let’s imagine the Managing Director of a medium-sized company whose employees, among other things, install solar panels on building roofs. To do this, the installers use an in-house app to plan locations, track installation steps, and check the system’s performance metrics. Our Managing Director places particular emphasis on this last point; he has high hopes for performance and diagnostics and consistently prioritizes these topics in relevant meetings. Remarks from product management indicating that there are significant usability issues with the app are downplayed. The app can be tested on the latest-generation laptop in the CEO’s office, projected onto the 75-inch smart TV screen facing the couch. At first glance, everything looks sleek. To put it bluntly, installers always have something to complain about, the real value lies in the new (and monetizable) features, according to the prevailing opinion.

Watching without being able to intervene

Now let’s imagine that a resourceful product manager has managed to secure the budget for a small on-site live test of the app with three plumbers and invites the Managing Director to observe this test remotely from his office, though without being able to intervene or communicate with the test participants. The atmosphere is tense; we’re watching a screen feed from the installer’s 10-inch job-site tablet, on which the app is running. Meanwhile, a mobile camera attached to the installer’s jacket films his face and records audio at the same time. The installer is supposed to perform specific maintenance steps on a system on-site according to the plan and document them in the app. In the process, he struggles with wind and light rain; his touchscreen gloves don’t respond quite reliably on the tablet due to the moisture, and the available wireless network at the job site repeatedly causes delays while the software loads.

Under ideal conditions, a maintenance task like this can be completed in about 20 minutes. In our case, the installer, who has already been standing on the roof for 15 minutes, is still struggling with the app’s controls. An easily preventable accidental tap has pulled him out of the context of selecting his equipment, and he has to laboriously navigate back to his maintenance checklist via a drop-down menu. The app loads slowly; an error message pops up from the bottom of the screen and disappears again after a few seconds, so our test subject can’t fully grasp the content. He was actually in the middle of checking a safety hook at the same time. Curses are heard, not just on the roof, but also in the boss’s office. Twenty-three minutes have passed; the first checkpoint is documented. At the next work site, another accidental tap leads to the help page. The installer’s strong fingers hit the wrong one of two buttons that are close together; we hear more cursing through the microphone, and our test subject eventually switches to paper and pen for his documentation, which he’ll enter into the app later once he’s in the car. That wasn’t really the plan.

Solutions need to be grounded

We can now place bets on whether the Managing Director from the example above will still be pushing for his performance and diagnostic features at the next prioritization meeting, or whether the experience has brought about a significant change in his thinking and actions.

The app’s usability issues were already on the table before the test; they had been repeatedly discussed verbally, but it wasn’t until we experienced the context of use firsthand and empathized with the installer’s struggles on-site that the discussion changed for good. And it’s not just about empathizing with the friction-filled workflow: from a purely business perspective, it’s also clear that efficiency, and ultimately money, is being lost here.

Not only was it evident that the entire maintenance process involved more effort than usual, but in the end, the installer also reverted to an earlier, more practical approach: namely, to simply ignore the app altogether. The solution, which the company had gone to great lengths to develop, felt more like a hindrance to the user working on the roof of the facility. A harsh verdict, but one that key stakeholders first needed to experience for themselves before it was given the appropriate sense of urgency.

Travel changes you

Whether an evaluation focuses on how people interact with a solution you’ve developed yourself or on gaining a fundamental understanding of the world of specific user groups, the experiences are most impactful when you, both figuratively and literally, step outside your comfort zone. It’s precisely the research journey you embark on yourself that changes you. It’s shaped by countless, lasting impressions that work on you both consciously and subconsciously. In contrast, simply glancing at, say, a postcard of the Golden Gate Bridge, filled with nothing but bare facts and figures, probably won’t have much of an effect on you. If anything, the image might even trigger clichéd preconceptions that aren’t challenged by anything.

A journey into users’ lives

That might sound idealistic given the looming milestones and tight budgets of everyday project work. But sometimes, figuratively speaking, it doesn’t have to be a two-week long-haul flight for the entire team to San Francisco just to publish a high-quality photo book afterward. Sometimes it’s enough to step outside your own building, or even just leave your desk for a moment, to visit another department and consciously engage with people and situations that challenge your own mental models. Maybe the assumptions you’ve made so far aren’t accurate at all, or perhaps there are doors opening that you didn’t even know existed.

Go where your users are

As a product manager, you should become an expert on your users’ world. The goal is to understand the people, their environment, their tasks, their circumstances, and their problems as thoroughly as possible, and to empathize with them. Take a moment to put yourself on the other side, in the shoes of the people who will use your solution.

For example, if you and your team are currently developing an app for train travelers, you might want to head down to the train station. If it’s a solution for assembly line workers, you should have gotten a feel for the atmosphere in manufacturing facilities. If it’s new tax software, you might soon find yourselves sitting in the home office corner of someone’s living room. The human-centered design toolkit includes a very effective tool for this: contextual inquiry, a method that allows you to gather a wealth of information about the environment and how the product is used in as realistic a setting as possible. Usability tests of existing solutions can also be conducted as closely as possible to people’s everyday lives or work routines. The app from the solar panel example mentioned earlier must work on the installer’s tablet while he’s standing on the roof, not on the smart TV for the boss sitting on the couch. And if possible, it should be tested on the roof rather than in the office.

Contextual research aims to be as unobtrusive as possible

Travel planning as a team sport

The journey toward a better understanding of users is, at its best, a team effort. Even though there may be dedicated researchers, involving all team members and stakeholders, beyond traditional role assignments, helps solidify and internalize the insights. For example, they can serve as observers or note-takers and provide support during preparation and follow-up.

In fact, there’s a lot to be learned even in the planning and preparation of research activities, if only through recognizing your own lack of knowledge. For example, when you actively work as a team to draft an interview guide or discuss which aspects of the software you want to examine, you can learn a great deal about shared assumptions or differing perspectives. And when evaluating and contextualizing what you’ve experienced, your insights can become even more refined and focused.

Only those who were there can tell the story of the journey

If possible, you shouldn’t skip any of the steps mentioned: neither the collaborative preparation for the research activities, nor the execution, nor the subsequent analysis and processing. All of this permanently changes your understanding of users. And this understanding then effectively carries over to all team members involved, who no longer need to be specifically briefed or convinced of what you observed in the field.

If you outsource the entire knowledge-gathering process, thereby completely bypassing the planning, execution, and evaluation, you not only run the risk of missing a lot of subtext and subtle influencing factors, but the lack of hands-on experience also makes it harder for that knowledge to take root within the team. Figuratively speaking, it’s the journey itself that changes you, not the postcard. For our day-to-day project work, this means focusing less on the final deliverable of a study and placing much greater emphasis on the shared experience and the collaborative development of understanding.

A case for connecting with real people

Here’s a quick summary of the benefits your team can gain from conducting your own research:

Gaining deeper insights

When you’re out in the field yourself, you’ll have a more nuanced sense of potential influencing factors, and you’ll be better able to probe deeper or investigate further when something piques your curiosity. Sometimes, in the context of actual use, you’ll discover doors that lead to other doors, behind which undiscovered knowledge lies dormant.

Developing empathy

If you can personally sense the pain points in your users’ everyday lives or share, even to some extent, in their moments of relief and success, you’ll be better able to empathize with the people involved. This way, they become real, flesh-and-blood people to the entire team, not just role descriptions in a text document. Now, solutions can be more carefully tailored to real needs.

Question your own assumptions

Through direct contact with people, you can specifically test and refine your own assumptions. If you work with personas, for example, this is a great way to reflect on the extent to which a human experience fits into existing concepts, or doesn’t. It’s important to gather as diverse a range of impressions as possible and not always talk to the same people.

Strengthening team ownership

When the entire team is personally responsible and actively engaged with users, a shared perspective emerges, and with it, a shared sense of responsibility for their own solution. If, for example, an unpleasant task arises, such as redesigning the main navigation, the entire team is more likely to go along with it if everyone has actively experienced that the old version didn’t work for the target users.

Establishing an efficient research process

Especially in the B2B sector, it can be helpful to build and maintain your own pool of potential participants for surveys or tests. Once this contact is established, you can usually reach out to these people easily and on short notice. Experience shows that when people are likely to be affected by a solution, they’re quite willing to participate in the design process, for example, through collaborative design workshops.

Clarity about what we don’t know

When you actively plan and conduct the research yourself, you make your knowledge gaps explicit from the start. What do we know as a team, how confident are we in these assumptions, and where do we still need to shed light on our understanding of our user groups? Discussing these questions is extremely valuable and helps sharpen our shared understanding.

Developing tailored solutions

When the product team itself has become an expert on the context of use and the people involved, many small decisions made in day-to-day project work are more effectively guided by this understanding. The solutions built for users are more accurate and sustainable from the outset when the team consistently transforms assumptions into tangible knowledge.

When should I seek outside help?

Following this impassioned call to take as much control as possible over the experience and learning related to the usage context, it can of course be helpful to seek outside support in certain cases. The request here, however, is to remain actively engaged in your own research interests, even when outsourcing data collection, and to bring the externally generated insights back into your own knowledge base and team in as vivid a way as possible.

Of course, this also applies if you’ve already clearly divided your in-house UX department into design and research teams. All the better: this allows you, as designers, to get even more involved and actively shape the knowledge transfer with your research colleagues.

Reasons for outsourcing this process may include, for example, when large-scale studies involving a large number of participants, perhaps even across languages and countries, are planned, and recruitment alone represents a considerable effort. Individual interviews may take place in different time zones or require native speakers to moderate them to ensure they’re conducted properly. Individual studies can also become highly specific and demanding from a technical or subject-matter perspective, requiring a level of support that would completely overwhelm your own team. As mentioned above, the key here is to process results, insights, and experiences in such a way that they become experiential knowledge for the team. Effective storytelling is particularly helpful in this regard.

A human-centered approach as a cultural mindset

Between large-scale studies involving thousands of participants and the other extreme, namely stubbornly self-referential development without looking outward, there are a multitude of effective and actionable methods for testing your own ideas against reality. If you’re willing, you can simply start talking to users without worrying about making a mistake. Sometimes it’s also worthwhile to have a preliminary discussion with your in-house sales or service department and to build a “friendly customer pool,” a group of accommodating companies whose selected employees agree to maintain ongoing contact to evaluate a solution and help refine it. Any initial contact, no matter how low the barrier to entry, is always better than no contact at all. In this area, there’s always room to learn and improve. Read a few good books on research methods or seek guidance on how to handle user feedback effectively.

Testing doesn’t have to be expensive

In practice, we often encounter the misconception that usability tests, for example, are quite time-consuming and costly, and thus pose a hurdle in the development process. But once a setup is properly established and with manageable preparation and follow-up time, these tests can often be conducted right in your own offices. Not only does this contribute culturally to a visibly more human-centered mindset within the company, but it’s also somewhat fun and, as a team event, can bring stakeholders and active team members together while breaking down long-established ways of thinking about your own customer base.

Those who can make use of this mindset, and embed it not just once, but permanently in their approach, have a good chance of establishing a vibrant culture of learning and also strengthening the team’s sense of self-efficacy. In the end, users benefit, as their pain points and moments of success become more deeply ingrained in the team’s mindset. Their needs are visibly incorporated into the development process, and ultimately, the business also benefits from satisfied customers.