How stakeholders should fit into user testing sessions
Having them just watch can be more effective than notetaking
One of my biggest mistakes, when it came to conducting user interviews, was when I was trying to get stakeholders involved in user research.
The stakeholder somehow joined the same link as the participant, and he happened to be the boss of the participant.
Because his name was visible on the call, our poor participant started to advertise how good the current product was.
The stakeholder and the rest of the project team had agreed that the product was in need of a re-design, but the user didn't know that. All he saw was that his boss was suddenly on the line, so he tried his best to stick to the official workflow rather than what he normally did.
In the end, we had to throw out that interview. But that interview got me thinking about if stakeholder notes are always the best solution.
Sometimes, it might be enough to just watch the users.
Understanding user research from the outside
When I was first learning UX research, I was shown an old video of people trying to figure out a copier and struggling.
Two people are fumbling around with a printer and scanner, trying and failing to figure out basic functions.
What's surprising is that these aren't average employees: they're highly trained Computer Scientists.
And seeing that struggle up close may be all that your stakeholders need to be convinced.
Having them get distracted trying to take notes, or follow a facilitator's guide, might be counter-productive.
Seeing users struggling, even with help from a facilitator, might point out fundamental problems they never saw before. Not to mention that getting answers directly from the users might be all stakeholders need to become advocates for UX.
But there’s one thing that you should first understand: for your stakeholders, sitting in on user research isn’t a user interview.
It’s a watch party.
Understanding watch parties
Even though people may say that “everyone is a designer,” learning to take good user notes isn't an easy task. Many of your stakeholders might not have the training or background to write down what UX is interested in.
But rather than trying to train your stakeholders to notetake with like UX researchers, there's a much easier solution. Have them just watch.
While it might be nice to get detailed notes from a business perspective, your stakeholders might not know what to expect with user research sessions.
But just having them watch and observe what's going on can be a very easy way to turn them into a UX advocate.
And having them act as an advocate for UX, after seeing how the users struggle with the tool, can be more valuable than any notes you get.
So here’s how to set up a watch party to make it easy for your stakeholders to observe what's going on.
How to set up a watch party
The first thing that you need to do is plan, plan, and plan.
Some of the most important stakeholders to convince are also those with the least amount of time, so you need to coordinate to avoid wasting time.
As a result, everything from the time and place all the way down to crafting e-mails to invite stakeholders should be planned ahead of time.
But there are a few things that you should keep in mind to make sure that your stakeholders can get the most out of your user testing sessions.
Support alternative forms of watching
Sometimes, you can't always get everyone on the call at once. But, with remote conferencing tools such Zoom, Webex, or even Skype, you can still get them to watch testing sessions if they're not directly on the call.
But there are also other things that you can consider, such as creating a highlight reel for your stakeholders.
The main purpose of this is to get them to pay attention to the users and draw their own conclusions.
But how can you make sure that they’re paying attention to the user research session if they’re not in the room with you? By scheduling a debrief.
Schedule some time to debrief:
One of the best places to build shared understanding in research is through the debrief.
These usually occur immediately after user testing, and it’s a good way for different researchers to talk about what they saw during testing.
This allows people to catch things they otherwise hadn’t noticed, ask questions about what was said, and consolidate all of their different approaches into a single unified feeling.
So having something like this with your stakeholders, soon after an even asynchronous viewing, can provide a lot of clarity to any of the questions they may have.
Regardless of how familiar the stakeholder is with a project, they may not know exactly what the user is doing or why.
Or they might have seen something that UX is not entirely familiar with, but they’re not quite sure how to explain it.
Having a debrief with them can build a shared sense of understanding with your stakeholders, which can have profound effects later on.
But to do that, we also must clear up an unusual argument that sometimes occurs.
Persuade the stakeholder that what they’re seeing is not unique:
One of the arguments that I’ve heard before, after debriefing with a stakeholder, is that what they saw was somehow an exception.
For example, these users were particularly dumb, but normally it shouldn’t be like this.
I’ve come to think of this as a stakeholder’s defense mechanism: they might have had an ideal use case in their head, and seeing users struggle with that is no doubt a blow to their ego.
But one of the things that you have to do is convince them that what they’ve seen isn’t that unique. You could mention other (anonymized) participants that you’ve seen this happen to or possibly show them other preliminary research.
The point of the watch party is not necessary to get detailed notes from the stakeholders: it’s instead to get the stakeholder to consider that not everything may be rosy from the user’s perspective.
And by doing that, you can get an ally that can advocate for UX on your side in meetings that you do attend (and even meetings that you’re not part of).
So if you want to invite your stakeholders to take part inf your user research, think about it from their perspective and consider a watch party.
That may be one of the most productive ways of getting stakeholders engaged in what you do.