How to get the most out of an interview with a subject matter expert
Understanding the difference between expert and user interviews
You may be losing a lot of useful information if you treat an expert interview like a user interview.
That’s one of the lessons I learned early on in my career.
On one of the first interviews I ever took notes for, my mentor seemed to veer off script, asking questions that weren’t part of the testing process and seemingly spending most of the time chatting about something else. I voiced my concerns during the debrief, only to be told that this person was one of the foremost experts in the field we were studying.
It was only further along in my UX career that I learned that was an Expert Interview with a Subject Matter Expert (SME), and more importantly, that it was best that we didn’t do the normal user interview. And that lesson only sank in recently, when I squeezed in an Expert Interview in the middle of user testing. Expert interviews have a different purpose than user interviews, and to treat them otherwise is to lose a lot of potentially useful information. To understand why, I’ll talk about the environment where I learned UX: Healthcare.
Catering towards Subject Matter Experts
For both my UX training and the early part of my career, I was in Healthcare UX. I interviewed surgeons, hospital administrators, and cancer registry personnel during this time.
Part of this process involved catering to their schedules. The pathway to interview a surgeon or videotape a user in a surgical environment required administration forms and months of waiting, and I’ve done user interviews at 4am to accommodate surgeons.
I introduce this to say that I quickly learned to get as much data as possible during these Expert interviews so that we could revisit the data for answers.
But while expert interviews offered a lot of valuable information, they also differed from other user interviews that I did: while they could give me detailed explanations about things that they were comfortable about, their feedback regarding other users was less useful.
As a result, interviewing someone with 25 years of experience would result in them imagining the experience for new users from their viewpoint. I only learned later on that this can lead to something called self-design.
The dangers of self-design with SMEs
One of the things many SMEs are likely to do is to do what’s called self-design: this is the process of utilizing their experience to make design decisions.
This can sometimes work well, such as when Apple initially designed the iPhone based on what they wanted the phone to do.
However, two criteria need to be fulfilled for self-design to be effective:
The SME must still use the product daily
The SME must be like the target audience
Often, the SME’s have joined the team because they no longer are doing the job: in this case, the decisions that they want to make with a product are based around experiences that may be decades old. Other times, the SMEs are now in more senior roles, such as Managers or Administrators, while your target audience might be entry-level technicians. And there can be oddities that you run into around this expertise as well: for many fields (especially healthcare), it may seem weird that you’re trying to seek out people with only a few years experience in a subject rather than the most senior surgeon with 20 years of experience under their belt.
But we can make great use of expert interviews if we just remember one key phrase: Expert interviews are a planned, formal, procedural meta-interview.
How to interview SMEs
Jared Spool introduces a better way of talking with SMEs: rather than using their expertise to make design decisions, it’s often better to treat them as a cultural guide and translator.
This is an approach that’s much different than an actual user, but it’s something that can provide immense value to your project. However, to make use of their value as a guide, we need to break down that phrase.
Plan: Send them questions or documents ahead of time
One of the other things that you have to keep in mind is that what you’re trying to achieve is not the same as with other user interviews. With a typical user interview, you want to get their unbiased view of a prototype, tasks, or workflow by having them try things in the moment.
With an expert interview, however, you want them to think in-depth about what you’re asking them, which requires time. Obviously, you wouldn’t send them everything ahead of time (because you still want an unbiased view on some things and you don’t want to overwhelm them), but sending them some things (especially things they might need to look up like policy or business workflow) will make sure that the time that you spend with them is productive.
Formal: Show polished design artifacts or prototypes
While it may be tempting to pick SME’s brains about everything, there’s a catch: you have to make sure that everything you show them is something they can provide feedback on. And Experts can’t serve as replacements for your users. As a result, certain things you’d normally do with users, such as showing a sketch of a design a design alternative and asking their opinion, tends not to be that useful.
In fact, it can be harmful: overwhelming experts with unfinished minor designs can be a quick way of ruining their objectivity towards the product. If that’s the case, what should we show them? The simplest answer is showing them things as they relate to the procedure.
One of the most useful things that you can ask an expert is where people run into problems. They may not have the most up-to-date knowledge on a product, but as someone who has done something more than anything else, they can give you insights into how things are done.
Asking them about processes or workflows is one of the areas where experts can explain many of the things that they know. Even better, asking them to sketch a diagram can provide your team with a resource that can aid the team in the future.
And this relates to the last element of expert interviews: the meta-interview.
Meta-interview: Understand the world outside of user testing
It’s always a good idea to talk with SME’s, but they can give you context-specific information depending on when you talk with them in the UX process. For example, if you were interviewing them early on in the process, you might want to build a workflow with their help and understand why the current system is how it is.
Your SMEs might have special insight into the best practices of the industry, regulations, policies or the history of what went into how things have ended up the way they are.
If it’s later on in the process, you might want to make use of their translation skills. If you’ve been able to interview other users and have gotten some feedback you can’t quite understand, you might want to ask about this. One of the most common examples is cause and effect.
Your users might speak about effects, such as working with whatever the system generates, but experts might understand the cause of why the system generates what it does. Experts may also be able to explain how the product that you’re working on interacts with other systems, legislation or organizational policy, and other factors that you might not be aware of.
Subject Matter Experts vs user interviews
Subject Matter Experts are sometimes a rare and valuable resource that can provide you with a ton of knowledge that can help you with a project. But making use of their knowledge requires thinking differently about the interview process. First, you must understand what the purpose of an expert interview and how it differs compared to a user interview. But doing so can yield you insights that you might have never gotten otherwise. So if you have the chance to interview an expert, take a second to think about your interview process and what you plan to ask.
You might find that a little preparation can yield a ton of valuable insights.