UX strategy and user research with Jared Spool

A transcript of of UX Podcast. James Royal-Lawson and Per Axbom talk to Jared M. Spool about UX Strategy and the need for user research.

UX Podcast
Nov 25, 2016 · 37 min read
Photo by (CC BY-NC 2.0)

Jared talked to James and Per on in December 2014. Jared is the founder of User Interface Engineering (UIE) and a co-founder of Center Centre.

Transcript

[Music]

James: Hello and welcome to UX Podcast, balancing business, technology and users every other Friday from Stockholm, Sweden. I’m James Royal Lawson.

Per: And I’m Per Axbom.

James: Hello again.

Per: Hello again, James. You doing all right?

James: I’m all right. And you? It’s like we haven’t seen each other for ages. It has been almost two weeks.

Per: Yeah.

James: To our listeners. It really is like that.

Per: Yeah. Some of them listen to several in a row, you know. Listening patterns are different for different listeners.

James: Oh, right. Yeah. I shouldn’t make presumptions.

Per: Because we’re not live.

James: And I shouldn’t make presumptions about my users without actually researching.

Per: No, we should actually research some of that before …

James: Yeah. I let me own preconceptions and biases take over my decision-making.

Per: You did, didn’t you?

James: Bad UXer.

Per: Happens to the best.

James: Yeah, it does.

Per: We’re interviewing none other today than Jared Spool. I think most of our listeners will know who Jared Spool is.

James: Well, I don’t know. We do have some listeners that are outside of UX a little bit.

Per: Yeah.

James: So it’s worth giving a little bit of background about who Jared is.

Per: Well, he’s like an icon in the usability world. He’s a founder of user interface engineering.

James: UIE.

Per: Yeah, he founded that in 1988. But he started working with usability and technology even before that in 1978, according to this Wikipedia page.

James: Exactly. He’s one of those ones that has worked with stuff before it was called what it is called now and before it was called what it was called before it was called now.

Per: And he worked before it even happened.

James: Yeah.

Per: Yeah. And that user interface engineering is also the host of a conference called The UIE Conference.

James: And they also have the Braincast Podcast.

Per: A Spoolcast!

James: A Spoolcast is what he started out as when Jared had his first podcast in 2006. Then it was Spoolcast.

Per: OK.

James: Now Spoolcast I think is just a category in the Brain Sparks.

Per: Brain Sparks, right.

James: Yeah.

Per: Which is an excellent show.

James: Yeah.

Per: I listen to it a lot.

James: Yeah. And well, do you have anything else to say about Jared? I was going to say a bit about why we’re interviewing.

Per: Yeah, go ahead.

James: What happened about — oh, in November a few weeks ago — yeah, a few weeks ago. Jared tweeted a tweet and this inspired us to talk to him and he happily said yes. So, well, we’re going to chat to him about this. I’m kind of avoiding saying what the tweet was because well, I thought we can –

Per: Yeah, we will mention that. We will probably have to remind him …

James: Because he won’t remember what he tweeted. I don’t know what I’ve tweeted. So I don’t think Jared does.

Per: And just before we will ring him, today’s show, it’s sponsored by UserTesting. Peek by UserTesting is the easiest way to see and hear videos of real people using your sites or apps. Visit UserTesting.com/uxpodcast for a free Peek usability test and find out how you can give your users an experience they will love.

James: They will be delighted. Let’s give Jared a ring.

[Music]

Jared: Hello.

Per: Hello Jared.

James: Hello there!

Jared: Wow, handheld mics!

Per: Yes!

James: Yeah.

Jared: You’ve got the beats going.

James: We’ve got too many cables. That’s what it is.

Jared: I expect you guys to start rapping.

James: Oh, you mean …

Per: Yeah, right?

James: I realised I’m coordinated. I’ve got the red mic and the red headphones.

Jared: I noticed that. You are full accessorised with a maroon shirt.

James: Yeah. Well, actually, yeah, it’s — you’re American. You won’t recognise this. The owl on the T-shirt is actually the owl from the BBC microcomputer, which was my –

Jared: Yes, of course, of course. I remember that.

James: Yeah, my first computer in 1982.

Per: I didn’t know that.

James: You didn’t know that?

Jared: Here I thought it was just a starting point for John Conway’s life simulation. Wow, that dates me.

James: Yeah.

Jared: That makes me old.

James: Don’t get started in the old thing. So every project we’re doing now, I seem to be working with younger and younger art directors and kind of projects now just seem to work into storytelling with James or I kind of — relate more and more kind of stories from the past and realise that these 22-year-olds I’m talking to are straight out of school. I have no idea about some of these things.

Jared: Yeah. No, that’s my life. I’m slowly turning into Don Norman.

[Laughter]

James: Maybe that’s how we all end.

Per: I love that.

James: It doesn’t — the world would end with a big bang and us all turning into Don Norman.

Jared: Yes, yes, it’s just, you know.

Per: I’m so happy you wanted to do this Jared with us.

Jared: Oh, I’m a big fan! I’ve been listening to you guys for years.

James: Oh, excellent.

Per: Really? Wow.

Jared: Yeah.

Per: That makes me blush actually.

James: Oh, yeah.

Jared: It’s not unusual — so I will admit I often listen to you at the gym. It’s not unusual that I find myself either mumbling under my breath or stopping my routine to take some note about something I want to write about later because of something you’ve said.

James: Oh, excellent.

Per: Wow, fantastic.

Jared: Often in support of whatever you’re talking about. Just it inspires me.

Per: Yeah, that was the next question, if we made you mad.

Jared: Rarely, rarely.

James: How cross did we make Jared. Yeah.

Jared: No, it takes a lot to make me mad.

Per: We actually met in 2002, which of course you won’t remember. But I attended User Interface East in Boston.

Jared: Wow.

Per: In 2002.

Jared: Oh, yes, that was a long time ago.

Per: It was. Now we met in the bar because I was whining to you about all the American examples. There were so many American examples that I couldn’t relate to and you told me, “Well, what do you expect? It’s an American conference.”

Jared: That sounds like me.

Per: Yeah.

Jared: We don’t get out much around here. So we don’t …

James: So I didn’t realise that was your first time you did that whinge because I’ve heard you do that whinge a few conferences.

Per: Yeah.

James: Because here in European conferences, it’s quite often the case. It used to be. Maybe they’re getting better but the Americans used to give American examples an awful lot.

Per: Well, it used to be the old Netflix example. But finally, two years ago, we got Netflix. So now, we can also relate to that.

Jared: Right, right. Yeah, around that time, it was hard to find non-American examples just because we didn’t — because most of the examples were localised businesses. They weren’t crossing borders and — yeah, it was a hard time for that and got into that problem a lot as the web was transitioning and still being very much — now, it’s less of an issue in western cultures. But you still for instance — Japanese or Chinese or Korean examples are hard to come by and Americans can’t relate to them really. They look completely foreign.

James: Yeah. No, I mean that’s a good point with the kind of cross-cultural examples. I mean it’s intrinsically tough because the cultural differences are important and when they get to a certain extent, then it’s going to make the example so abstract that you’re unable to get your head around it.

Jared: Yeah.

Per: And then there are the Ling’s Cars example that everyone loves to use.

James: Oh, Ling’s Cars. Yeah.

Jared: Well, Ling’s Cars is an international …

Per: Yeah, exactly.

Jared: It crosses border. Hey, if you want a fun homework assignment, go to Ling’s Cars and do “view source”. I’m not kidding. It is worth the two minutes.

Per: Oh, wow. Excellent. I’m going to do that.

James: I’ll go and do that. Yeah.

Jared: The thing that got me — I put her up once as just a gag slide and it was at a conference of 500 web developers and within seconds, everybody went to visit the site, to go see if this is for real and she tweeted immediately that I was making the server under her bed vibrate.

James: That’s so good at keeping on brand. Even like the comment is exactly on brand.

Per: Yes, it is. Maybe we should get on with the show.

James: Yeah, we could.

Per: Yeah.

James: The seed to the show, we could call it, was a tweet that you tweeted on November the 15th.

Jared: You’re going to make me remember how — if I was on brand on November the 15th.

Per: Yes.

James: Yeah. I can read it out for you, so you don’t have to remember exactly what it was. “If your new UX strategy process doesn’t start with deep user research, then your strategy isn’t about UX.”

Jared: Oh, I remember making that. Yeah. I would still agree with that. It has only been 26 days. Yeah.

James: Yeah. But it’s — one of our other listeners asked if we could talk to you about it.

Per: Yeah.

James: And we thought, yeah, that sounds like a good idea. You agreed too as well. So here we are.

Jared: Yeah, I’m out there.

Per: And I think the reason this tweet really struck a chord with us is because it has been up as a topic before as well with other people that — well, do I have to do the user research for me to be able to call it UX or can I read a book and call it UX? That’s sort of the gist of it. But I mean if we start dissecting your tweet which talks about UX strategy, what is UX strategy really?

Jared: So I mean I could take a piece of bread and I could put down some herring and some mayonnaise and then some lettuce maybe and another piece of bread and I could call it a hamburger. And if I’m big enough or if there’s no one there to listen to me, then it becomes a hamburger. I mean you can call it whatever you want, UX. But if you’re trying to convince other people that this herring sandwich — which could be absolutely delicious — is a hamburger, you’ve got this cognitive dissonance between what they believe a hamburger to be and the sandwich that you’re giving them.

So why call it a hamburger? I guess that’s sort of where I was going with this, which is if you’re serious about UX strategy. I think the reason I tweeted this was because I was talking to somebody who in fact was telling me about their organisation’s UX strategy and they had this whole timeline laid out and it wasn’t until way down in the timeline that they were going to introduce user research into their strategy process.

It’s not — users in a strategy or a strategy — it could be any other type of strategy, right? It could be a market-driven strategy. It could be an engineering-driven strategy. But it’s not a design-focused, experience-focused UX strategy if it doesn’t have research at the very core of it because it — at some level, you have to have reflection on the user’s experience. Now what that research is can vary dramatically from one organisation to another or in one institution. But that — yeah, that’s where that was coming from. Plus I had just gotten out of an all-night binger and I was just …

Per: I know that feeling.

James: Yeah.

Jared: Yeah, yeah. Grumpy as …

Per: The best tweets are born that way.

Jared: Yeah, yeah. I probably haven’t been flying for a few days and had run out of tweeting material. I always resort to design and UX stuff but I don’t have anything to do with …

James: I think the — one thing I start thinking about when we talk about UX strategy is — we fall straight back to UX maturity and I think — well, if we take the scale of the UX maturity, I think there’s that model with like one to six levels of UX maturity. You have unrecognised at the bottom to fully embedded and following more like the kind of — the Disney example. I think you’ve blogged about the other week where it’s completely embedded in the entire organisation and everyone has got buy-in and everyone is enjoying it and you’re doing a really good, integrated product.

I’ve got this hunch and this theory that UX strategy comes in only in the middle part of that maturity model. You know, when you’re kind of — maybe you’ve got a UX department. You’re getting recognition at the board level. But it’s not fully there. So that’s when you need the UX strategy. The early stages, they have no idea what UX is. You’ve got to use other words to communicate what’s going on in ideas. Then when you get to the fully mature end of the scale, then we’re into just business and products and UX again …

Jared: So a strategy is just an approach to a desired outcome, right?

James: Yeah.

Jared: It’s a — so in order to have a strategy, you have to have a desired outcome and this idea that you are doing strategy or not doing strategy is really a — it’s not something you do, right? It’s a mindset about how you’re going to get to the outcome. It’s actually a military term, right? Strategies are how force commanders communicate what the overall outcome is and then divide up the work amongst the forces.

You’re going to protect the flag to the right and I’m going to take my forces and come around to the left and we’re going to have an air cover. It’s this multi-group organisation to achieve an outcome and the outcome is to move the force to an objective and so that’s what strategy is and in terms of a business, a strategy is the same thing. It’s how we’re going to approach something.

To me, a UX strategy is a strategy by which the user’s experience is the driving decision maker behind how we choose what’s in and out of the strategy. So in a UX strategy, what you’re basically doing is saying it doesn’t matter whether the product technically works. It doesn’t matter whether it meets our business objectives of generating the right amount of sales or producing the right amount of leads or whatever that is. It only matters if it’s designed well. Then we will ship it and everybody needs to understand that that’s the criteria by which we have decided we’re done.

A UX strategy is a — to do that, it means that the organisation has to have conversations. They have to have conversations about what does “good enough to ship” mean. This is not new because if you’re engineering-driven, you also have to have a conversation that says what is good enough to ship. If you’re market-driven, you also have to have a conversation that says what’s good enough to ship, right?

In engineering-driven, it has to be technically-reliable and do what it needs to do. In a market-driven organisation, it has to meet the market place needs. So that notion of having that conversation isn’t new to an organisation. What’s new to the organisation is that it’s about what is the user’s experience.

A good UX strategy has rubrics that you can test, so that anybody in the organisation can tell whether they’ve achieved it for their piece or they haven’t. One of my favourite examples is this apocryphal story about Steve Jobs and one of the smaller iPod Minis. The goal was to make the iPod as small as possible.

The team brought him a design and he said, “You need to make it smaller,” and they said, “We can’t make it smaller. We’ve packed all of the wires in. We’ve packed all the circuitry in. This is the smallest we can make it.” He took the iPod and he walked over to the fish tank, as the story goes. He dropped the iPod in the fish tank and air bubbles came out and he says, “There are air bubbles. You still have room.”

Per: That’s fantastic.

Jared: OK? Now what’s really important about that story is not that Steve Jobs was a complete dick. What’s really important about that story is that he then gave the engineering department a rubric. They no longer needed him. They just needed a fish tank.

Per: Yeah.

Jared: So now that they have the fish tank, they could test and make sure that the next prototype they brought him met that rubric.

James: They could test and they had a visible goal as well. When the bubbles stop, they’re there.

Jared: Right, right. And everybody could see it. It’s objective. Everybody interprets the data the same way.

James: And it’s clearly communicated. Yeah.

Jared: Right. Yeah. And so a UX strategy has to have that and part of the goal of user research is to help figure out what that is, right? Part of the goal is to say when users do X or feel X or think X, we are done.

Per: Nice.

Jared: And so you use the user research to figure that out. So that’s where this came from. If you are not doing user research, you can’t have a UX strategy because you can’t set objective measures that everybody can see to tell if you’re on strategy or not.

Per: And the desired outcome of this case would be the tininess of the iPod Mini. Like, it has to be really small.

Jared: That was the goal that was set. Now, whether that was the right goal is the same question as to whether the general is deciding to take that city or that castle was the right goal, right?

Per: Uh-huh.

Jared: And textbooks will be written in business schools for years as to whether the iPod Mini had to be as small as it was or whether another goal was a better goal. Just like military academies study whether taking that city was the right thing for the general to do, right?

There’s a lot of Monday morning quarterbacking, which is a definitively American term.

Per: Right.

Jared: For second guessing what everybody does.

Per: Yeah.

Jared: But that’s a different part of strategizing, right? Picking the right goal.

Per: But that also means you’re not using the user research to find out if you picked the right goal. You’re still sticking with the goal but you’re using the user research to find out if you’re actually accomplishing the goal.

Jared: Goals get set in lots of different ways. So I can definitively see context where which you would use user research to define the goal. But that’s different than using user research to tell if you’ve met the goal.

James: Exactly. Here we’ve got user research as an input to strategy work and then you’ve got user research as a validation tool during the tactical implementation of said plan.

Jared: Right, right. So where you see the difference, one of the glaring places you see the difference, between those two different outcomes, is in the world of advertising because when you’re using user research to determine if you’ve met the goal, if the goal is get users to see as many ads or click on as many ads as possible, because that’s how we make money, then your user research is all about figuring out ways to get users to click on ads or to see ads.

But if you start with we’re going to create this publishing platform and we’re going to figure out what makes users happiest, what delights users most, before we set our goals, you would never end up with advertising as a business model because advertising actually is about disruption. Advertising is about destroying the user’s focus. It’s about getting users to click on things that have nothing to do with why the user came to your site or what they’re trying to accomplish. So suddenly, you would pick something different for a business model if your goal was to delight and not frustrate your users.

But if you don’t have that as a goal, if that’s not part of the input process, because you didn’t do the user research upfront to say our users are unhappy with the way our site works, then you just make it more disruptive. You make it more interruptive. So for example, you might take an article which would be easiest to read in a single shot on a single screen and break it into 25 pieces, so that you can serve more ads.

You might put ads in the middle of the text. So you will be reading two paragraphs and suddenly there’s something that is an ad or something that links to another article, so that the user thinks that they’re supposed to click on that when in fact you’re taking them away from the article they’re reading to bring them something else, so that you can serve more ads.

Those types of patterns emerge when the goal is not a user-centred goal. The goal is a business-centred goal. But we’re using user-centred techniques to achieve the business goal. That’s really an important distinction that a lot of folks are struggling with right now.

Per: Yeah.

Jared: That just because you’re doing user research doesn’t mean you have a UX strategy, which is the inverse of the tweet that I made. I have to make that one next year.

Per: Exactly.

James: Yeah.

Per: Now that’s a really key point. Yeah.

James: Yeah.

Per: I like that, because that’s what the people — a lot of people misunderstand when they hire me is they think I’m there to actually help people understand the website the first time they look at it. They have to understand every element within the first few seconds they look at it. That’s how they try to measure how well I do in the project while I’m trying to figure out, “Well, what are you trying to accomplish with the website? How can I help you do your business better?”

James: Yeah, exactly. Often you turn up to a project and you’re expected to deliver something or it’s actually the first — the first step is, “Well, is what we’re doing right?” That’s kind of what I’m seeing more. I’m understanding a little bit more now by what we mean by the tweet. We’ve got this validation using deep user research or interviews or whatever technique it is to help us understand that with the thing where you think we need to produce is actually worth doing before we jump in and start doing it whereas in a lot of occasions we do actually already start doing it. The decision is already made to make that web product or make that website — we come there and then say, “Well, we need to do user research.” But a certain amount of that has to be done beforehand to even have the vague idea that we’re on the right path,

Per: Exactly. You need to define those goals and those desired outcomes. But how do you know what is enough user research? Because one of the components of your tweet, which I’m still dissecting, was that we have to start with user research. But you never stop doing it. I mean you keep doing it iteratively until you accomplish your goal. But how do you know how much user research you have to do in each iteration to actually get to where you’re going? How do you know you’re using the right methodology?

Jared: How do you know when your house is cleaned?

Per: I never do. My wife tells me …

James: It’s quite easy. I have two kids and our house is never clean.

Jared: OK.

Per: The definition of clean is different for me and my wife, definitely.

James: That’s true, yeah.

Jared: I mean there are things that are never done, right? Particularly if you really are a UX-driven organisation, then user research becomes part of the DNA of the organisation. So you’re never thinking in terms of it being done. It’s just part of how you do things. So for example, if you run a restaurant, you have tasks built into your daily routine about cleaning the restaurant. There are things that the health department mandates but that’s probably a minimum. You probably have a higher standard than what the health department mandates if you’re running a decent class restaurant.

Depending on how much you charge for the food and the sort of environment you’re trying to make, how you clean and what it looks like when it’s “ready for a customer” is a shifting bar than — you know, for some organisations, it will be different than others. But there are certain minimum standards that you will come across and you will say, “This is unacceptable and this is acceptable,” and part of any organisation’s role, the management’s role, the executive role, is to help define that bar of when something is acceptable or not acceptable and again the more objective you can be about that.

So one answer about how much user research you do is — comes back to what we call the — my brain just completely — my vocabulary server crashed. Hold on. Rebooting. What we call the moment — damn it! I just had it.

James: This feels like my world.

Jared: I should know not to drink so much.

Per: I love that we have video now as well. I love that.

Jared: Yeah. I got it. OK. I got it now. What we’re talking about is the moment of least astonishment. This is what we call it, the moment of least astonishment, right? So you go out and you are able to predict what you’re going to learn in the research. Now you have to be able to not confuse that with confirmation bias, which is a different phenomenon. So one of the exercises that I do with teams that I’m working with on a regular basis is I am always asking them what they believe will happen when we go out and do the research.

So if we have task-based research, I ask them — I tell them what we know about the participant who’s coming in to do the tasks and I say, “What do you think we’re going to see this participant do?” OK? What problems will they have? How quickly will they be able to accomplish the task? What pieces of his design will feel familiar to them? What pieces of his design will feel foreign to them, right? We have a whole conversation about this before we’ve watched the research and if we’re going out into the field, I have the same conversation about the context and about the environment and about where this person fits in the field. If we’re doing stuff that’s more phenomenological and less task-oriented, I ask what phenomena we may actually observe, what sort of things we would expect to see and I get them to start thinking and projecting what they believe the user’s experiences will be.

Then we have something solid to compare it to, right? We’ve got what they expected and then we’ve got what we actually observe. We can see how often they get it right and we can see how often they get it wrong. We compare these two things to each other and frankly, if it turns out they’re like 90 to 100 percent accurate at predicting what the users will do and the parts where they get it wrong really aren’t influencing the design outcomes at all, we can scale back on the research because these guys are literally living the dream. They are living what users do. They know that experience well.

But if they can’t predict that, then there’s still a role for you as a researcher.

Per: Right. And all my clients expect me to be able to predict it all the time because they don’t want to pay a lot, because they ask for a price. We want usability testing. What does it cost? We want user research. What does it cost? So that’s the difficult part. So …

Jared: My question for you is, “Why is it an option?” Right? So now, do you tend to sketch — you primarily do design work, yes?

Per: Yeah, a lot of design work. Yeah.

Jared: Yeah. So now when you’re doing your design work, do you tend to sketch your designs first or do you tend to go straight to the machine and code up in the browser?

Per: Sketch.

Jared: OK. Do you give them the option for sketching?

Per: No.

Jared: Why not?

Per: Because that’s how I work.

Jared: Well, it costs more for you to sketch it first.

Per: Yeah.

Jared: Than to just …

Per: That’s excellent, yeah. Yeah.

Jared: Right? So why are you letting user research be an optional thing, but sketching not? This is what I don’t understand, right? I completely get it. Once you tell people things are optional, right? You can have a car and the tires are extra. Well, like I said, I don’t need the tires, right? Once you go down that road, you’ve created these decision points for people, for which they are the least qualified people in the room to make the decision. Why are you giving them the choice?

Per: It’s depending also on how they ask the question. Well, it depends on of course what type of project I’m in. If I’m on the long-time project, usually I work the research into that. But if they’re sending out like — to several different companies asking for help and they respond — and in a similar fashion, they usually do, the companies that I’m aware of in a sense that they have the user research. They have like two testing periods with development in between and stuff like that. That’s how they compare different suppliers.

James: I think this is a bit about — well, your target audience of your deliverable and sometimes — I mean I think that’s perfectly OK to not give the option of user research at least in certain stages because maybe in the initial stage, the point is communicating a certain concept or idea or method or something to the next person in the chain, which might be your client or it might be a designer or it might be something — it’s a step towards maybe doing the user research. It’s not like you can avoid it every single time or kind of duck it. But at that point –

Per: Well, no, but that also depends on the clients.

James: Yeah. But at that point, it’s — you’ve got to communicate. So you’ve got to succeed in communicating and the sketch map is good enough.

Per: You need to agree on the problem as well.

James: Yeah.

Per: It’s what you’re trying to do. They need to get confidence in — that I know what I’m doing.

James: And if the sketching communicates that and gets aligned in agreement, then it’s not necessarily that expensive, Jared. It’s — it might be more costly to kind of do it straight in the browser because that wasn’t the communication goal you had at that point with that deliverable.

Jared: Right! I mean it — you get to make the call as to whether this is the right thing to do in your process or not. But my point is, is that you’re not making every element of your process a la cart.

Per: Yeah, yeah.

James: Yeah.

Jared: Right?

James: Got it, yeah.

Jared: And so every time someone complains to me, well the users won’t — you know, the customers won’t pay for this. I’m like, “Why did you give them an option?” The doctor doesn’t give you an option to wash his hands before he does the surgery. This is — he’s not breaking that out, right? If this is a necessary component of the work, it is by definition a necessary component of the work.

Per: Yeah.

James: Yeah, that is true.

Per: We’re in complete agreement. Well-described.

James: Yeah.

Per: So where were we?

Jared: I mean you’re talking about a la cart usability services.

Per: OK.

Jared: But before that, we were talking about user research as part of the project. That’s all because I tweeted after my binger.

Per: I actually had one more question that I wrote up earlier today, that I thought about that people often — when this comes up again, do I have to do the user research to actually do it right? And I’m designing a web form. Is it enough that I’ve read Luke Wroblewski’s book? He has done all this research about how to design the best web form. Is that enough to actually start working on my web form or do I have to actually go out and understand my users as well?

Jared: Well, if you use the rule of the — the point of least astonishment, right? You have to do the user research to answer the question.

Per: Yes.

Jared: Right? You have to do the user research to find out if you need to do the user research. And so unfortunately the answer is yes, because you don’t know. Now, what it really does answer is how much user research do you have to do, right? If you have a definition of clean in your restaurant or the definition of clean is that there should not be any spots on any of the plates or silverware or tablecloths or anything, that’s a fairly objective measure. If you can see a spot that is not the natural look of that thing, it needs to be cleaned.

If you can’t see the spot, it doesn’t need to be cleaned. Now it doesn’t mean that there aren’t all sorts of grubby microbes crawling all over those things, that you can’t see. That’s not the rubric we’re using. So if the rubric we’re using to determine whether I need to do user research or not is, “When do I do research, do I learn anything I didn’t know?” then I have to do the research, just like I have to look at all the silverware to know whether it’s clean or not.

In some restaurants, it’s acceptable to let the customer discover it’s not clean.

Per: Yeah.

Jared: And say, “Hey, this fork is not clean. Can I have another one?” In other restaurants, that is an unacceptable behaviour and if that happens, there’s a question as to what in our process could we change so that next time, the customer isn’t discovering a piece of dirty silverware.

Per: Yeah.

Jared: Right? And there’s a retrospective discussion about that and for folks who are into process improvement and have that rubric and have that principle, customers should not be the ones who decide if this is a frustrating experience or not, they’re going to come to a different strategy and therefore a different set of tactics that get them to that outcome than people for which — well, why don’t we just build something based on what we think is right and we will ship it and we will see what the customers say?

In essence, every time you ship something, that’s a user research process. You’re doing user research when you ship. The question is, “Is the outcome of that user research the outcome you’re going to be happy with?” If customers hate you because you’ve created something really frustrating, you now have lots of data to work with. But was that the right way to get the data?

Per: We’re actually doing a show soon about minimum viable product and I think that’s excellent input to that show as well.

James: It is.

Per: You’re describing everything so well and I think it all seems to come down to that people don’t even know why they’re doing user research. They don’t have the rubric. They don’t have the set goals or the desired outcomes really defined. So they’re actually just trying to find problems when they’re doing user research and they say, “Oh my god! That user didn’t understand that link. We have to fix that.” They’re just chasing after these misunderstandings and these hurdles that people fine.

Jared: So if you’re — going back to the military examples, right? If you’re a bunch of farmers who just — every time somebody walks on your property, you take a pitchfork and a torch and you go chase them off your property, that’s not a military tactic, right? That’s not a strategy. That’s just get off my lawn, right?

We were talking earlier about this notion of at some point in maturity, which is — you know, maturity is just a fancy word for growing older and gaining experience.

James: Turning into Don Norman.

Jared: Right. So yeah, turning into Don Norman. That’s exactly right. So there are a few people more mature than Don Norman. So that’s going to be a tweet.

There’s this notion of sort of awareness of there’s a specific outcome I’m trying to get to. So I think people who don’t have a strategy are just not aware of the outcome that they want to get. They haven’t thought about the outcome. They’re just reacting. Reacting is not an outcome. It’s not working towards an outcome.

Per: Yeah.

Jared: Reacting might be keeping things at bay and maybe that’s the outcome is I don’t want things on my property. So I’m going to push them away. So maybe that is a strategy. But if you haven’t even given it that much thought, it’s just I don’t like that guy. I’m going to chase after him with a pitchfork and a torch, which is a very Don Norman thing to do.

That’s not a strategy and so the moment an organisation gets a strategy is really because there’s — they’ve had this realisation that there’s an outcome they would like to attain. That’s sort of an epiphany moment. Then they have to figure out all the tools and methods for attaining and gain the experience to get there and start making good judgments, right?

There’s an old saying which is, “Good judgments come from experience and experience comes from bad judgments.” You sort of have to go through that process of making mistakes, which is really what the user research is there for. The user research is there to give you a feedback mechanism, to tell you whether your approach to attaining the outcome actually worked. When we do preliminary research, when we start looking at research upfront, for that input thing, what we’re really doing is asking the question, “Have all the things that have been tried to-date working?” Right?

And what is it about them that doesn’t work? So oftentimes, you even have to go into that research with a sense of an outcome, right? The outcome may be we want to delight our customers or we want to delight them more than we’re currently delighting them, right? That’s our outcome. But you have to go into the research with that outcome so that you have a focus for the research, so you can then come back and say, “Well, here’s where we’re delighting customers. Here is where we’re not delighting them at all and here’s where we’re delighting them a little, but we could do better.”

Now we don’t want to break the places where it’s working. We want to completely fix the places where it’s not working and we want to at least improve the places where it’s working a little, but not as good as we imagine.

James: Another analogy here, it’s like your car is leaking oil and you take it to the car mechanic to get it fixed. It’s the mechanic’s job to fix the leaking — the oil leak, right? It’s not their job to kind of asses the kind of aerodynamic performance of your car and make sure it drives faster. It’s their job to just fix the oil and I think this is maybe a professional thing for us.

Jared: I guess that depends on the mechanic you go to.

James: Exactly. And I think that’s the professional side of us, that we’re sometimes — in the roles that we have, at times we are being the mechanic and we’re meant to be fixing and tinkering things. It’s a tactical thing. It’s a change. You’re fixing maybe a usability issue. Then other times, we’ve got a different role and we’re a little bit higher — on a higher level, you know, holistic, and we’re looking at something more strategic.

Jared: Let me take that analogy apart for a second. OK? We go to the mechanic. Our intention is to get the oil changed, to fix the oil leak, right? In the process of fixing the oil leak, which the mechanic can do, the mechanic discovers that there are two belts, that in the next five weeks, based on his experience, those belts are going to break and at best, they’re going to leave you stranded because the car won’t move. At worst, it will happen while the car is in motion and it could be a health hazard, a safety issue, right?

Should the mechanic who was hired to do the job of changing the — fixing the oil, suggest to you that you get that repaired while you’re there.

James: Yes, because they’re fixing a problem that’s fixable. But what the mechanic should also do is report back to the car manufacturer and say, “Hey, maybe you should look at the material you make your belts from because I think we could improve the material we make it from to make it last longer.” That’s not something the customer is interested in at that point. They want it fixed. But the mechanism has to be there for him to report back the kind of improvement potential for the overall product, while still fixing the problem for the user.

Jared: Oh, OK. So in the States, we have two types of mechanics. We have independent mechanics and then we have mechanics who work — actually they’re all independent because the way the laws work in the United States, a car manufacturer actually can’t sell their own cars. So, all the dealerships that sell cars are independently-owned institutions.

James: Yeah.

Jared: And they actually don’t like talking to the manufacturer. In fact, they’re not that interested in making the cars require less maintenance because it doesn’t help them, right? Their incentives are someplace else. So this actually a really interesting problem because in that construct, in that context, the — it’s not the incentive for the mechanic to report back saying, “We should change the material of the belts.” It’s the manufacturer who needs to somehow give the mechanic incentive to report on the field results.

James: Yeah.

Jared: OK? And this is part of any research process is to understand that business context and the incentives that are in place because the mechanic has no reason to tell the manufacturer to get rid of the part — the very lucrative part of his business where he replaces a $2 belt and charges $150, right?

So this is — there’s a timing chain on Toyotas that goes at the 75,000-mile mark or at least it used to. I don’t know if it still does. But when cars hit about 75,000 miles, this timing chain just gives out. The car stops moving, right? It’s a chain that runs inside the engine.

Toyotas that have this problem, they — everybody knows about this problem. The actual chain costs $15 to make. OK? But you have to take apart the entire engine to replace it. So it’s a $1500 replacement. But it’s also a safety issue. So you’re always playing this game of do we let the customer — we know this thing breaks somewhere around the 75,000-mile mark. Do we let the customer wait until that moment? And it may never break because for some reason, you might be the lucky one, right? Or do you — because the odds are good and it could be a safety issue. Do you let the customer — do you pretty much force the customer to pay you the $1500 because they’ve reached this point, right?

And the mechanic is less incented to say, “You know, we should make a better chain,” than Toyota is. Toyota can come down the pike and say, “Look, we’ve remade this model of the car with a better chain. You will no longer have the $1500 expense at the 75,000-mile mark. So therefore we’re going to charge you $1500 more now,” or $700 or whatever it is, right? We’ve given you a better product.

That will make the customer happy. But that won’t make the mechanic any happier.

Per: Right.

Jared: So you have to decide who your customer is, right? Some businesses will tell you that the mechanic is their customer and some businesses will tell you that the end user is the customer and these are the types of decisions that you make that will change your strategy and in both cases, it’s still UX.

Per: Yes.

Jared: Because you’ve done the research and you’ve set a strategy and you’ve picked your user and you’ve realised you no longer can make both users happy. So you have to pick the user that makes the most sense to the business and you go there. A competitor might pick a different user and that would give them a competitive advantage with that user group. That’s what competition is for. Seriously, right?

James: Yeah.

Per: We just went into Blue Ocean Strategy as well.

Jared: Yeah. I’ve yet to figure out what that is.

Per: I think you’ve cleared out a lot of our questions actually. I’m going to have everyone I know listen to this show because I’m thinking that from that tweet, we had some preconceptions about what that tweet meant and we had some arguments about — well, of course, you can call yourself a UX designer but that wasn’t really what the tweet was about. Now that you’ve explained it, you’ve explained the essence of UX strategy, why we do user research and how to go about setting up your research and how to actually assess when you’re done with the user research as well, which was amazing in my mind. So thank you for that Jared.

Jared: You’re welcome. I’m really glad you didn’t ask about the sheep, because that wasn’t true. Whatever you’ve heard, it’s not true. I mean it happened. But I wasn’t there.

James: No, not anymore.

Per: OK.

Jared: I mean I was there. But I didn’t have that much to do with it. Really, not that much at all.

Per: This has been loads of fun. We need to do this again.

James: Yeah.

Jared: Yes! That would be great!

James: I hope that when this episode pops up — which actually — this is actually going to go out later today.

Per: Well …

Jared: Wow.

James: So next time you’re at the gym, I hope you can cope listening.

Jared: I will take notes. It will make me angry.

Per: Exactly.

James: You have to ring in and complain on our next user phone-in.

Jared: No. I would be happy to do this again. This would be fun. You can find some other tweets I’ve said and ask me what the hell I was thinking. I will construct an entire scenario around it, as if that’s what I was thinking when I said it.

Per: It will be a new theme show with Jared’s tweets.

James: Jared Watch.

Per: Yeah.

Jared: Yeah, help us interpret Jared’s tweets. Sometimes they’re just inscrutable.

James: And still get a hundred retweets.

Per: Exactly.

Jared: Yes! My wife accuses me of that all the time, that I can write a tweet about a fart and it will get retweeted a hundred times.

James: She’s right. It’s true.

Per: You can test that of course.

Jared: We have tested that actually. It was a hypothesis and I think it did actually get retweeted.

James: Right. Well, thank you very much for joining us Jared.

Per: Yeah, thanks. We will talk to you soon again.

Jared: Oh, well thank you for having me. Yes! Yes! Please and …

Per: We have to come over to one of your conferences of course.

Jared: You should come to one of my conferences. You know, we had a conference last spring, the UX Immersion Conference, and yeah, if you go to , you will get to see the videos of the presentations that people gave. We have videos from Luke Wroblewski and Karen McGrane. I gave a talk. Nate Schutta, Cyd Harrell, and they’re up on the website now and you can watch it.

Per: That’s fantastic. Thank you.

Jared: That conference, the 2015 conference is going to be in Salt Lake City April 13th through 15th and we have got a great line-up which we will be announcing on January 7th for that event. I think it’s one of our best. So it’s going to be a lot of fun and we’ve been working on all the materials for that. So I’m excited about it.

James: Actually I am in America for at least one of the dates of that –

Per: Really?

James: Yeah, but I’m flying back on the 14th.

Jared: Dude, you should change — come to Salt Lake. Yeah, yeah, yeah. So Salt Lake and Stockholm have a little bit of a similar feel. There isn’t as much as the old city of Stockholm but in the more modern parts, Salt Lake and Stockholm feel very, very similar because the climates are very similar and the people are like the nicest people on the planet. They are just like Stockholm and …

Per: At least in the summer time.

Jared: Yes, yes.

James: They’re quite nice in winter too.

Jared: April is such that it’s just towards the end of skiing season but your travel is easy at that point. So it all works out really nice and we’re really excited to be there. So it’s going to be a good event. OK.

James: Have a great day Jared.

Jared: Time to get work

James: Yeah. Bye-bye.

Jared: Bye-bye.

[Music]

James: You know, I’m going to have a problem.

Per: What?

James: Well, whenever I’m recording UX Podcast now, I’m going to have a mental image of Jared on the rowing machine in the gym.

Per: And after you said that, now I am as well. That is my biggest takeaway from this interview.

James: Mental imagery.

Per: Yes.

James: No. There were plenty more take-homes from the interview.

Per: For a few. I see you’ve probably heard. I was really impressed by how he — how clearly he can just break it down and have real world examples of just everyday life and make it seem so simple. That it’s so obvious to you that of course, this is how you do it. How else would you do it? I also hate that — when he calls me out, that — why am I allowing the actual customer to have options. Why don’t I just solve the problem for them? Which of course is also true. But I’m thinking, well, if you’re Jared Spool, you can do whatever you want. But I’m just a struggling freelancer here.

James: No, you’re Per Axbom. You can do whatever you want too so you can get away with it.

Per: Na, I’m sure he can get awaay with whatever he wants

James: No, it actually was good fun interviewing someone who really does know their stuff because they’ve got an answer to everything.

Per: Yeah. Yeah, his points about maturity and experience, I mean that’s really important as well when you have actually struggled and found out the hard way.

James: I’m going to try and process now and think a bit more about this — I tried getting the point across a bit during the podcast. These two roles within research, user research that we have — we often have both. Jared didn’t buy completely my car mechanic oil analogy. But I’m going to stick with that. I’m going to play with my head a little bit more that it feels at times, you kind of go in there to do usability testing or research that aims to fix — that tinkers with little things to make things better. So I’m improving bits of the website and making kind of a button perform better. I’m doing micro changes I suppose. Maybe just changing a little text — line of content.

That to me is kind of fixing a little oil leak. But then there are other types of research I do to validate whether this is even the right website to have.

Per: Right.

James: And there are two roles — sometimes we really do only have one of those roles. That is really not on the cards for us to do that kind of more holistic big picture thinking and feedback about how it all is.

Per: So I don’t think Jared was disagreeing with you about that. But he was just saying those are different strategies.

James: Yeah, yeah. No, I wasn’t saying he was disagreeing.

Per: Yeah.

James: I’m going to keep working on it a bit more.

Per: So when you were talking about this analogy with the car mechanic, I was all the time thinking about how that applies to a content management system where you actually — if you’re working with a content management system as a consultant, as an expert on that system, for a client, then you’re perhaps tweaking and changing texts as well. But maybe you will find something with that CMS that is sort of broken that it really hurts you or hurts the client in some way. Then maybe you were thinking — yeah, then you will give the feedback to the actual vendor of that CMS to fix that — whatever mistake it is that they have …

James: The customer’s not interested.

Per: Yeah.

James: Yeah.

Per: But what Jared has been implying is that maybe you don’t have the incentive to do that if it gives you more work.

James: Yeah. I mean it’s the whole client-agency-vendor mishmash of problems and stuff and — yeah. But I’m going to chew on it and think about it a bit more.

Per: OK.

James: Yeah. Well, that was that for today then. But before we do go, before you all switch off and skip to the next episode of whatever you’re listening to, remember you can visit and you will find links and stuff. Links that Jared has mentioned now will be up there and a few bits and bobs and all previous shows. If you’ve enjoyed the show, then make sure you like follow us on various places and tell your friends and if you really feel kind and warm and lovely now and it’s Christmas, give us a Christmas present and that Christmas present will be give us a review somewhere.

Per: Yes! That would be nice.

James: Please give one as charity.

Per: Give us a review. Do it.

James: And also remember to visit to try Peek for free.

Per: Yeah. And now that we’ve dedicated this entire show to user research, visit that link. Try it out.

James: You can get some free real user research.

Per: Yeah.

James: So take the chance with that.

Per: And also remember to keep moving.

James: See you on the other side.

[Music]

[End of transcript]


This is a transcript of a conversation between , and recorded for UX Podcast in December 2014. To hear more episodes visit or search for UX Podcast in your favourite podcast client.

UX Podcast

Written by

A twice-monthly user experience podcast brought to you by @beantin and @axbom. Balancing business, technology and people within the realm of digital media.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade