arrow-forward arrow-back hamburger facebook linkedin medium twitter modal-dismiss caret menu cross telephone map-marker
Skip to main content

Blog

Little Big Mistakes in Research

I think we’ve mentioned here before that we’re huge fans of Marty Cagan. Sometimes, I have to confess that I squirm a little when he recommends that anyone and everyone, but particularly Product Managers, should go talk to customers at every opportunity. Of course, he’s right, but what gets me squirming is imagining a young, enthusiastic Product Manager who’s rightly excited by a particular idea talking to people and missing tons of great information by making a few easy-to-make mistakes during the interview.

This came up for me again when helping a friend run a “how to do research” boot camp. We decided to script a few interviews to show how things might go badly awry, or very well, in an interview. The most interesting interview, though, was the one we put together that was “subtly wrong”. I realized that by ignoring some basic rules of thumb, it was really easy to get totally different information.

Here’s one of the scripts:

C: Hi Participant Patty, I’m Carla. Thanks for agreeing to talk with us today.

P: Hi Carla.

C: Patty, I’d like you to start by telling me what kind of car you’re looking for. Tell me, step by step, what research you’ve done so far.

P: Well, I’ve looked at Toyotas, and that’s about it. That’s really my only option in my price range. I don’t like Hondas… (short pause)

C: Lets talk about the Toyotas you’re looking at. What web sites are you using?

P: Well, the Toyota web site. And Google.

C: Do you use the web daily?

P: Yes, daily.

C: And what do you do online besides car research?

P: News mostly. I read news. And email. Does that count?

C: Sure. Do you use Facebook?

P: No, I don’t really use Facebook. Although Farmville is fun.

C: So you do or don’t use Facebook?

P: I don’t.

C: Can you tell me why not?

P: I don’t know, it just… it’s… (struggling)

C: Time-consuming?

P: Yes, time-consuming.

If we were to write up some quick notes about this person, we’d say:

  • Looking at Toyotas
  • Doing research online using OEM sites and Google
  • Uses web daily, not into Facebook

Now let’s look at a slightly different approach from the interviewer:

C: Hi Participant Patty, I’m Carla. Thanks for agreeing to talk with us today.

P: Hi Carla.

C: Patty, I’d like you to start by telling me what kind of car you’re looking for. Tell me, step by step, what research you’ve done so far.

P: Well, I’ve looked at Toyotas, and that’s about it. That’s really my only option in my price range. I don’t like Hondas… (short pause)

C: (silent)

P: I got in a car accident when I was in high school, and even though I know they’re technically safe, it was a really bad accident and I just don’t feel comfortable with them.

C: I’m sorry to hear that! You said you’ve looked at Toyotas. What do you mean by that? How are you looking at them?

P: I have been doing some test driving. That’s mostly it. And I travel for work so when I do that I always try to rent one of the models I’m thinking about. Also a little bit online: the Toyota web site. And Google.

C: I’d love to talk more about the test driving and rentals, but first let’s talk about the web sites. How often do you use the web?

P: Maybe three times a week.

C: And do you do anything online besides car research?

P: News mostly. I read news. And email. Does that count?

C: Sure it does. I wonder, do you use Facebook?

P: No, I don’t really use Facebook. Although Farmville is fun.

C: Farmville is fun?

P: Yeah. I don’t really count that as using Facebook. But I probably spend spend half an hour on it every time I log on. It’s the first thing I do. But I wish I did it less.

C: Why not?

P: I don’t know, it just… it’s… (struggling)

C: (silence)

P: I worry a little bit about Facebook, and the whole privacy thing, you know?

C: You’re worried about privacy on Facebook?

Quick notes, from an interview with the same person, would look more like this:

  • Shows concerns about safety, privacy
  • Doing test drives, renting cars, and research online with OEM sites and Google
  • Uses web three times a week, uses Facebook every time she logs in

Laying aside the shortcomings of this kind of research, what I love about these scripts is what a different picture you get of the interview participant. The first interviewer didn’t do a terrible job by any means, but really got different information. If you accept the premise that these scripts are plausible (and I certainly have seen examples of these kinds of things many, many times), then there are two basic rules that we probably all already know, but are reinforced neatly here:

  1. LISTEN. Just be quiet and listen.
  2. Stay neutral (be aware of leading, even subtly).

Partner

Audrey Crane

Contact icon

Let's talk about your needs and how we can help. Get in touch

Contact icon

Interested in joining the DesignMap team? Let us know