004 | Customer interviews - a skill AI cannot do

EDITION 004 | More Room to Think (Archives) | Formentor Labs​
​READ TIME: 6 minutes

🗓️ New this week

The single most important research skill that anyone can master — is customer interviews.

Interviews give us real customer evidence straight from the source. We prefer that over an aggregate AI answer any day, in any market.

With the arrival of AI a lot has changed in the research space;

Good -- using AI for admin tasks; so that humans spend more time with humans. AI has allowed us to complete more interviews, faster, and often with more attention because we're no longer taking our own notes.

Bad -- forgetting our empathy. I see posts about teams asking AI to generate a summary of what customers want; using simulated artificial audiences, or having AI attempt to conduct interviews. Yuck. Garbage in garbage out.

Conducting our own customer interviews is an art and a science, and will become one of the most sought-after skills for PMs and growth leads in this new era.

Make a little room to think Now is a perfect moment to take a breath. 8 seconds in, pause, 8 seconds out.

🔬 This week's topic

Why interviewing customers, ourselves, still beats asking any next-gen AI model.

(But Claude does make an excellent stenographer)

THE PREMISE

All LLMs are non-deterministic - this means an algorithm will produce different outcomes when given the same initial conditions.

The same prompt returns different probability-scored responses.

When we're researching customer needs and preferences, we don't want a distribution - that's what split tests are for - we want emotional exposure to the pain points they suffer.

Those pains change. Quickly. If we read any annual state of the industry or benchmarking report, we notice how quickly surveys can change from one edition to the next.

AI training data doesn't keep up with those changes. Probabilistic LLM models are pulling from ancient content sources, but we need today's data.

WHY IT'S IMPORTANT TO GROWING IDEAS

To sell our big ideas inside a company we need evidence leaders can buy into, skepticism-free.

So first, we must bring the most recent snapshot of the market.

This is why pull-quotes and verbatims are so powerful. They're first-party research. Verbatims from customers are some of the most powerful and easy-to-get tools we have to convince leadership of our ideas' validity.

Second, we need a moat. We need data as defence. Our competitors prompt Claude, too! Their AI access is identical to ours.

We have zero edge, zero moat, when we ask an AI for answers.

It's like standing on Earth, asking what walking on the moon feels like. Anyone could guess. Only the astronauts brave enough to go out and touch it themselves have real credibility.

HOW WE CAN USE IT THIS WEEK

Running interviews is a skill we develop. There are five stages from setup to getting results ready to present, and we'll tackle those in future newsletters.

To get started with the basics, try these steps this week:

• Set time aside. 3 hours in a week is enough to get results.
• Frame the goal, describe the research topic, draft the questions.
• How many interviews do we need to get insights? 5 to 8 is enough to start seeing patterns. (Nielsen)
• Record strong verbatims and note popular responses.
• AI can handle the menial tasks. That's made it easier for us to spend more time with humans. Scheduling, script-writing, finding patterns, pulling quotes - all great tasks for Claude.

Some of my go-to interviewing guides include Michael Margolis, Learn More Faster and Steve Krug, Rocket Surgery Made Easy.

I'm days from releasing my own interviewing guide - Customer Collabs.

A system built using free-only tools to help teams conduct effective customer interviews to gather fresh useful insights.

That will drop right here in the Room to Think newsletter.

Pssssst, hey

These are the early editions of Room to Think.
I could use any support to nurture my growing audience:

• Forward this to the author of your favourite marketing newsletter. Collabs are how newsletters grow.
  ​
• I'm an active speaker and looking for more speaking stage and guest podcast appearances. A quick intro is more than helpful.

Did you learn something from this newsletter? Did you put it into practice? I'm dying to hear how it went. Please email letstalk@formentor.ca and share your thoughts.