B-Side: Steve Blank and Intercom's Des Traynor | Customer Research

00:00:03:05 - 00:00:32:05

Patrick Campbell

From profile. Well recur it protect the hustle a show about those who are in the trenches actually doing the work. I'm Patrick Campbell. And on today's episode, we're going deep into customer research, why it's important and how to make it a bedrock of your company's success. Be sure to listen to the end of the episode for how you can get in on this week's book giveaway.

00:00:33:15 - 00:01:04:17

Steve Blank

Unless some of you have been working in a specific domain for the last 20 years or so, the odds are that anything you're thinking about customers and markets are nothing more than a guess. The real notion is how do you take your idea size the opportunity and figure out whether your hypotheses, your guesses about customers, about the problems you're solving, about your product, are correct.

00:01:06:04 - 00:01:32:21

Steve Blank

And the first thing I teach students and coach young entrepreneurs is while your hypotheses are great, have all of them you want. You've got me excited. The next step is get the hell out of the building and test them on a web company. You could do that virtually. You could get users to click on your page. If it's a physical product, you're physically leave the comfort of your dorm or office and you go out and you talk to people.

00:01:33:10 - 00:01:55:23

Steve Blank

And the first step you want to do is leave any notion of your slide ads or website or whatever at home and go out and test some of the fundamental hypotheses that you have about your business. And this is hard because if you're a passionate entrepreneur, you look at somebody like me and go, Steve, you don't get it.

00:01:56:15 - 00:02:20:02

Steve Blank

I'm right. I'm passionate. I got it, I've nailed it. I've talked to 20 people. They're all over me. I'm done. We're launching and I'll usually go, Great. So this ought to be a very quick exercise. In about 20 minutes, you could go out of the building or get some people to click on your website and confirm just what I said.

00:02:21:03 - 00:02:51:13

Steve Blank

For those of you thinking about companies, and particularly those of you building web companies, you could run this exercise in a week and trust me, it will be very sobering. The goal is not to convince you not to do a business. The goal of a customer development process is simply to take the hypotheses about your business and product as if and see if there's customers in a market outside the building.

00:02:55:12 - 00:03:19:02

Patrick Campbell

That was Steve Blank, a Silicon Valley veteran speaking to a class at Stanford on the importance of customer development and using customer research to get close to your customer. This whole notion of knowing your customer is something we hear about all the time. It's one of those startup platitudes that every luminary of Silicon Valley, your business advisor, likes to use to distill down their experience and advice to one little nugget of truth.

00:03:19:10 - 00:03:47:14

Patrick Campbell

Like it's somehow the Rosetta Stone of business success. Just focus on your customer. And while I wholeheartedly agree, I don't think most of us do. Or rather, most of us don't care or aren't willing to put in the work to truly focus on our customer research. Take, for instance, this data. When we surveyed 1800 founders and executives and asked them How many customers in your organization are you talking to in a non sales capacity per month?

00:03:47:22 - 00:04:13:20

Patrick Campbell

Or put another way. How many customer research conversations are you having in any given month? Seven out of ten businesses who responded said that they were talking to less than ten customers and this was consistent even amongst companies doing 50 million or more in annual revenue. Something's wrong here because as we'll chat about in a bit, those companies doing customer development are growing at 2 to 3 x the rate as those who aren't.

00:04:14:05 - 00:04:35:05

Patrick Campbell

But we aren't doing our homework. We either don't get what focusing on our customer actually means. We don't know how to research and understand our customer, or we simply don't need to focus on our customer. And it's just unimportant. Even though all of these smart people say the contrary. So that's what we're going to talk about today. Customer research.

00:04:35:12 - 00:04:58:16

Patrick Campbell

Why is it important? Is it important? What exactly it is and what you can ultimately do to get your feedback cycles humming to truly focus on your customer? Because at the end of the day, if you don't know who your customer is, then how the hell are you going to know what to build, how to sell, how to price, and just who in the world you're working so hard for.

00:05:03:22 - 00:05:20:19

Patrick Campbell

It's a bit pedantic, but before we explore why customer research is so important, we need to go back into why the hell you're in business in the first place. There's a lot of reasons. You're probably building a business, you know, you want to work for yourself, you're passionate about the space you're in and all kinds of fun stuff.

00:05:20:19 - 00:05:55:21

Patrick Campbell

But at its core, whether you're a nonprofit, a tech business, selling coffee, whatever it is you're doing, there's a product and then there's a customer on a very, very fundamental level. Your job as a business is to use things like sales and marketing to drive a particular customer to a point of conversion, a point where they say, this is the time that I'm going to pay you for this product, and then everything else in your business on the product side, the product team operations, etc., is there to make sure that it's the right product for that customer.

00:05:55:21 - 00:06:25:09

Patrick Campbell

The right product, It's the right customer. Now there's a whole host of things that influence the customer and product side of your business. There's the channels through which you acquire those customers, the positioning of your product. And on the product side, there are different features and plenty of scrutiny of the experience of the product. Yet there are always these two pieces held together by your pricing, which is the exchange rate on the value that you're providing that customer.

00:06:25:09 - 00:06:26:08

Kaitlyn Gustafson

Thank you very much.

00:06:26:12 - 00:06:49:18

Patrick Campbell

So where does customer research come into play? Why is this so important? Well, if you remember back to our episode on feedback. Customer research essentially shortens your cycles to building a great business. Your job is to create the quickest path to learning in order to make sure you have a flywheel moving of understanding your customer and building that which provides your customer value.

00:06:50:06 - 00:07:16:17

Patrick Campbell

Now, some of you may be thinking, you know, that's great, but we've been building technology online for the better part of 30 years now, so why should I care today? Well, it comes down to the speed of the market, which is supported by the data before we get to the data, though, we need to understand just how much the world of building product has changed in the past three decades.

00:07:16:17 - 00:07:38:05

Patrick Campbell

We've gone through three major shifts. The first took place between 1980 and roughly 1995. I call this the magic era of building technology because essentially everything that was produced was magic. You didn't have a lot of products out there. There just weren't even a lot of computers. So customers didn't have a concept of what was expected when it came to technology.

00:07:38:11 - 00:08:10:00

Patrick Campbell

So everything you made was just magical. There was competition and a really steep learning curve to teaching customers about your product, But the market was expanding slowly, so finding a customer wasn't easy, but it was also not that terribly difficult. Let's say a three out of five in terms of difficulty building product was extremely difficult, though. Let's say a five out of five on a difficulty scale because you were inventing new standards over and over again and you actually had to be good at hardware and software.

00:08:10:10 - 00:08:30:23

Patrick Campbell

It just was something that was terribly difficult. And because of that, customer research was pretty important at this stage because building product took so long. In a lot of cases you would have over a year before your product hit the market and a lot of times the technology would be the limiting factor to your success because by the time that you shipped, you'd actually end up being quite obsolete.

00:08:31:12 - 00:09:08:20

Patrick Campbell

After the Magic Age, we moved into the Golden Age from roughly 1995 to about 2012. Here's when it was amazing to be in business. Standards for building companies online were developed and solidified at this point, so buying products online slowly became the norm in essentially the entire developed world was online and could be marketed to building product dropped to about a 2.5 out of five, mainly because all of these standards were already established and all of these DevOps tools, the cloud and the like made building product or the actual technology quote unquote easy, or at least much easier than the previous age.

00:09:09:04 - 00:09:33:17

Patrick Campbell

And finding the right customer was a two out of five and difficulty because people were coming online so quickly. There was just this mass market that could be marketed to and ultimately sold. The technology that you were building in a much more rapid space than the previous age. We hit a sweet spot where basically customer research wasn't really as important because if you could build quick enough, you didn't have a ton of competition and could take the market.

00:09:34:02 - 00:10:01:19

Patrick Campbell

Think about Netflix or Amazon. They're obviously titans of technology, but the market also helped them immensely. The age we're in now, which started right around 2012, is the post golden age. Now building the technology is close to a 1.5 out of five for most of our businesses. Building the right technology is always going to be difficult, but that's a function now of the market, which finding the right customer is actually jumped to a four out of five.

00:10:02:06 - 00:10:25:22

Patrick Campbell

It's never been easier to run ads or get access to a customer, but it's never actually been harder to acquire a customer because consumers are getting hammered by offers and all types of fun stuff. Everyone in their mother has an e-book. Everyone is able to ship features at a reasonable rate and it's no longer a world where you can just throw something out there and allow the market to essentially will your growth into existence.

00:10:26:14 - 00:10:41:12

Patrick Campbell

We're now in a world where customer research has become so important because the pace of the market is faster than we can keep up. So building the right thing can't be won by simply guessing and checking your way to success.

00:10:46:15 - 00:11:21:02

Patrick Campbell

We see this phenomenon in the actual market data. For instance, the average number of competitors that you would have in your business in the first year of you in business has gone from two a decade ago to nearly 12 today. Your customer acquisition costs in both B2B as in business to business companies as well as B2C as in business to consumer companies, has actually risen by 55% in the last five years, which means that customer that cost you $100 five years ago is now costing you 150 to $160 to acquire today.

00:11:21:14 - 00:11:47:15

Patrick Campbell

The relative value of features has actually declined pretty substantially, mainly because software isn't magical anymore. We saw in nearly a million different consumers and in their willingness to pay that an average feature that might have been valued at about $100 five years ago is now dropped to about 30% of that particular value because all of a sudden there's just a lot more technology out there.

00:11:47:23 - 00:12:15:22

Patrick Campbell

And what's kind of amazing is that in the context of all of this, consumer sentiment has actually gone down. No one's going to stand up and say that software is worse today than it was ten years ago. But what's fascinating is that when we measure Net Promoter Score or NPS, which is the measure of customer satisfaction, we've actually found that NPS has gone from an average of 34, five years ago to now an average of about ten.

00:12:16:08 - 00:12:38:23

Patrick Campbell

So we're sitting in this world where software is still kind of magical, but it's lost a lot of its allure. And since we're sitting in this post golden age when it comes to acquisition, it's actually much harder to acquire customers and find your market than it was to actually build your cool piece of technology or your product. Product will always be difficult to build.

00:12:39:06 - 00:12:49:11

Patrick Campbell

But the world of distribution has actually been upended and growth is actually under attack by the density that's happening in the market.

00:12:54:07 - 00:13:12:16

Patrick Campbell

Great. So customer research and knowing your customer is important, but how in the world are we supposed to do this? Customer research is definitely going to be a little bit different for each type of product, vertical stage, etc. And I think the most important part to keep in mind is that it's a framework through which you learn about your customer and the market.

00:13:13:03 - 00:13:38:04

Patrick Campbell

What we found after sifting through millions of customer research data, points through price intelligently and proffer well, is that customer research typically has three parts a research question, then qualitative research, and then quantitative research. Your research question is pretty important because it provides the parameters through which you should be thinking about the research that you're conducting. Normally, a good research question has some sort of specificity around value for us.

00:13:38:04 - 00:13:57:20

Patrick Campbell

The research question that we're going to be talking about is what aspects of a cup of coffee drive the most value? Presumably all of you know what a cup of coffee is. So it's a really good example for us to talk through when it comes to customer research. Imagine we're doing this as a firm trying to launch a new type of coffee or trying to take on a specific coffee competitor.

00:13:58:01 - 00:14:19:15

Patrick Campbell

Once you've nailed down your research question, it's time to do what Steve Blank tells us to do, and that's to get out of the building and do some qualitative research. When you're doing this type of research, you're not trying to find the perfect answer by talking about your product or asking about the future. You actually shouldn't do that because it biases your answers, but it also doesn't really get to the core of what you're trying to do.

00:14:19:23 - 00:14:47:00

Patrick Campbell

You're more of a prosecuting attorney, trying to drill down into the psyche of your interviewees to scope down their problems and all of the facts around their problems that you're trying to solve. Most importantly, though, you got to get out of the building, which is exactly what we did for our research. Question surrounding coffee. We went out to the streets of Boston to talk to Dunkin Donuts, Starbucks and Blue Bottle Coffee customers to understand how they thought about coffee, what they valued in coffee.

00:14:47:00 - 00:14:59:06

Patrick Campbell

And ultimately, if we were on the right path with coming out with a new type of coffee.

00:15:03:12 - 00:15:22:22

Patrick Campbell

Excuse me, Do you like coffee? I'm doing a report on coffee. Do you guys like coffee? Do you like Duncan? Okay, no worries. I'll say a quick question about coffee. No. All right. Thank you. I You have to go up to strangers and ask them what they want. Can we ask you a quick question? Quick question, but ask you a quick question.

00:15:22:22 - 00:15:41:00

Patrick Campbell

Quick question. Do you mind if I ask you a question or two? Coffee and coffee. Coffee. Coffee and coffee. Coffee. Quick question about coffee. I'm not selling anything. Just a really cool question about coffee. Literally two questions. Oh, no worries. No. Got it. No worries. No, no, no. Got it. It's just not easy.

00:15:44:04 - 00:15:44:09

Kaitlyn Gustafson

Yes.

00:15:45:06 - 00:15:52:15

Patrick Campbell

Is that cool? What's your favorite brand of coffee? What's your favorite coffee brand? Okay. What's your favorite brand of coffee, if any? Starbucks. Starbucks. Starbucks.

00:15:52:17 - 00:15:55:07

Kaitlyn Gustafson

I don't really care. Starbucks. Dunkin Donuts on sale.

00:15:55:07 - 00:16:03:04

Patrick Campbell

Wow. So why Dunkin? Why did you choose Dunkin Donuts? Does the brand matter? Or is it more just like, whatever, coffee?

00:16:03:10 - 00:16:04:04

Kaitlyn Gustafson

Because they're better?

00:16:04:10 - 00:16:09:05

Patrick Campbell

That's awesome. And how many cups of coffee do you drink? How many cups of coffee do you think you drink per week or per day.

00:16:09:05 - 00:16:13:13

Kaitlyn Gustafson

Or week Per day? I would say three, but three a day.

00:16:13:17 - 00:16:14:09

Patrick Campbell

There's a second.

00:16:15:04 - 00:16:19:19

Kaitlyn Gustafson

Till one in the morning and one in the evening, probably at one coffee a day every three months.

00:16:19:22 - 00:16:53:17

Patrick Campbell

Every three months, once every three months. Thank you so much. Appreciate it. Nice to meet you. Appreciate it. Thank you so much. That was awesome. Thank you, Patrick. Nice to meet you. We had I was going to do a little pound there. There you go. All right. Thank you. I should ask that guy. He was very angry. So doing your qualitative customer development, it can be really, really tough, especially when you're literally going on the street and basically asking strangers for information.

00:16:54:00 - 00:17:14:12

Patrick Campbell

As you saw there, a bunch of people were just hard nosed. They didn't really want to talk to me. But it's really, really important because at the end of the day, you have to get out of the building and actually talk to the people who are going to be the ones theoretically purchasing your product. You're getting out there because you're trying to get over the preconceived notions that you have and hypotheses that you have.

00:17:14:12 - 00:17:19:07

Patrick Campbell

You're trying to validate and validate them. And so it's so important to start.

00:17:19:07 - 00:17:20:08

Kaitlyn Gustafson

Qualitatively.

00:17:20:17 - 00:17:40:05

Patrick Campbell

Because at the end of the day, you're going to have even preconceived notions about the data that you're going to collect. And the way that you start qualitatively is really getting out there, finding the right people and ultimately asking the right questions. Now, finding the right people will probably be a lot easier than I just went out and found the right people because you're going to get warm introductions.

00:17:40:11 - 00:18:05:22

Patrick Campbell

You already have people who are using your product that you can actually email and hopefully get a good response back. But ultimately the questions are key. And to go a little bit deeper on how to ask these particular questions, I got a little help from Dez Trainor, the founder and head of strategy over at Intercom, who's going to go a little bit deeper into what's a good question and what's a bad question.

00:18:05:22 - 00:18:25:12

Des Traynor

It's always useful to ask to avoid dead end questions. So like dead end questions are things like, you know, are you doing this or is like, will you do this? Or how many times to be doing this? Or is it true that now these are all dead end because they can be answered with yes or no and you won't get much data out of it?

00:18:25:12 - 00:18:48:19

Des Traynor

Right. The definition of a dead end question is that it can be answered very, very touristy without giving you only context or information that's useful whatsoever. And there are lots of them. So the opposite of dead end questions are what we call the for W.H., which is if you ask questions like who uses the product in, in, in your company or what do they use it for or when do they use it or where do they use it or why do they use it or how do they use it?

00:18:49:03 - 00:19:11:00

Des Traynor

They're all much more valuable questions to ask because they're open ended. You will get more information out of them when you're asking questions. This is my least favorite thing to they don't. Couple independent questions like do you eat healthy and exercise? I do One of those things How usable and well-designed is our product? How reliable and fast is our product?

00:19:11:05 - 00:19:36:21

Des Traynor

You're asking two different questions and I'm going to answer with like, you know, it's really, really good. Yeah. And they're like, Really good. What? And like, you didn't specify or how important is the reports? And projections feature two different features. Did you know also provide a meaningful timeframe for them? So like if you say do you exercise, I think everyone like will sort of answer yes, but like a better question actually hits that reality is like in the past week, like how many times have you exercise in the past week?

00:19:36:21 - 00:19:57:00

Des Traynor

How did you think about exercise in the past week? What are your plans for exercise in the past week? Like? They're actually what gets to reality, basically, or, you know, so like it also helps you to avoid aspirational stuff because you're focused on reality. Focus on recent real recent realistic behavior.

00:19:57:00 - 00:20:19:17

Patrick Campbell

Does is background here on how you ask the questions is absolutely crucial because you're going to be trying to move from a point of ambiguity to a point of understanding. When you start to hear the same answers over and over again to the people that you're interviewing, you know, you're on to something. In our pursuit of figuring out which aspects of a cup of coffee drive value, we started to hear the same thing.

00:20:19:17 - 00:20:46:06

Patrick Campbell

Around 47 people in building a software company here at Provo. Well, and price intelligently, we typically start to hear the same answer, probably around 20 to 30 people on average. When it came to coffee, though, the main attributes people thought about when it came to why they drink, the coffee that they do were first brand, second taste, third temperature, fourth country of origin and fifth the price.

00:20:47:00 - 00:21:08:23

Patrick Campbell

One other huge aspect of collecting data and information from customers is understanding who is responding with which answers. Because you want to segment these answers appropriately to find trends. And frankly, if you're just collecting data in a blind way, oftentimes the data isn't going to be that useful because the next question is going to be, well, who is saying this?

00:21:08:23 - 00:21:28:00

Patrick Campbell

Should they be credible? Are they our target customer? And yes, you're trying to find your target customer. So there's a bit of a chicken or the egg here. But ultimately, that's why you want some level of demographics and some level of psychographics of your respondents. In the case of our coffee research, we tend to define three main groups.

00:21:28:10 - 00:21:47:09

Patrick Campbell

First, we ended up talking to a lot of what you would call hipsters. They didn't really drink that much coffee, maybe one coffee per day, but the coffee they preferred was always very, very high grade and they cared deeply about the country of origin. Second, we talked to a lot of functional blue collar coffee drinkers, or at least that's what we're calling them.

00:21:47:18 - 00:22:05:21

Patrick Campbell

These are people who didn't really care too much about the fancier aspects of coffee, but they cared a lot about the price and the taste. Finally, the third group that we talked to a lot was the brand loyalists. These folks sometimes drink a lot or sometimes didn't drink much at all, but they always stuck with the brand that they knew.

00:22:06:02 - 00:22:31:22

Patrick Campbell

With a few hours of our time, we honed in on these attributes and these customer profiles because they kept coming up again and again in our interviews. Now, if we were making a quick small budget decision, maybe some ad copy that we were going to test this data, maybe enough in our research would end here. It probably was a lot just to test some copy in an ad, but we want to go deeper because what if we are making a really big decision?

00:22:32:05 - 00:22:54:12

Patrick Campbell

What if we were about to spend $1,000,000 in time and materials to bring this new brand or this new type of coffee to market? A lot of you don't think about your business in this way, but you really should because remember, research is feedback that shortens your learning cycle. And at this point in the market that we're all in, we're trying to make sure we're collecting information to validate our assumptions.

00:22:54:20 - 00:23:08:04

Patrick Campbell

But most importantly, we're trying to save ourselves as much time as we actually can. And that's where quantitative research can actually come into play to do that. Quantitative research. We are old friend math.

00:23:08:15 - 00:23:11:07

Kaitlyn Gustafson

That might have math. Now.

00:23:12:04 - 00:23:41:08

Patrick Campbell

Don't worry, though, this type of math is pretty straightforward. My background is in econometrics and statistics, but I promise you, if you haven't even taken a college level math course, you'll be able to do this all in Excel. You can make it super complicated if you want, but we typically recommend starting pretty simply because if you haven't done this research before, it's one of those things we want to get your confidence moving and we want you to start pretty small to basically get into the groove of doing this type of research.

00:23:41:20 - 00:24:09:15

Patrick Campbell

So to quantitatively test your ideas, we start with our qualitative findings. If you remember, we found that we had three groups of Target customers who kept talking about brand price taste, temperature and the country of origin when it came to the attributes they value in coffee. We spoke with 47 people, but now we want to expand this to a few hundred people that we can segment on attributes like how many cups of coffee they drink per day or what their household income looks like.

00:24:10:00 - 00:24:33:23

Patrick Campbell

This will give us more confidence in making decisions about our new coffee brand, and we'll provide a little bit of statistical rigor to essentially the assertions that we're going to make based on actual data now, rather than just kind of qualitative anecdotes to get this data, we're going to think back to our seventh grade science classes, which I don't know about you, but was basically the time of my life at Bajaur Middle School in West Bend, Wisconsin.

00:24:34:07 - 00:24:53:11

Patrick Campbell

And we're going to use one of the most important things we learned in that science class, which is the scientific method, which means to answer our research question, we want to go out. We want to collect data from the right people in these groups. And this is essentially setting up our experimental design, which is a fancy way of saying who are we going to ask which questions?

00:24:53:22 - 00:25:16:21

Patrick Campbell

We then want to take that data from those questions clean out of outliers, clean it of bad data, clean it of as much bias as we can, and then we want to run some calculations on that data. We'll finally look through those calculations, analyze them and make some decisions based on that data. For our business to collect data from hundreds of different people, we're going to need a preexisting list of people.

00:25:16:21 - 00:25:43:02

Patrick Campbell

But also, depending on your current situation, we're going to use market panelists. Now, market panelists are essentially people who are your Target customers, but you're accessing them through a third party whose entire goal is to sell you access to a certain demographic of respondents to take a survey. These companies are wide and vast and can get you access to anyone from a soccer mom and dad in the middle of Kansas all the way to a Fortune 500 CEO in South Africa.

00:25:43:12 - 00:26:05:13

Patrick Campbell

There's definitely a cost here, though, but think about this in the context of time. A survey targeting small businesses or an end consumer can actually get filled with responses in a matter of hours for less than $1 per response, the Fortune 500 CIO may take a couple of weeks and cost hundreds of dollars per response, but you're trading time for money here.

00:26:05:18 - 00:26:29:20

Patrick Campbell

And if you ask questions in the right manner, essentially that data and that research is going to be really, really powerful in shaving more and more time off of your development cycles. Now, depending on your research question, we typically recommend collecting data from people who have heard of you who are in your preexisting lists, your existing leads, your existing customers, and then people who probably haven't these market panelists.

00:26:30:07 - 00:26:49:07

Patrick Campbell

Next up, we need to ask the right questions, which normally starts with demographics. For our coffee example, we're going to ask our respondents general demographic questions like their age, their gender, their household income and the like. But we're also going to ask them about their coffee consumption, what's their favorite brand, how much coffee do they drink per day, etc..

00:26:49:23 - 00:27:15:16

Patrick Campbell

Keep in mind, you shouldn't ask questions you already know the answer to some of the worst experiences I've had taking surveys and probably the reason most of you out there end up not liking surveys is when the first question after the surveys emailed to you or me is quote, What's your email address? You've got mail. Obviously this person isn't respecting my time and it's also training me not to really respect the survey.

00:27:16:03 - 00:27:38:16

Patrick Campbell

Also, if you're not compensating your survey. Respondent Really, the survey needs to be under a minute, which is maybe five questions max. If you compensating that particular respondent for their time, then you can get 15 minutes of that person's time before the quality of responses tend to go down beyond demographic questions. We also want to make sure we're asking questions that get us the right answer.

00:27:39:01 - 00:28:02:23

Patrick Campbell

Unlike qualitative questions that does spoke about here, we want to get more specific and most importantly, for respondents to make decisions when it comes to comparing our attributes the brand, the price, the taste, etc. We recommend using something called MAX diff, which essentially presents the respondent with the different attributes and then forces them to pick a most and least important option.

00:28:03:11 - 00:28:34:09

Patrick Campbell

So in the case of our coffee example, we showed respondents the five options brand taste, temperature, country of origin and price, and then ask them when you're choosing coffee, which of the following is most important? Which of the following is least important? By forcing the respondent to make a decision and doing some basic math that will put in the shownotes, we're getting high value data, and in the case of our coffee example, we found that out of 200 respondents, most people really care about the taste.

00:28:34:16 - 00:29:04:13

Patrick Campbell

And then there's very few people, but some of them exist who care about the country of origin, and everyone was pretty indifferent about things like temperature, meaning it's not really a valuable feature, but it's also not really a least valuable feature either. Now what's really fascinating is when you start to slice this data by some of the segment or demographic information that we collected in our coffee example, we notice that as people drink more and more coffee, the less they start to care about the country of origin and the more they start to care about price.

00:29:05:10 - 00:29:35:05

Patrick Campbell

Now, what's fascinating is this other axis of value that we talked about previously is around the actual willingness to pay. And when it comes to willingness to pay, everyone wants some sort of secret formula that basically breaks down how you can get into someone's mind and actually understand their actual value for your product. But in reality, collecting willingness to pay data is actually much simpler rather than kind of tricking people or maybe asking them point blank, which is a really hard question for someone to answer.

00:29:35:13 - 00:29:56:16

Patrick Campbell

We can take advantage of what psychologists and economists have been studying for decades, which is the simple fact that the way that we think about value as human beings is on a spectrum. We know, for example, that this computer or this phone that you're listening to this on is definitely more valuable than maybe the cup of coffee or the water bottle that might be sitting next to you.

00:29:57:02 - 00:30:19:21

Patrick Campbell

Now, if I change your situation, they put you in the middle of the desert and don't have you drink any water for four or five days, then that bottle of water is definitely going to become more valuable than that actual computer. This is how we think about value on this spectrum as humans. So we can actually take advantage of this by asking people ranged questions when it comes to value.

00:30:19:21 - 00:30:40:22

Patrick Campbell

So there's four key questions that we recommend that you collect when you're trying to figure out willingness to pay. The first question is, and in the context of our coffee example, is at what point is a pound of coffee way too expensive that you would never consider purchasing it? At what point is that pound of coffee getting expensive, but you'd still consider purchasing it.

00:30:41:17 - 00:31:08:20

Patrick Campbell

At what point is that pound of coffee a really, really good deal? And finally, at what point is that pound of coffee too cheap that you'd question the quality of it? Now, if I was collecting information about a very specific type of coffee or a certain origin of coffee or some sort of flavor profile of coffee, I would want to make sure that it actually prompt the survey respondent with some information about the product that I was about to ask them about.

00:31:09:06 - 00:31:27:17

Patrick Campbell

Similarly, if I was asking about a software product or some sort of subscription service, I would definitely say at what monthly price point would this be? Way too expensive and things like that. The important part here is this is a tool that you can use to apply to your product and also the aim of the research question that you're putting out.

00:31:27:17 - 00:31:57:03

Patrick Campbell

Now. What's fascinating is that when we slice our coffee data, we start to learn that the more coffee that people drink, the less they're actually willing to pay. Per pound of coffee. And what's really fascinating is when we start to cross-reference this willingness to pay data with the relative preference data that we talked about previously, meaning if we take the people who really, really care about taste and compare them to the people who care about country of origin, interestingly enough, the people who care about taste are more of them.

00:31:57:03 - 00:32:19:02

Patrick Campbell

And those people are actually willing to pay 30% higher than the median of the entire group now. Well, there weren't a lot of people who cared about country of origin. Those that did care about country of origin were willing to pay about 40% higher than the median of the group. Now, how do we use this data? Well, maybe we want to be a bespoke offering, exotic country of origin type coffee.

00:32:19:10 - 00:32:39:22

Patrick Campbell

Or maybe we want to go to the mass market and basically go after the higher coffee drinkers that only care about tastes and are willing to pay for lots of different cups of coffee. But all of a sudden, with a few questions, we've now taken those 47 qualitative respondents and basically explore did them with data that can be segmented to our heart's content.

00:32:40:10 - 00:33:00:23

Patrick Campbell

And some of you listening to the podcast version here may be a little confused. It's a lot to follow via audio hack. Even some of you watching the video may be a little bit confused, but don't fret if you go to protect the hustle icon, we have a whole list of supplemental resources, including 150 page book that we wrote that goes deep into these methodologies.

00:33:01:05 - 00:33:21:08

Patrick Campbell

The important part here is we increase the fidelity of the data we needed to make deeper and deeper decisions. There's also a whole host of other customer development tips and tricks that we'll link to in the show notes. But the big thing, again, we're not necessarily taking exactly what the customer said and only following what the customer values.

00:33:21:16 - 00:33:52:06

Patrick Campbell

We're instead taking this data, filtering it through, and then making decisions based on our business goals and ultimately where we want to go in terms of that customer value to build our business. All that being said, I can feel some of you through the speakers here exclaiming that there's going to be bias and that Steve Jobs didn't do customer research.

00:33:52:06 - 00:34:03:10

Patrick Campbell

So you need to either. Well, before you get too far, let's walk through a bunch of fun objections to doing customer research one by one to give you some perspective. Objection.

00:34:03:10 - 00:34:09:14

Kaitlyn Gustafson

One If I asked customers what they wanted, they would have told me a faster horse. They can't tell you what you should build.

00:34:09:20 - 00:34:32:19

Patrick Campbell

So a couple of things here. This is exactly right. But keep in mind, you're not asking them what the solution should be. This is the biggest misconception when it comes to doing customer research. You're asking them about their problem, what they value and everything around the solution. When you get into user testing, you will likely show them what you've built for reaction.

00:34:33:04 - 00:34:57:18

Patrick Campbell

But the larger point here is that your job as an operator is to take in all of this data and input out there, filter it, and then make some decisions. If you think you're God's gift to product and building a business, then keep shooting from the hip. Ignore all the data. But keep in mind, all of the market data indicates that we're really bad at understanding where value truly is when it comes to our customers.

00:34:58:03 - 00:35:24:23

Patrick Campbell

And unfortunately, it's getting worse because of the density of the market. Also, just as a side note, Apple definitely is one of the top five spenders on market research every single year. And Steve Jobs, although definitely someone who was ahead of his time, was ahead of his time, and it took multiple decades for basically the market to catch up to his dream of having a personal computer in every single home.

00:35:25:08 - 00:35:52:02

Patrick Campbell

It doesn't take away from his accomplishment, but it does show with research he may not have jumped to the personal computer as early as he did, mainly because it wasn't necessarily going to be great for the business. And with Henry Ford, who have a bigger bone to pick with, you don't really realize that after Ford released the Model T, they actually got spooked by GM because GM realized that people wanted more colors in a car than just black.

00:35:52:02 - 00:36:24:16

Patrick Campbell

And they also wanted different features and different styles of cars where Ford basically said, We're not going to do this. We're only going to stick with a black boxy Model T car. So long story short, you can dig your heels in, but there are going to be some consequences. So it's a careful balance of, yes, building the future and not asking your customers if they want that faster horse and instead making sure you understand the problem, understand the market, and ultimately filter all of this data together to build the right product and build the right company.

00:36:25:20 - 00:36:47:17

Patrick Campbell

Objection to Isn't this all just going to be biased? How many respondents is statistically significant? This is a great question, and I have an enormous secret for you that we are releasing exclusively on this podcast. All data is biased. You're never truly going to know if that feature that you ship is going to be well received or not until you actually ship it.

00:36:48:08 - 00:37:09:07

Patrick Campbell

But keep in mind, we follow best practices that have been developed over centuries to clean as much bias out of data as possible. We remove outliers, we remove incomplete data. We use questions that remove bias by asking in the right manner, and we make sure to collect enough data to get to the statistical significance that's warranted for the decision that we're trying to make.

00:37:09:17 - 00:37:34:06

Patrick Campbell

And remember, statistical significance isn't a specific number. It's a measure of variance. So the number that makes something statistically significant is going to fluctuate. We've included a bunch more on this, including a book we wrote on statistics for the subscription executive, so you can brush up on your number chops. But keep in mind, the power of data is not in the actual numbers, it's in knowing the limits of your data.

00:37:34:15 - 00:37:57:05

Patrick Campbell

I'm not going to make a $10 million decision on ten data points, but I'm also not going to collect 10,000 data points to determine what I should test my hard copy with as the example we used previously shows. Ultimately, data has bias, but you're using data to basically hedge as much risk as possible. So when you put something in the market, you're basically hedging that you're right.

00:37:58:07 - 00:38:15:20

Patrick Campbell

Objection three. Oh my God, isn't this too much work? I'm exhausted just listening to this. I really never understand the work argument. I'm going to be really frank with you because on one hand, building anything great requires work, but on the other hand, building a company isn't a battle between work and not work. It's a battle of efficiency.

00:38:15:20 - 00:38:35:03

Patrick Campbell

As we've spoken about at length, feedback is a much cheaper type of work than shipping stuff and just seeing how it does in the market. In the coffee example we went through above, we probably now know more about the Boston coffee market than any local coffee roaster with an afternoon's worth of work, but it's probably not more than Starbucks.

00:38:35:03 - 00:38:59:13

Patrick Campbell

Dunkin Donuts and humble coffee because they spend a lot of money on this type of research so they can make smart decisions. Ultimately, when you're hedging your bets and being more efficient, it's going to take some work. But keep in mind, you can't brute force your way to success anymore, as is indicated by all of the market data that we are seeing and the rest of the market is seeing objection for it.

00:38:59:13 - 00:39:03:17

Kaitlyn Gustafson

Does it matter once they collect the data because it's just going to be obsolete once I get it?

00:39:03:20 - 00:39:34:00

Patrick Campbell

I really like this objection because it's not entirely wrong. Data doesn't live on a shelf forever. That being said, this is why customer research is an ongoing process. If you're buying market research once a year, you're probably feeling pretty good about yourself, like you're doing a really great job, but in reality you're basically missing the point. You should be collecting data even if it's a little bit on a quarterly basis or just baking, getting feedback and different data into an ongoing process within your support structure or your sales team.

00:39:34:06 - 00:39:54:23

Patrick Campbell

And just within your product and marketing teams, because you never know when you're going to want to look back in a month and basically see that data or at least see a trend over a longer period of time, you should be constantly caching the data that you're collecting, understanding that most market research is going to expire every six months or so as the market and your customer moves.

00:39:56:06 - 00:40:13:18

Patrick Campbell

Objection. Five What if I don't have anyone to talk to you? I can't find people to survey or interview. I'm afraid that if this is true, you have a bigger problem. There's no excuse for not being able to find people to interview or collect data from. There are your current customers and if you don't have those, then there's your prospects.

00:40:13:18 - 00:40:31:15

Patrick Campbell

If you don't have those, then there's market panelists that you can buy. If you can't find any people through those channels and you can can't get any objections from anyone and you can't get any ads to basically drive people to take a survey. And you can't stand outside where your target customer frequents to ask people for an interview.

00:40:32:00 - 00:40:53:16

Patrick Campbell

Then basically you're either lazy and wrong or you just don't have a viable business. Think of getting research responses kind of as a proxy for acquiring a customer. It's so rare that I almost say it's impossible that somehow you can't buy someone's time to interview them, but they'd be willing to go through a sales process. It just doesn't make any sense.

00:40:53:16 - 00:41:16:07

Patrick Campbell

And you probably just need to try much harder to get the research going within your business. So I feel you through the speakers again. And this whole notion of Jesus, that was a lot. And you're totally right. So let's recap this mofo and get you underway to research success. Ben, do you have the research group ready to go get the beat for me?

00:41:16:08 - 00:41:50:02

Patrick Campbell

The freestyle here. Got it going. Let's hear it. Yeah, I'm feeling it. Okay. Yeah. All right. One, two, three. No, The last thing we need is another startup CEO about something related to software. Let's just do a straight up recap. First up, remember customer research. It's essentially shortening your cycles to building a great business. Your job is to create the quickest path to learning in order to make sure you have a flywheel moving of understanding your customer and building that which provides your customer value.

00:41:50:10 - 00:42:12:05

Patrick Campbell

Since the market is moving so quickly, you can't just guess and check your way to building the right product. You need to be more diligent around why you're building what you're building, especially since all data indicates what our product teams think they're building for. Value is not what's actually providing value. Next, there's three layers to good customer research.

00:42:12:14 - 00:42:50:08

Patrick Campbell

First, a good research question. Don't go too broad that you're basically trying to answer everything. And a good rule here is that you want to be able to answer the question within a reasonable level of certainty within 4 to 6 weeks. Next, go after qualitative research, go out of the building and actually talk to folks. You don't have to get out of the actual physical building, but make sure you're talking to folks in a high fidelity manner, asking them broad questions so you can basically test your own biases and start to figure out as people begin to repeat things, what you should go deeper on depending on the scope of your question.

00:42:50:12 - 00:43:10:13

Patrick Campbell

And the final piece of the research puzzle is go quantitative. If the question or scope of the risk you're trying to warrant it, don't be afraid to go quantitative by collecting data from the people you want to learn from and using the right question and survey methodology to medicate your bias as much as possible. Data in the aggregate isn't going to be that interesting though.

00:43:10:20 - 00:43:30:00

Patrick Campbell

So remember to segment segments and segment some more. Now, the final piece of this whole puzzle is keep in mind that your goal is to understand your customer better than anyone else, because that's going to let you make the best decisions when it comes to your business. You're not trying to get your customer to tell you the answer, what to build and all that kind of fun stuff.

00:43:30:09 - 00:43:58:16

Patrick Campbell

You're trying to get them to give you feedback in a scaled and organized manner to properly shorten your cycles of learning. You got to stay on that quickest path to learning. That's what's crucial to success. After running countless, I mean, we could count them, but after running a lot of different research studies, both internally and externally for different customers, what I recommend doing is making sure you start small and create a culture of feedback and customer research.

00:43:59:02 - 00:44:18:21

Patrick Campbell

Give this research focus to a PM or marketing manager for 20% of their time and. I promise you it'll pay dividends. It doesn't even have to be that complicated. It can be related to exactly what you're already building, just getting some validation and some context around the feature that you're going to go out and put so much time into.

00:44:19:07 - 00:44:40:17

Patrick Campbell

There are dozens of other resources out there on different methodologies and just thousands of tactics from simply adding a piece to your support tickets or your sales tickets to running good user testing for design and UI research. We've link to a bunch of them in the show notes below so you can explore to your heart's content. The important piece here, though, is just a start.

00:44:41:05 - 00:45:04:18

Patrick Campbell

If you don't take anything away from this talk, make sure that you go out and just for the love of God, talk to your customers or your target customers. Put a number on the board. Ten research calls ten research conversations, one survey that you're going to send. Just some scalable thing that you can do to make sure that you're actually moving towards a culture of doing research.

00:45:05:03 - 00:45:31:19

Patrick Campbell

The market is moving too quickly and you just can't rely on your gut anymore to build greatness because guessing and checking your way to success just isn't an option anymore. It's arrogant and just going to set you up for failure. Instead, do your research, filter your research and keep that feedback coming. Because at the end of the day, research is the only thing that's going to help you get closest to the customer.

00:45:32:08 - 00:45:40:03

Patrick Campbell

And if you remember from previous conversations before, whoever gets closest to the customer is ultimately going to win.

00:45:46:09 - 00:46:10:01

Ben Hillman

And protect us all is produced by Dan Callahan and Ben Hillman with help from Robert Byrne and Alissa Chan, written and produced by Patrick Campbell. We are giving away copies of Steve Blank's book. All you need to do is head over to iTunes. Give us a five star review and leave a nice healthy comp screenshot that and send it to Patrick at profit welcome and we'll send you your much deserved copy of the book.

00:46:10:05 - 00:46:27:01

Ben Hillman

This week's episode is brought to you by Sweet Fish Media, allowing you to partner with your ideal clients to create industry specific content through an interview based podcast Sweet Fish Media dot com.