We've been historically focused on outputs from our work, but Nate Walkingshaw from Pluralsight discusses how to reframe our thinking and work backwards from desired outcomes to build more effective product teams.

In this presentation from SaaSFest 2017, Nate manages to find a meaningful metaphor for outcome-based teams in the unlikely setting of a mowed lawn.

Starting out making $7.14/hour as an EMT

My name is Nate Walkingshaw and this slide really represents a lot of the things that I've done in my career. 

I started out as an emergency medical technician making $7.14 an hour, and I invented a whole series of medical devices. I started in hardware, and then migrated from hardware into software IOT, and then took my career into software development. If you really were to tie where my heart still lives, it still lives around patient care - around fire and EMS.

We're gonna go through a whole series of emotions and events as I go through my presentation. For those of you who haven't seen my presentations in the past, I’ll reference the product talk that I gave in June called The Heartbeat of a Product, and my goal here really is to serve you. 

Product development at Pluralsight

This slide here is how I think about product development - this is what I think about product management for software, and there are a couple of things that I really, really care about. At Pluralsight, we're the world's largest developer, IT, and creative training library in the world, and the mission of the company is to create progress through technology that lifts the human condition.

We have 25 functional, co-located product development teams all with product management, user experience, design, engineering, data science, and content - this is the collection of these human beings, and we're human centered. In the last three years we've done over 7,000 qualitative interviews - just a massive set of interviews - to truly understand what Patrick was talking about regarding building products and features that your customers actually value.

I don't believe in features; I believe in experiences. I believe in unified user experiences that are contiguous; that when somebody drops into your product, they have this very holistic feeling. When I got to Pluralsight, the product was basically a library. We had 6,000 courses and we called it a bookshelf, and it was great. You could Command+F and then look for a course and you could find what you were looking for.

We just thought we could do something a little bit different. What we thought about was, not only do we want a native web application, but we'd like Roku, Chromecast, and Apple TV.

We understand learning styles through data science and engineering; we really wanted to go deep on how people would experience learning, and for all of you that are running your product, my hope is that as we go through this journey today, I want you to put yourself in in my chair and think about your product from this chair.

Outputs vs. outcomes - the future of product development

Now today we are going to talk about outputs versus outcomes - it's one small section, however, I think that this section specifically is one of the most important, because it's the journey that we're in the middle of in product management today. We've come from waterfall product development, then we moved to agile, then scrum fall/scrum bond - we really don't know where we are, and I really want to give you my perspective on it.

Historically, we've been really focused on outputs, and what does that mean? It means that we've been committed to shipping a thing at the end of the quarter, and at the end of the quarter that thing gets a Viking send-off, and then we hope and pray that this new experience is going to engage, surprise, and delight our customers, but we’re really not measuring it. We really don't know if what we're shipping is having the impact that we want on the business, and when I talk about impact my context really comes from three core KPIs, or metrics - headlights, taillights, and happiness.

Headlights is usage - how somebody uses our product. Taillights are billings, but it really is the intention of an outcome that I'm going to share. And the last piece Patrick talked about is Net Promoter Score, so happiness. When I got to Pluralsight, our NPS score was 42. We have a whole series of personas, and our aggregate NPS today is around 70, and we climbed that over the course of three years.

Modern-day Industrial Revolution

The other frame of reference that I'd like you to know is that we're authoring the pages, and I don't know if you understand this or not, but data science, data engineering, augmented reality - all of these things that are occurring right now I liken to the Industrial Revolution.

Somebody walked into a cotton field, picked up a piece of cotton, and we're wearing clothes today. You look at the cotton gin, sewing machine - all of those things collectively came together and built the textile industry; the Industrial Revolution built cities, it built all these amazing things that we have.

Here at MIT, there was a graduating class that invented this nanotechnology bot that got infused into fabric. This bot causes the fabric to actually breathe like human skin, and I love this because this is the collection of textiles, technology, three-dimensional printing, and biotechnology all coming together in one ecosystem. We're authoring the pages of this - we're finding out what we're ultimately going to become, and it might be 10 or 20 years from now, but we'll get there.

Inspired by a lawnmower

This is creating the whole product story. The origin of this was really inspired by a lawnmower, and I know that might be weird, but I run every single morning and when I go on this perpetual run, I've been thinking about the problem around outputs versus outcomes and how we get the whole industry to build software products around an idea that isn't committed to continually shipping just outputs - to really dig deep around what an outcome for their company looks like.

This is really the story of a lawn mower. I usually sit down in a room and ask someone to draw a lawnmower. Mike Baird, the guy that helped me design a lot of this presentation, he actually illustrated this lawnmower. When I asked him to draw a lawnmower, I said, “would you mind telling me the most important parts of a lawn mower? Could you give me the morphology of it? Could you design it for me? And inherently, these are some of the things that happen.

The next thing I say is, would you mind placing the word “team” by each of the components. I have no idea how Honda does this - this is all just a story for context that I made up, but if I was Honda I can imagine that they have a propulsion team, that they have an engine team, that they've got a handle team - they've got a whole bunch of cross-functional, co-located, autonomous teams working on a product called a lawnmower. 

Designing human-centric teams

When I talk about building teams around a product and around a lawnmower, they have to look a certain way. Patrick alluded to this - about companies building software without talking to their customers. To me, this is the core - you have to have a product manager, a user experience designer, an engineer, and then whatever your core competency is, working on that experience.

Our core competency is data science, engineering, and content - content is a huge piece of our core competency, so whatever your company's core competency is, this is the foundational team, and then you supplement it with whatever your core is. My preference is for them to be co-located, so they can all see the sausage being made at the same time. 

Outcomes and measuring happiness over time

When Honda gets together and they go design their lawnmower, let's hope that they've organized the teams correctly. Let's hope that they’re cross-functional, co-located, and then the thing that I've drawn in the inside of here is the intention - this is an outcome, okay? And the outcome is very different from how a product is used. The outcome looks like 12,000 hours, 6,000 mows, 0.38 acres. This is what the lawnmower needs to do over the course of a life.

I think that this would represent probably 80% of the customer demographic that would use a lawnmower, and when you're looking at the use compared to the outcome, you realize that as the person is using your product, you have this happiness over time curve, and the hope is that it will make it through 20 or 25 years of somebody doing this every single week to mow their lawn.

Sharing the narrative to achieve the best outcomes

If you really break apart the user story and the life of the mower, what ends up happening is you begin to see something pretty clear. When I've ran massive product teams, what occurs is that we have these teams that get in each other's way constantly - there's constant infighting on who owns the narrative, who owns the story, who works on what, and just to be clear, all of these teams share the narrative, but they work cross-functionally together. In software products this happens to be a huge problem because they all think that they own the narrative in its entirety.

I have the outcome of the mower that I ended up shipping, but there's also a service provider - somebody that actually maintains the product. And when I come back to the service provider, these teams have been working on the iterations of the lawnmower and I could extend the life of that mower over time. This is really the impetus - everyone is working on the narrative, everyone is working on the user story, but it's going to affect the outcome of what we work on over time. 

Solving complex product problems using corner cases

I love talking about how you solve massive product problems at scale. If you were at Honda and you had to go build a lawnmower for a hundred million different people across the world in 150 different countries, that's a daunting task and we're kind of up against the same thing when we're building our software products.

This is a case that I really care about around corner cases. Most of the time every complex product problem that I've solved over the last 20 years really comes from focusing on the extreme. When I go talk to customers about a lawnmower, I'm not going to talk to someone in Utah that lives on the Upper East Side bench that mows his or her lawn every single week. What I would like to do is talk to the person that uses a mower 40 times per day, because what happens when I do that is it usually covers 80% of your core market.

More importantly, you also get the outliers that end up being one-step adjacencies. This could be a trimmer - I wouldn't want a mower to do a trimmers job. I wouldn't want a mower to do a blowers job. The mower needs to do the mowers job and this is really useful and really helpful in software.

Sandy Parks and Rec - Honda's corner case

I also like to show an as-lived example, and this really was the whole purpose of wanting to to give this talk. This is this run that I go on every single morning, and this person is Colby. Colby's missing something - he's missing a bag, and you're gonna see what happened.

This is Sandy Parks and Rec in Salt Lake City, Utah and their job every single day is to mow 40 or 50 lawns. This is an 8 acre park - it’s massive. When I was running around this park, I end up running through the grass trimmings, and it's super frustrating! I'm like, why don't you have your bag on Colby? It's ridiculous, and if you were to go back to Dante there's a little green bag sitting on the ground. Just think about the thought that she's taken into this; not only is she going to mow it, but she's already got the bag to bag the grass so no blades of grass get anywhere on the concrete.

It's cool the way that she's thinking about it, but Colby - he's a mess! And I was like what in the hell is the deal with that? I want you just to ignore the amount of sweat dripping off my forehead because there's gonna be a lot of it, and I just want you to listen to what Dante says:

Nate: “All right Dante, tell me why Colby doesn't put the bag on there.”

Dante: “Because he's mowing the squares, and when he puts his bag on it will hit the bottom and scrape the bottom, so then the grass is gonna come out anyway.”

Nate: “Yeah, that’s awesome - thanks man.”

Do you guys understand what happened there - do you understand what's going on? Colby’s hitting park strips all day, and when Colby hits a park strip it creates this massive mess of grass everywhere. I live in Salt Lake City - the elevation changes from 2,000 to 5,500 feet and we call it the bench, so everything that occurs in Utah has an incline on it. You're trying to tell me that this mower - every single time I hit the park strip that has an incline - rips the bag off and dumps grass.

I won't tell you what the math is, but you could assume that Colby makes between $10 and $15 an hour, and you could look at Sandy City Parks and Rec, and the cost of that single lawn mower and the time and the money that's being spent on a design flaw by not talking to customers; by not talking to the corner case of what's occurring for probably hundreds of millions of mowers that exist in the marketplace for years. That's crazy.

The outcome, if I look at the happiness over time metric - Colby and this team is really impacted. Think about all of the individuals that live on the bench around the world or on any type of park strip that has an incline - this has a massive impact.

Using the narrative to stay focused on your personas

I always want to go back to personas, to these people that we designed for. The customers we should be talking to. You've got the manufacturer Honda, I've got a corner case that uses a bag, I've got corner cases that don’t use bags - that's an extreme use case - and then I've got this service provider, and the way that I want to measure the impact or the outcome of this is happiness over time. I wanted to plot out a graph around how this occurs for me, so on the far right - this isn't me - this is more like my father-in-law. My father-in-law is amazing; he's had his mower for like 30 years, and he services it, he takes it in for random service, he winterizes it. This is really me - I use a lawn mower, and I never service it, and then I replace it every five to seven years and Honda loves me as a customer.

You could look at total wallet share for services over time - hopefully your mind is going there. You could look at replacement cost of mowers. All these different people that are thinking about using this mower at scale. The other thing you can count on with users is - peeps, you know, they be doing weird things with lawnmowers. This is one thing that you can count on and I just want to say when you're designing a product for scale we tend to listen to these types of customers.

When you do customer interviews, you listen to the loud majority that tell you that there's a problem with your product, when in actuality they're not even using it through your intention, through your outcome, through the 12,000 hours, 6,000 mows, 0.38 acres - they're not following this simple narrative that the cross-functional, co-located team is on. They're using it weirdly - I wouldn't want to build a team around that use case, or that narrative, or that user story. You could imagine what it potentially could look like and that brings me back to versioning and how you think about versioning your product.

Putting metrics in your product to measure outcomes

I'll go back to that simple narrative - when these teams are working cross-functionally on this narrative, each one of those teams is going to pull just a sentence out in it, and this is the thing that they're going to perpetually innovate over time. I always want to come back to Honda - and this is just a regular push mower, but if you think about the entire line of Honda mowers that exist, there's a trained team focused on wheels; they own the version of every single one of those wheels, just like our product teams today.

We have something called learning paths, or skill paths - they own skill paths in perpetuity forever, and the version control of that. They also own an outcome in our product called the time to learn, or the time it takes someone to effectively apply their knowledge. And so when you're thinking about versioning, they thought about - this was the gas cap that they designed, and this is a ungloved hand. They also think about what a gloved hand, or maybe a new round gas cap could look like and how that affects retention and happiness over time. You're constantly thinking about that because it can go either way; you could launch something into a mower that they come in for a service interval, or launch something into your product and happiness and retention goes down. You want to constantly put metrics into your product that are monitoring retention to find out likelihood to buy, length of stay, usage - all of these key components that we care about driving usage, billings and happiness when we're versioning these experiences really matters.

Know the outcome you want at the end of your narrative

I'm gonna go all the way back to something that we care about a ton, which is tying this back to software. I would just ask - what is your intention? I know our intention within our product, and this could be akin to the same outcome that I've given you for the lawnmower. The intention of our product, or the outcome of our product, is this time to effectively apply learning. We're a learning company, so we better care that when a human being comes into our product, they actually get taught something and then apply it in their life. It took us three years to learn that the time to learn and the time to effectively apply that knowledge is the key metric that we needed to care about.

Do you know what the outcome you would like to have for this person at the end of your narrative, at the end of your use case, at the end of your story? This doesn't directly apply to Pluralsight - it should apply to the way that you build products every single day. We looked at the corner case; the corner case for Pluralsight looked like somebody that had to learn in a very short window of time, like their career depended on it. They were trying to move up in a role, or maybe had just come out of college and they were getting a job for their very first time and they were trying to move from novice to expert. We're in 150 different countries, three and a half million accounts, so we were able to look at that section of hyper users and trust me - it solved the problem for nearly everyone in our product at scale, and I'm going to show you how it worked.

This was our corner case persona. There are lots of goals and objectives for each individual learner, and it really just solved the problem. I've got three different personas and depending on the size of the company you're calling on it's going to yoyo. This one person - if it's a small and emerging commercial company, all three of these people could be one person - or if you're a big enterprise company, these three people could be totally separate human beings, but the things that they're going to do within your product experience are going to be very similar. And it's a shared outcome - at the very back of this is how your product is going to stitch an experience together to create an outcome over here called time to learn.

Empowering product teams to build narratives with customers

I want to wrap it up. I think the the biggest piece for me is, when you get together with your product organization, leading with an intention and leading with an outcome is probably one of the best things that you could possibly do. I don't know how you're organized, but how you're organized matters so much. If these teams aren't human centered, if they don't have product management, UX, and engineering building the simple narrative, building the product all holistically together and then talking to the customer before they ever build a thing, I think you're really going to struggle to have product-market fit. The other thing is that when you don't build the right thing, when you don't ship the right version, and the engineering team gets to see customer relation or customer dissatisfaction - man, that doesn't massively motivate the population of engineers to go solve that problem over time.

The other thing is - they want to write the user story or the narrative with you. The shared outcomes and shared narratives, I think that's just systemic within our products, and a problem that we're working to solve. I think data science and data engineering is really going to help us, and measuring happiness and retention and versioning your products over time is really, really key.

The thing the last thing I'd wrap up with is I've written a ton about this; you guys are getting a very small subset of the overall narrative and story. There's a book that I wrote called product leadership and there's a ton of content and medium types out there. The one thing I would like to do is just give Patrick and the Price Intelligently team a plug. We've used these guys for the last three years and when I got to Pluralsight our pricing was a mess. We really were a B2C business, and we built an enterprise B2B platform. Over the last three years we just didn't have an enterprise worthy product; we were mostly small and emerging in mid-market, and Patrick and this team really helped us partner, and the way that I thought about them was really from a financial perspective - they're kind of like my audit committee. They're my third party neutral away from my confirmation bias on what people should really experience and they offered a perspective and a view from an analytical perspective that we really couldn't bring to the table.