profitwell paddle

BIG NEWS: Paddle acquires ProfitWell to "do it for you"

Dan Wolchonok on Running Retention Experiments

Jordan T. McBride Jun 14 2020

Note: Dan Wolchonok of Hubspot presented the following framework during his talk at SaaSFest 2015. We're going to break it down further and explore how you can take your business to the next level by running retention experiments to compound growth.



Let's face it. There is no band-aid for crappy retention. It doesn’t matter how cool or great your product is, or how skilled you are with the latest growth hacks. “If your retention is poor then nothing else matters” - a quotation from Brian Balfour that holds true everyday of the week. The main objective of user/consumer engagement is to attract new users and keep them coming back to use your product.

Getting users hooked on your product is great; but sustainable growth isn't achievable until you understand the multifaceted dimensions of your retention. Recently at SaaSFest, Dan Wolchonok of Hubspot gave a phenomenal talk on this very subject and spoke about everything there is to know about retention and how to improve it for your business.

In case you missed the talk at SaaSFest, we’ve got you covered with this post. Moving forward, we will take a look at Dan’s framework and break down the most important aspects of retention.

To master retention, you must know what good retention is, how to break it down into actionable segments, and be willing to experiment to achieve sustainable growth. So let’s hop right into it.

"If your retention is poor, then nothing else matters - Brian Balfour"ico-twitter-yellow

Understanding Bad Retention v. Good Retention

When it comes to retention there are two types that your business will come across: bad retention and good retention. The differences between the two of these represents the difference between making money from cohorts over time, or losing users and dollars. To gain more understanding, let's talk about bad retention first.


Bad Retention

Simply, bad retention happens when you are churning users at essentially the same rate you are gaining them.

Looking at the cohort example below we can see that the company is gaining about 10k users per month. The chart represents bad retention because what’s happening is for every 10k users gained, the previous 10k is lost. The previous users are dragging the retention down creating a stagnant effect that eventually just plateaus the growth of the user base. When you have poor retention the cohorts inevitably stack on top of one another amounting to nothing in the long run.



Good Retention

Good retention is a bit different. Instead of losing users at the same rate, you gain them and you retain a percentage of your original cohorts as those people stick around over time. You essentially get layers of cohorts on top of one another as you can see below and you continue to grow your monthly users instead of flat-lining.

With good retention, your cohorts will become more valuable overtime.  When you capture a new cohort in month 1, they aren't really worth much monetarily. As time passes, you get users upgrading, attaching add-ons, etc., thus creating an exponential growth curve for your business as you can see in Dan’s example below.


Bad retention prohibits your business from growing. It’s like going to pull water from a well with knitted sock - it just isn’t sustainable. An ideal cohort analysis should have that beautiful exponential curve that shows you are retaining a percentage of previous cohorts and compounding the growth over time. With that being said, let’s move forward and take a deeper look at how you actually obtain and maintain the good retention growth curve.

"Bad retention is like trying to pull water from a well with a knitted sock - it just isn't sustainable"ico-twitter-yellow

How to Segment Retention

If you’re looking at retention as one big nebulous concept, you're probably setting yourself up for failure. To truly understand the metric, you need to break it down into different segments, as Dan outlined during in his SaaSFest presentation. Here’s a look at the different segmentations you should have for retention.


Segment 1

 Short-term Retention - Week 1 is the first retention test that your business needs to pass. The benchmark here is: “Can you get your users to use your product more than once?” 

Segment 2

Mid-term retention - After you clear week 1 retention, the next benchmark is: “Can you establish any pattern of usage?” This is the area where you want to create habits with your users. If they have a problem or need, your product should be the solution that serves as their “go-to."

Segment 3

Long-term retention - This is the portion of your retention where you figure out: “How does your product become an indispensable tool?" In other words, how do you get users to become so hooked on your offering that it's a part of their daily routine.

Running Retention Experiments

Later in Dan’s talk, he raises the question of “how do you improve retention?” The tactical goal for improving your retention curve is to shift the curve up and make it plateau.

To shift the curve up you want to fine tune your week 1 retention. It’s the best spot to improve because it cascades the improvements onto the rest of the retention segments.

If you’re going for truly authentic growth and you’re able to convey the value of your product immediately within that first week, these things will be reflected in your cohort analysis and indicate compounding growth. To flatten out the retention curve, there are several areas that you need to optimize through experimentation and data collection:


Feature tweaks

Try fine tuning features that may be confusing to users or not functioning as they should. Here's an easy way to make up for any shortcomings in the product as well as making a huge retention impact.

Retention hooks

Employ mechanisms that keep users coming back to your product. This will definitely require testing to see what works as a hook and what doesn't work. None the less, your team should be able to pin point some key things that get users coming back time and time again 

Example: LinkedIn's emails to their users saying "Who's viewed your profile" is a great tactic for getting users to come back to their site.


Product quality

Make your product bigger, better, stronger, faster. There's plenty of data to collect around this category. Talk with your users because they are always happy to tell you what's lacking from your product. Use that information to work out any kinks that may be keeping the product from reaching its full potential. 



Reactivate lost users.

    • Example: Evernote does a great job of getting users to come back over time by reimphisizing the value of the product. This causes users to re-download the app any time they switch devices, creating the smile graph you see below.


It’s important to master these four elements of experimentation because it’s something over which your business has complete control. A lot of time, we, SaaS companies, create a product that we’re so excited about and can’t fathom someone not understanding it. Yet, not understanding what your customer needs or how your customer uses your product will surely lead to a failed product.  

Dan actually opened up about Hubspot's failed product experiences and gave us some retention experiment examples from their Sidekick tool which had some trouble in the early days.

The Sidekick tool had problems such as too many CTA's, too many notifications, and users not having enough time to figure it out. With all the feedback and data that was collected from users, Dan and the Hubspot team were able to make some changes that simplified the experience of the product. Simplifying the user experience is key to reducing churn and increasing retention. A lot of churn is caused by confusion of the product and a clunky user experience leads to frustration and annoyance.

Going through the experimentation process is what led Dan and the Hubspot team to a solution that polished the product and made users happy. So, you should talk to users and discover their pain points with the product. Use that data to drive your retention experiments. Learn from the experiment results, and repeat this process until you have a polished product that is aligned with your cusotmers' needs.


It's worth repeating: as Brian Balfour says - “If your retention is poor then nothing else matters.”


Check out Dan Wolchonok’s SaaSFest deck and be sure to share it with a team member. :)

SaaSFest 2015 - "Retention is King" by Dan Wolchonok of HubSpot from Price Intelligently

By Jordan T. McBride

Subscription market insights you won't find anywhere else.