Double click on the love
A micro case study in debugging retention and unlocking growth
I've had several conversations with product leaders recently who want to grow their product to the next level. In each case, one of their top three KPIs has stagnated. They’re feeling the pressure.
Read below, or listen now on Apple, Spotify, Google, Overcast, or Youtube.
The initial impulse is to dig into the data and figure out what's going on. But if you start from the wrong question, you can easily mislead yourself and draw a logical conclusion that is wrong. We can always interrogate data to make it fit a narrative.
“If you torture the data long enough, it will confess to anything.”
– Ronald Coase, renowned British economist
The evil of premature scaling
If premature optimization is the root of all evil in engineering, premature scaling may well be the root of all evil in product growth.
A common mistake that growth-seeking product leaders make is to try and turn on the growth jets too soon. It’s a common cause for high-potential products to either die on the vine or implode—especially among founding CEOs. It's incredibly tempting to pour all those funds raised into a growth engine! That's what all them moneyz are for, amiright?!!
Yes, and: it's a bad idea to ignite fuel before the rocket is tightened up, all systems go, and pointed at the right place. Rockets are much harder to course correct once launched: small differences in bearing lead to massively different destinations over time, and change is harder with momentum.
The getting-ready-to-grow phase is a time where leaders need to exercise real discipline, resist the pressure, and think.
Let's make this more concrete with an example.
Leaky bucket, seeking growth (mini case study)
Here’s the context:
You're the head of product at a Series B startup. You feel pressure to scale the product, and to do so yesterday.
KPI: 28-day retention, tracked in weekly cohorts
Average cohort retention has flatlined around 25% for the last 3-4 months.
Based on your product category, your goal is to get this number up to ~50%.
New accounts get a 2-week free trial
First choice: do you dig deeper into the 25% group, or the 75% group?
You say: the 75%, obviously—that's where we're losing the most people!
Digging into churn data
So off you go, and this is what you find:
What’s going on here? You dig around the rest of the analytics, and get a bigger picture:
Now you have even more questions! Such as:
What's up with that 47% of users (63% of churners) that never did much? Were they just kicking the tires? Were they the wrong people in the first place, i.e. the problem is higher in the funnel? Did they ever even activate?
What about the 21% customers (28% churners) that canceled around day 10? Why are they canceling then, and not earlier or later?
Maybe we should focus on onboarding? A ton of accounts never really activated. Or what about that 9% of churners activated and looked promising, then ghosted.
These are all reasonable, logical questions. And they will mislead you.
Which question are you trying to answer?
The key is to figure out what question you're really asking. There are at least three different questions in the background here:
What's going on with onboarding & activation—why are we losing so many customers there?
Why are some people using the product a bunch, and then bailing before the trial is up?
What do the best customers love about this product?
What makes this hard is that these are all good questions. None is absolutely right or wrong.
It's a sequencing issue. The right question depends on where you are.
Given the data above, this product is not ready for growth yet. The overall goal to get retention to 50% is good. But the starting point for growth is not fixing the leaky bucket. Fixing the leaky bucket is step two.
Step one is to understand the people who love and use your product the most.
What you really want is lots more accounts in that situation. If you could 10x, 20x, 100x only your best customers, wouldn't you want that?
This is easy to miss. The people you need to focus on most are the smallest group numerically, at just 6% of all accounts (24% of the 25% retained). The clue is in the overall low retention and small percentage of highly-activated users.
If there were 100 accounts in this example cohort, the answer you need—step one for growth—will be found in the 6 customers that love the product and use it almost every day. Not the 75 customers lost. Not the 19 low-to-moderate usage customers.
Focus on the 6, not the 94.
You can dig into the cancelations and onboarding all you want—and you definitely should before you start pouring on the rocket fuel—but those people will never tell you what to build. They can’t tell you where the magic is, only what’s getting in the way of it.
The core value of the product—the “magic moment”—lives with the best customers, who love your product the most. First, understand them better than they understand themselves. Once you understand what the magic is, and the value it unlocks, then you go to work on the rest of the funnel. Then you know where you’re leading these customers that have chosen to trust you.
Your best customers are the strategic beachhead to more people like them. They are the signal in the noise.
You aren't building for the haters or tire-kickers.
Double click on the love.
If you have any questions about this article, or are interested in exploring product coaching, please reach out here.
Takeaways
Get the lay of the land before you zoom in too much (in this case, if you only looked at the 75% you'd never even see the tiny 6% where the real answer is hidden)
Prior to product market fit (PMF), growth strategies are mostly a waste of money. You must know where the magic moment is in order to (a) position/market your product well, and (b) get users within the product to that magic ASAP.
If you're trying to find the magic in the product, zoom in on the users who love it most.
If you're working on scaling (or prepping to scale), and already have demonstrably strong PMF and retention, fixing the leaky bucket is where to focus. You must know where the magic is before you try to scale. Otherwise, you’re flying blind.
If you aren’t sure you know where the magic moment is, you don’t. It’s like being in love. If you’re asking the question, you have your answer.
If you don't have analytics in place to even start to answer these questions, please go fix that now.
Thanks to Travis Corrigan, Jordi Bas, and Eric Lodge for thoughtful feedback on drafts of this.