Discovering Perfectly Predictable Mistakes – Before You Make Them

There are no failures in Thoughtland – that mystical place where great ideas for new products and services are first born. But if you want to travel to the real world, you’ll need a different approach.

26 years ago, my co-author and I published “Discovery Driven Planning” in the Harvard Business Review.  It took off in a big way in the innovation community.  Clayton Christensen was an enthusiast and included it as a chapter in his 1997 book The Innovator’s Dilemma. IDEO and Innosight brought the practices in the article to life with their clients.  We eventually responded to pleas from people who wanted a book-length version, which became Discovery Driven Growth.  

Then came 2011 and the publication of Eric Reis’ The Lean Startup and whoa – all of a sudden everybody was talking about planning to learn, keeping things fast and cheap and testing aspects of a business before committing to the whole thing.  As Steve Blank observed in his 2013 article, “The Lean Startup Changes Everything.” 

So let’s review.

The Core Idea: Managing the assumption : knowledge ratio

When you are in a predictable environment, your ratio of assumptions relative to knowledge is low. If I had wanted to predict sales of strategy textbooks to undergraduate students for 2019, they would have been remarkably similar to the sale of strategy textbooks in 2018.  In the contrasting case, where you are in a highly uncertain situation, your ratio of assumptions to knowledge goes way up. 

Human beings are terrible at processing assumptions.

We forget them altogether.  Russ Ackoff, a Professor at the Wharton School, found that the half-life of an organization’s ability to remember critical assumptions was….wait for it….6 weeks!  Yes, after 6 weeks nobody involved was able to remember critical assumptions. 

Even worse, we take our assumptions and magically turn them into facts in our minds.  The result?  Business plans riddled with untested assumptions that lurk away there until exposure to reality reveals them to be untrue.    

Even worse than that, we confuse our preferences with predictions. We refuse to take in any information that might disconfirm those preferences.  The fancy name for this is the confirmation bias – the tendency to embrace information that supports previously held convictions and reject information that calls those into question. Ever tried to convince someone to take a different position than one they hold by burying them with facts?  Did it work?  Thought not.  

So, we realized, what was needed was a framework which could allow people, even those who have quite different ideas about the facts (which will happen under genuine uncertainty) to have a productive, common experience for learning what they need to learn while pursuing a new business.  Hence, a discovery driven, learning based focus for planning.    

Start with the future, and work backward

The first thing you do in a discovery driven plan is define what success would look like at some point in the future – say three to five years, when you are past the launch stage and your offering is in the market.  Success could be dollars of profit. It could be impact, as in a social venture. It could be in amount of behavioral change.  Whatever you pick, you’re going to drive the rest of the plan backward from that.  

The second part of defining success is creating the reverse income statement or reverse frame for the business.  So let’s take a business that is burgeoning right now among the typing classes – the emergence of individual paywall subscriptions that you might put up on a platform such as Substack.  

In Thoughtland, this sounds like an awesome idea – write for your readers without pesky editors and publishing deadlines! Get paid for what you love to do! Create your own personal brand that might compete with other media properties!  Forget about paid word counts.  Fantastic.  Even better, you ask friends and family whether they think it’s a good idea and would they subscribe? And they all say yes.  I see a glorious future of self-defined publishing and freedom on the way.  

So what would it take for you to give up whatever your day job is and become a personally paid freelancer?   I have no idea.  But let’s say as a starting point you’d like to make a middle-class income which in many places, according to the Washington Post, is $100,000 per year.  So let’s keep it easy and set up our model like this:

Required Profit from my subscription business of $100,000.

The next thing we have to do is figure out what this might imply in terms of how much revenue your subscriptions would have to bring in.  At this point, someone usually says, “well, that’s obvious, $100,000.  Not so fast.  Don’t forget, Substack isn’t a charity.  They’ll charge 10% of whatever your subscription price is.  

So you’re going to have to earn $111,111 in subscription revenue to make your profit number.  

Another tiny little issue… how are your subscribers going to pay you?  Through Stripe, Substack’s payments provider of choice.  They will charge 2.9% of the transaction, plus a 30 cent fee per transaction.  Meaning if your subscription price is $10, let’s say, Substack is already going to take $1, so that leaves you with $9.  Stripe will take another 2.9%, or 29 cents, plus 30 cents for each $10 transaction.  

So, keeping it simple, let’s assume that each $10 of subscription revenue comes with costs to you of $1 for Substack hosting, $.29 for Stripe processing and another $.30 for Stripe transaction fees.  That means almost 16% in revenue is going to disappear before that payment ever touches your bank account.  So instead of having to sell $100,000 worth of subscriptions, you need to sell $119,048 to make your desired profit.  Is that realistic? Let’s think about it.  

Sticking with the $10 subscription price, to keep the numbers simple, that means you need to sell $9,921 worth of subscriptions per month.  That in turn implies that you need to have 992 regular monthly subscribers to make your numbers. 

But wait, how are those subscribers going to find out about your great subscription site?  This brings us to a difficult discipline we call the deliverables specification.

Getting real: deliverables specification

In a deliverables specification, you think through everything that is going to be needed, at a high level, to literally deliver the business.  

In this model, you have to think about how customers are going to enter your for-pay universe.  How many visitors to your Substack space are going to become paying subscribers? Services like Piano offer some ideas about what is realistic to expect.   They point out a wide variation, from .02 percent to nearly 100 percent.  Let’s say your conversion rate – of visitors to paid people is 20% (which would be really high).  In other words, you need 5 visitors to generate 1 subscription.  So your required visitor number is about 5,000 new people to visit some early part of the marketing funnel for your subscription.  

I won’t bore you by going on and on about this, but you can very easily see that getting enough regular subscribers for you to live on your writing is going to be a big lift in terms of PR, outreach and so on.  And don’t forget – every bit of additional cost for promotion, PR, packaging, social media and everything else is going to increase your required revenue number.  I call these allowable costs, which is the difference between the revenue you need to make and the profits you need to pull out to make it all worthwhile.  

In the case of a subscription business, you’d also need to think about how much of your time your writing, the promotion, and managing your subscriber base is going to take.  Working through this forces you to think hard about what would have to be true for this to be a success.  

Benchmarking:  Market discipline

At this point we’ve created a reverse income statement and put in our current assumptions (an assumption is something like the ratio of visitors to buyers).  At least we’ve got a few pinned down so that we can test them.  That isn’t enough though – you’re in a world swimming in potential subscriptions as well as all kinds of other media people could consume.  This brings us to the next phase of the discovery driven plan, which is to do a little market testing.

How much do people earn on Substack?  Well, according to this source, the top earners can make as much as $39,000 per month.  Superstar writers like Heather Cox Richardson who writes the newsletter “letters from an American” makes over $1,000,000 annually from her thoughtful daily takes on the news of the day.  But she is a major exception.  How will you break through into followership such as that?

You will also want to look into the pocket of resources that you’ll be tapping into.  A recent report said that Americans spend $20.78 a month on app subscriptions and about $33 per month on entertainment subscriptions.  A web site called Next Gen Financial Services estimates that a typical household spends $237 on subscriptions every month, many completely underestimating how many are hitting their credit cards on a regular basis.  

So you’d want to see whether that revenue you are planning to generate looks crazy or not.  

Documenting Deliverables Specifications and Assumptions

The next phase in doing a plan like this is to continue what we’ve been doing, which is writing the critical operating considerations (things like conversion rates) down, as well as the assumptions we’re making when we do that.

In the case of a business like this, you’d want to document the assumptions you are making about customer journeys, value delivered, and of course things like how many touchpoints you’ll need to have with a customer before they become a subscriber, what happens to those who unsubscribe (and at what rate), what’s going to happen if you have an emergency and can’t pump out those newsletters, and so on.

The real action of the plan begins in the next phase, which is planning by key checkpoints.

The Checkpoint Planning Approach

The biggest difference between a discovery driven plan and a more conventional one is that you don’t plan the whole thing (with the hockey-stick diagram of revenues in the spreadsheets at the back).  What you do is frame it in terms of what would have to be true for various outcomes to occur.  Then you design (or encounter) checkpoints at which you will test your assumptions.  Think of a checkpoint as the opportunity for an experiment.  

For this business, an important hypothesis to test is whether readers have an interest in your topic at all!  You’re making the assumption, of course, that they do.  So a checkpoint might be to open up a Substack account, create your newsletter, offer it for free, promote it on free places like Twitter and LinkedIn and see if anyone subscribes to the free version.  If you can’t even get people to do that, you’re highly unlikely to get a lot of paid subscribers.  So that might be your first checkpoint. 

Then you’d want to test conversion to paid.  Say you’ve been at the free version for 3 months and have acquired some folks who are interested.  Next, articulate what Alberto Savoia calls the “x y z” form of a hypothesis.  X =  percent of target market.  Y=target market.  Z=desired action.  In your case, that “x percent of existing free subscribers will convert to becoming paid subscribers at the $10 /month level.”  That’s what doing an experiment is all about.  If you expected 20% to subscribe to paid, and got 2%, well, you’re going to have to generate a lot more unpaid people to potentially sell to, which means you might need to go back to the framing numbers and increase the outreach…you get the idea.

Wrapping up

Discovery driven planning helps you get to important answers in your business, quickly.  A couple of additional points.  

• Don’t argue about who’s right. Argue about what the right experiment is that could yield an answer.
• Don’t reject conflicting hypotheses to your own preferred one.  Let the data speak for themselves.
• Keep your spending and cost as low as you can till you have evidence that your spending won’t be wasted.
• Don’t get market feedback from family and friends – they are likely not representative of your target market!
• Don’t let your colleagues get away with vague references to “billion dollar markets” or some such nonsense. Figure out what would make this effort worthwhile and build that.

Over the years, we’ve used DDP with startups, corporate ventures, acquisitions, not for profits, you name it.  It really works in high uncertainty situations.