Succeeding at the default metrics will kill you

Folks who know nothing about your business will evaluate it based on the default metrics: traffic & revenue. They will advise you to increase these 2 numbers. Accepting the default metrics is bad.

The top cause of startup death is trying to grow before the foundation is solid. The default metrics are dangerous because they compel you to scale prematurely and kill your company.

It’s tempting, too. Everyone understands and is impressed by a growing traffic graph. It’s harder for people (including your own team) to intuit the implications of your retention rate.

Plus, you don’t even have to make a conscious decision to use these bad metrics. They’re default. You must actively reject them.

It is therefore doubly important to be rigorous.

Picking good metrics

The lean startup book’s engines of growth is a great starting point for picking good metrics:

• Growing virally? Focus on referral.
• Users never leave (e.g. subscription webapps)? Focus on retention.
• Buying new customers via ads or salespeople? Focus on cost of acquisition versus lifetime value.

If those aren’t relevant then take your pick from amongst McClure’s pirate metrics, Ellis’ 40% disappointed test, the net promoter score, or something of your own creation.

Ultimately, it’s the CEOs job to decide on the metrics you care about[1]. It’s one of the most important things you’ll ever do. Learn from the starting points listed above and then make your own decision based on what your company needs.

The metrics you care about will also evolve over time. When one has been refined to near perfection then it’s time to shift some focus to quantifying the next big open question.

Slipping back to the deadly defaults

Here’s the rub: you know that vanity metrics are bad and that traffic is [usually] a vanity metric. So you’re ignoring your traffic for now to focus on “product”. That’s not good enough.

You need to be rigorous about your numbers and hold yourself accountable or you’ll slip up at two predictable points:

1. When you’re convincing other people how great you are
2. When you’re deciding where to invest your team’s finite energy

We once re-designed a site 3 times because “not enough people are using it.” Poor usage seemed like the sort of thing a “better” site would fix. We wasted our time there not because we were stupid, but rather because we hadn’t spent the time to figure out which metrics did matter at the time. So we slipped back to the defaults.

Another time, I was pitching a VC and mentioned a bit of adoring press coverage we had received. I intended it as evidence that the product was resonating and we were building something people loved. She responded with “Yes, I saw that spike on your Alexa graph… It looks like it went back down.” The moment I failed to re-orient her toward the metrics which mattered, I had lost the meeting and the pitch. We were now talking about the site’s crappy traffic instead of its fanatical early adopters.

My first company’s first product, Fuzzwich, was once demoed on live TV in front of 7mm ideal users. We had a traffic graph that looked like a telephone pole: straight up and straight down. The cruel irony is that we had spent the preceding months ignoring our one crucial metric (referral) while trying to figure out how to use clever marketing to get new visitors in the door. When we “succeeded” and the traffic windfall finally came, we weren’t in a position to reap the rewards. We had focused on the default metrics in an attempt to seem impressive and had built our early traction on a foundation of sand.

Figure out which numbers matter to your business and don’t let anyone frame the conversation or decision back around the defaults.

 

[1] The acronym (crapronym?) for these is KPIs, which means Key Performance Indicators. The term carries a lot of baggage (in the form of vanity metrics) from the corporate world, so I try to avoid it. Still, you should know the word for when you talk to investors.