CryptocurrencyInvestingBanksReal Estate

How startups can use metrics to drive success

April 5, 2011, 7:18 PM UTC

Mark Suster joined GRP Partners in 2007 as a general partner after selling his company to He focuses on early-stage technology companies. He blogs at

One of the things I discuss the most with the portfolio companies I’m involved with is that “you manage what you measure.”

It’s a very important concept for me because startups are constantly under pressure and have way too many distractions. Having a set of metrics that you watch and that you feel are the key drivers of your success helps keep clarity.

And the more public you can make your goals for these key metrics the better. Make them widely available inside the company and share your most important goals with your board. Transparency of goals drives performance because it creates both a commitment and a sense of urgency.

Commitment and urgency are key drivers of success in startup businesses.

You already know it from your personal lives. The surest way to run a marathon is to tell everybody you’re going to do it (transparency). Even better is to tell them which race you’re going to run in the near future (urgency). The best yet is to raise money from them for a good cause – then you’re sure to run it (commitment). Nobody likes to raise money then look like a loser.

I ran my first marathon in London this way in 2003 raising $3,000 for Parkinson’s disease (and finishing in under 4 hours — my publicly stated goal). For what it’s worth, my private goal was 3:45 but I missed that.

I know with the recent emphasis on measurement from Dave McClure and Eric Reis, you’d think everybody is measuring. My experience has proven that even some well known companies are under-whelming in this department.

On measurement

I was recently talking with a startup that wanted me to try its product, a mobile app that crashed too much for my liking. In our next meeting I asked if it crashed often. Only one guy in the room knew –the tech lead.

He told me in some combinations of device / OS / network they are crashing 4 times per 100. I’m a big believer in product stability and performance before adding too many features. Once you churn a user due to stability or performance problems, it can be hard to get them back.

4 times / 100 means if a customer uses your app frequently (say 10-20 times / day) then they are crashing nearly every day. That’s not acceptable.

But what is the industry standard? Is it 4/1,000? 1/1,000? And given your stage of development, you better at least know what your goal is. All applications crash and this is especially true in the nascent mobile world, where dealing with device types, operating systems, and networks adds one hell of a configuration management problem.

What I know for sure is that if you don’t have a stability goal stated for the company and if you don’t regularly measure how you’re doing against this goal you won’t have your resources focused on the right priorities in the company.

Most companies have some measurements, but I would argue that people often measure the wrong stuff, measure with the wrong precision (either too high of a level or sometimes too detailed to draw conclusions). I see this more often than I see good practices.

The best way is to start by asking yourself at the management team level: what are our company objectives and how do we best measure them? Because it can be hard to define or agree on company objectives at an early stage, I believe most people avoid them.

Don’t. If you change your company’s objectives or measurements later, that’s fine. In fact, I would argue that if you’re producing charts that nobody is reading or acting on, you’re probably measuring the wrong stuff.

And if you’re not meeting as a team to discuss these metrics and have a regular debate about how you’re doing and what needs to change, then I can assure you that you’ll never reach your destination. You’ll have no idea when you’re off course.

You will likely have multiple sets of metrics you keep, depending on the company’s stage, one’s function in the company and level. For example, I highly recommend a set of board metrics that the CEO communicates to board members at every meeting. With a set of metrics, the board can know whether the company is on track with its objectives.

Here are some measurements I think about. How you implement them will obviously depend on the type of company you have — there is no “one size fits all” approach but there are pretty universal measures.

1. Customer Acquisition
At the highest level you’ll obviously want to track how many customers your adding every month (and for some businesses that have hit scale, this is measured on a daily basis). If you can break this down by the channel that you’ve acquired them from, this is obviously better.

How many adds came through organic SEO? How many through affiliate deals? How many through SEM? Do you have a customer referral program? If so, make sure you can track which leads come from this. Measuring viral adoption is obviously important.

Usually you have a catch-all bucket for “direct” or similar that often came through PR or word-of-mouth.

If you have multiple versions of your product, how many are web vs. mobile? How do the mobile customers break down by device type?

The next step after measuring the customers you’re adding is to add the “cost to acquire” by channel. This is important because it will later tell you whether you have a scalable business or not. In the early phases, if you can’t acquire customers cost-effectively enough, you’ll need to diagnose why and determine how to fix it.

Make sure that you count the “true” cost to acquire customers. For example, if you have developers, content people or SEO folks working on SEO programs, you’ll need to allocate their time / costs to this effort. SEO is seldom “free.”

It might sound obvious but if you’re paying $1.50 per click on an SEM basis, this is not your cost to acquire a customer — you need to add conversion rate. I see this mistake all the time, actually. So if you convert 12.5% of the people who click on Google paid links, then your true cost to acquire that user is actually = $12 ($1.50 / 0.125).

Now you can go with one of two methods to get your cost-to-acquire down. You can find out how to more cost effectively buy search terms (i.e. lowering $1.50 to $1.10) and you can focus on improving conversion (i.e. increasing conversion from 12.5% to 18%). Those two things together would lower your acquisition costs nearly in half to $6.11.

If you don’t have very clear metrics on how much you can make from a person who converts into a customer, you better not be spending $6.11 per customer! That’s for people with very clear monetization results from customers.

Ironically, there are times where it may actually pay to increase your customer acquisition costs. In a fast growing market where you have clear monetization that greatly exceeds your cost of acquisition, increasing your average acquisition costs can have two clear advantages: 1) you pick up a lot of additional customers that were falling off due to not buying enough ad inventory and 2) you make it harder for less optimized companies in the market to compete.

I suspect some of this is going on at Groupon and LivingSocial right now. Their monetization is so sick (LA-speak for good) right now that it’s hard to compete with them for customers — you have to have more clever sources of customer acquisition.

I’m guessing this was also the case over the first few year’s of Zynga’s growth on Facebook. Once they knew how much money they could make with virtual goods / customer then they seemed to buy up much of the Facebook ad inventory.

2. Retention and churn
Measuring customer acquisition is clearly not enough because not all customers stick around. This is especially true in the mobile space where apps are either free or cheap. At 99 cents they’re disposable.

Most people underestimate the challenge of winning “share of mind,” the least understood concept with tech entrepreneurs. Everybody thinks, “if I build this cool app people will come and use it.” Sure, but will they still be using it in a year? In six months? In three months?

The biggest limitation we tech consumers have is our time. How many social networks, picture sharing sites, new aggregators or blogs can we really spend time on? It has to come from somewhere. You need to win share of mind.

But there are other reasons people churn — low product quality, inability to understand the value of the product, costs, competitive products, etc.

You need to start by measuring your “churn” or attrition. I like to break this down into to buckets –immediate (think almost like a bounce rate on a website) and other churn. In the mobile world many apps are downloaded but never used or perhaps only used for one day.

This type of churn is likely different from garden-variety churn and therefore ought to be measured separately because the remedies are likely to be different. Fixing a problem with somebody who downloads your app, uses it once, and churns versus somebody who quits after 30 days will require very different solutions.

Make sure to poll your users to find out why they’re churning. The majority of churn doesn’t come from your app being deleted, it happens when the app just isn’t used. If you could send a message to a subset of these users and ask them why they didn’t use your product, you will probably learn a lot. One suggestion I give is to message them with a $5 Starbucks gift card. Many people will give you a small bit of time in exchange for a small gift

3. LTV
The other obvious measurement is the “lifetime value of a customer” or LTV. In the early stages of your company, you’ll have to estimate this because you don’t know how long each customer will stick around or how your monetization will change over time.

Many times businesses can get away without measuring this in the earliest phases but nonetheless it’s good to have a goal. If you plan to spend any serious amount of money on customer acquisition you better have a handle on LTV (or estimated LTV).

4. Revenue Metrics
Revenue metrics are one of the first things I ask for from the startups in which I invest. I like to think of revenue drivers. If you’re an ad business, for example, you’ll want to measure things such as impressions served, fill rate and eCPM (effective costs per 1,000 views).

Once you have a baseline then we can have a discussion every month about those three drivers: how are you doing at getting your impressions up, how are we doing on fill rate, and what is our eCPM? They are each independent components with different actions to improve performance.

And they are revenue drivers in that simplistically impressions x fill rate x eCPM equals revenue. At the highest level (and with a board) these are great metrics to keep focused on.

As you get more granular, you’ll start to break down premium inventory vs. remnant and you’ll measure “custom buys” (sponsorships) versus standard. Once you “bucket” your revenue into different types, you can have more intelligent conversations.

An example might be, for a mobile app company:

  • 35% of our revenue is coming from home page take-overs, we allow 2 / day
  • 40% of our revenue is coming from remnant banner ads served by ad networks
  • 10% of our revenue is coming from direct sales of our banner inventory
  • 15% is coming from in-app product sales (25% of these with cash, 75% with “incentivized offers.”)

Now we can have an intelligent discussion about the size & shape of your business.

  • Should we increase home page take-overs to 4x / day? Or will that ruin the user experience? Or should we be lowering it to 1x?
  • If we increase home page take-overs, can we reduce our total banner ad inventory to improve the user experience?
  • If we’re getting $1 eCPMs on banners sold through ad networks, could we focus on getting our direct fill rate up in stead where we get $15 eCPMs?
  • What would that take? How many people would we need to hire? How long would it take for us to recover their costs?

Metrics drive more intelligent conversations about your business amongst your management team, with investors and with knowledgeable advisors. No metrics = high level, more generalized advice.

5. Quality
Already stated above, but know what you’re shooting for in terms of load times, crashes, known bugs, etc.

6. Salesman Metrics
I don’t want to go in depth here because it could take a whole blog post, but make sure to have performance metrics in place if you have direct sales teams.

It’s obvious stuff you’ll want to measure: revenue / sales person, leads, win/loss ratios, etc.

Just be careful because nowhere is it more true that “you manage what you measure” than in sales. If you start measuring calls / day, call length, meetings / week, etc. and especially if you make the results public then you’ll notice a change in salesperson behavior.

If you measure the above metrics and believe they are the right ones for your business, that’s great. But in some businesses, call volumes might incentivize your reps to get off the phone quickly, which, in some businesses, is the wrong strategy.

So start having a discussion with your teams and your boards about what the right objectives of the company are and what are the best data to measure them. Don’t wait for others to give you the recipe — you’ll be waiting for a long time.

Happy measuring. If you have any good tips for others, feel free to leave them in the comments section.