Stats for Startups with Tomasz Tunguz
This interview is a part of our Stats for Startups series, brought to you by Charge Ventures and the Kauffmann Foundation. You can find the technical companion to this interview on the Stats for Startups website. If you are an investor with a quantitative take on venture, we’d love to hear from you: hello at Charge dot vc. Thanks for reading!
Brett
Welcome to Stats for Startups! It’s a quantitative look at early-stage financing. I’m Brett Martin from Charge Ventures. I’ll be your host. And today we’re going to be speaking with an old classmate of mine, Tom Tonguz. Tom is an investor at Redpoint Ventures, where he has led investments and a host of pioneering data-driven companies, including Looker, Chorus, Gremlin and Expensify, among others. When I thought about who would be a great fit for this interview series, Tom was the first person who came to mind. So we’re really thrilled to have him join us today to talk about startup metrics. Tom, if you want, just give us your quick, personal overview- a little more about yourself, and your work, and what gets you up in the morning.
Tom
Yeah for sure. My quick background is I started my career as an entrepreneur.
I started a business with my dad when I was 17 in South America, and I got bitten by the startup bug. And then after school, I went to work at a startup started by three of our classmates. That business actually ended up going public. Then I went to Google where I was a Product Manager on the AdSense team for a while, and then I joined Redpoint about 12 years ago and focused on early stage software companies. I had actually joined as a consumer associate but then moved into software because I loved it so much.
Brett
Yeah, I saw how that thredUP investment snuck in there!
Tom
Yeah! They’re still doing great. But, you know, that was a super quantitative business, and there were all kinds of concepts inside of that business that actually lend themselves to software. As a curious aside, I was just talking to a guy who was one of the first heads of marketing at Netflix. He was telling me the story of how a lot of the quantitative metrics in marketing actually started in and around Netflix. I had no idea about that history, and that was fascinating to hear.
Brett
As in the marketing folks were bringing the measurements in, and then deploying them into the SaaS model?
Tom
Yeah. So Netflix was doing cost of customer acquisition. They were the first ones to look at and define churn because they have a subscriber base and they wanted to figure out the unit economics of acquiring an individual customer. There was a marketing team there that actually developed a lot of the tools that we use today, like cost of customer acquisition, cohort analysis for churn, understanding customer LTV and looking at paybacks on a per channel basis.
That was something that was pioneered there.
But SaaS is not that. I mean it’s 20 years old, but a lot of the more sophisticated concepts are actually not that old. Maybe they’re like 12 years old. And so it actually started at Netflix and then came to the rest of the world because of the same way Netflix is a subscription service. The unit economics may be tougher, because typically consumers just churn more and they typically don’t expand the way that SaaS companies might expand on an annual basis and so then, SaaS companies started looking at that discipline and saying: Hey! Why don’t we use that here?
Brett
Interesting! You know, it’s funny. When I think of SaaS, I would have thought it went the other way around: people “meter”-sizing things for a business sales, B2B-type purpose, rather than coming from the consumer angle.
Listeners, if you haven’t already guessed, Tom is a very quantitative guy. In addition to his day job of investing, he also writes one of the best blogs on the Internet in my opinion, about SaaS metrics and is also the author of a book, right?
Tom
Yeah. Yep. But called actually ‘Winning with Data’ that I co-authored with the CEO of Looker.
Brett
And so Tom, were you always so quantitative in the way you looked at the world or was that something you developed for your job?
Tom
It’s definitely something that I developed for my job. When I started in venture, I was in this fortunate position to be attached to the hip of three really great venture capitalists at Redpoint. And I got to see all kinds of board meetings: Pitch meetings, Hirings/firings, public company board meetings, private company board meetings, and just started recognizing a pattern that entrepreneurs were asking questions like:
- How much ARR per CSM?
- What should my ARR per employee be?
- What should my payback period be, given my Annual Contract Value (ACV)?
I had worked at Google before, so I was surprised that a lot of these metrics weren’t on the Internet when I would go Google them, right? Like “Oh this is gonna exist.” And they didn’t!
So I was like, okay. If this one founder is asking the question, I’m sure lots of other founders were asking the same question and so I started doing the research and publishing it. And once you start asking questions, you keep pulling its strings, then other people will ask follow-up questions. And so that led to where we are today.
Brett
Questions beget more questions. You’ve focused on SaaS and you just gave us a quick preview on the origin of it. Stepping back because this is for early stage entrepreneurs: When you say SaaS what exactly does that mean? How does that business model differ from, say, e-commerce or other types of businesses and why do you focus there? Because you initially started in consumer but then you moved into SaaS: What got you [to switch]?
Tom:
The way I define SaaS is:
‘software that is sold to B2B companies and is paid for on a subscription basis.’
Initially, it was focused on things like Salesforce.com or Zendesk, so people really constrained SaaS to applications that end users use. But now it’s grown to encompass infrastructure, like MongoDB software; which is what you might call it infrastructure as a service (IaaS), and the idea is if I buy a database on some subscription basis for TWILIO I would fall into that category. So it’s growing. Most enterprise software companies today are SaaS.
The model that preceded it was licensed software. Remember when you used to go to CompUSA (for those few young enough to remember CompUSA)? You would go and literally buy a box with the CD-ROM in it and the software authors would get paid once. If you wanted to grow from 10 million last year to 20 million this year, you had to sell twice as many boxes. So that was a tough business model. There definitely was a recurring part, like a services component to it, but only about 10 or 15% of the initial sale actually recurred.
Licensed software is just a harder business, you see a lot more volatility. That was when people moved into ASPs — Application Service Providers in the early 2000’s. But then that very same concept was started to be called SaaS around the time of Salesforce. It was the Salesforce IPO. That’s sort of the stamp. The idea of SaaS was a really big wave.
The first reason that people like SaaS is if I book $10 million a year in revenue this year, I’m not going to see very much churn. So I’ll go into next year with $10 million already in the bag, so I just keep adding to my ARR.
The second part that people really like about it, is that accounts tend to expand. There’s a concept called net dollar retention, and the way they think about that is: it’s like a bank account with a really high-interest rate. For the best companies in the world on account that’s worth $100,000 last year is gonna be worth $160,000 this year. And if you’ve got lots of those, you don’t have to do anything and the business will grow at about 60%. That’s super compounding.
The last part that people really like about these businesses is when you sell these contracts, customers typically pay for the entire annual contract upfront. That’s basically a free loan from your customer because you provide the software from January through December. But you get all the cash on January 1st.
SaaS businesses differ from other companies for a bunch of reasons. If you take an e-commerce company, for example if you buy a mattress from Casper, you’re going to buy a mattress once, and you’re not going to come back next year and by another mattress in the year after that buy another mattress, and you might not buy 1.4 times the number of mattresses you bought last year. So that means that a consumer company (like a Casper) has to make all of their money on a customer in the first transaction. So all the AdWords that they spend in order to acquire a customer paying for all the operations of their business. All of that has to be paid with the first sale of a mattress.
And there’s an important concept we should introduce which is basically the quality of revenue of a company.
If you look at publicly traded companies like Salesforce.com or Sealy Posturepedic you’ll notice that the values of those businesses are different. And what I mean by that is: If Salesforce generates a billion in revenue, that (or whatever SaaS company generates a billion in revenue), it’s probably going to be worth something like $15 billion to $20 billion if it’s growing at a particular rate. If Casper goes public and does $1 billion in mattress sales, it’s probably going to be worth about a billion to one and half billion dollars.
So there’s a 10x difference in the value of that business. The same dollar of revenue is worth 10 times more. A lot of the businesses that you and I talk about if they and when they do go public, will trade as a multiple of their revenue.
Consumer companies typically trade at somewhere between 1 to 2 times. Marketplaces tend to trade at like, 5 to 6 times net revenue or one times GMV. And then SaaS companies right now are trading at all-time highs, like 15 to 20 times next year’s revenue.
And so that’s really important right? If you’re starting a business you should think about what multiple you’re going to get on this business. It may be equally hard to start both businesses but you may have a 10x or 15x more valuable business if you start a software company.
Brett
So in terms of why you chose SaaS, you follow the money, it sounds like.
Tom
Yeah. I wish I were more creative and I could look at a consumer company and imagine the future. You know, I think we all sort of admire a lot of the great product leaders and thinkers and I’m just missing that gene. And so it’s just a whole lot easier to look at numbers and try to predict from that.
Brett
Well, I’m sure your LPs are happy with that! So this is great; and it’s a great segue into the meat of the interview which is you know really around the metrics. The purpose of this series is to talk about investors in different categories, and what are the metrics that matter, in this case for SaaS? So Tom, what do you think of as the, two or three, most important SaaS metrics, and why are they important?
Tom
At the series A, the first metric that we care about is ARR growth or Annual Recurring Revenue growth. And, a lot of the times we’re investing in companies that grow between 3 to 5x or more. Sometimes it’s off a really small base and that’s just the reason that metric is so important — it shows that the company has demonstrated product market/fit and that there’s a lot of pull from the market.
ARR or sometimes as it’s called cARR, with lower case ‘c’ or ‘committed ARR’ is this is the sum total of all the contracts or sum total of the value of all the contracts the company has booked for a given period. If I look at all the contracts that are open and outstanding, it can be in month one or it could be in month eleven of the contract; if I sum all of the face value of those contracts over their annualized life, that will be my cARR.
There’s ARR which is what we just talked about, the sum total of all the value currently outstanding contracts. Then you have cash collections which is: when do my customers pay me?
And that’s just going to depend on when you close the contract and they’ll pay you. And then there’s your P&L statement, your profit and loss statement, which shows your revenue. And, in the past what people have done is if you have a contract that goes over 12 months you basically divide that. You divide the value of the contract by 12 and you recognize 1/12th of the value of the contract in each month. Your revenue is typically going to be lower, something like 50 to 70% of your ARR. And if you’re a company that sells a SaaS product and you deliver it on somebody else’s computer (that was called on “on-prem” or “cloud-prem”) you actually recognize all of the revenue in the first quarter of that contract.
The second most important metric is the payback period with gross margin burden payback period. The payback period answers the question: How many gross profit dollars can I generate from an incremental sales and marketing dollar? So, if I invest one more dollar into sales and marketing, how much gross profit can I get back? And we look at gross profit instead of revenue because lots of machine learning SaaS companies have come about, and their gross margins tend to be less than that of a typical software company. The median publicly traded SaaS company’s gross margin is about 72%.
But the difference between revenue and gross profit is COGS or Cost of Goods Sold. And that’s all the stuff that it takes to deliver the software, and typically, people in customer success, professional services, if they have that, and then cloud hosting costs. But you can imagine, for an AI company, you’re going to be spending a whole lot more on model training and so your margins will be thinner. That’s why we look at gross margin-burdened payback period, because you can normalize it across lots of different companies.
We just ran a survey across 520 companies actually. Most companies were looking at a GM-burdened payback period of something like 14 months. If you’re typically like a $5k ACV, you typically want to be on the lower end of that spectrum, but if you’ve got a $100k or $150k product, your averages are closer to 18 to 24 months and that’s totally, totally fine.
Shorter is always better, and the reason shorter is better is it basically means you’re able to grow faster. Because if you spent $10 million this year on sales and marketing and it takes you two years to get it back, you’re going to get that $10 million to come back within two years, but if you can get it back in six months and you can take it and invested twice within the course of a year, you grow doubly fast on the same cash!
Brett
Yep. So in that sense then, payback period is a measure of how quickly the company will be able to keep growing. But the first measure is how you know how much revenue the company is making — ARR. And the second measure is, Okay: How quickly will it be able to keep growing that?
Tom
Yep, exactly right. If we as investors invest $10 million and $1 million of that goes into sales and marketing, how much revenue of gross profit dollars do we expect from that investment over what time?
Brett
Got it. And if you only had a third metric to choose?
Tom
NDR — net dollar retention. Net dollar retention is another really terrific metric and that’s basically: “If I have a cohort of customers from last year and they paid me $100. How much do they pay me this year?” And that includes if they grew their accounts, shrank their accounts, or stayed the same.
Brett
Got it. So it’s really a measure of: Do people want it? How quickly can we keep growing it? And then once we’ve sold: How much more can we sell to the same folks?
Tom
That’s way better than my explanation! That’s so simple.
Brett
What would you say is: average, good, and great for each of those metrics?
Tom
Yeah I would say the average would probably be:
2.5x ARR growth 2.5x / 3.5x / 5x
On payback period, It would probably go 24 / 18 /14. Every once in a while you see a company that has 12. And then net dollar retention: The best I’ve ever seen is 220%. But the top decile is 160%. You know, the top quartile is around, like 125%. And then good would be, like 110%. You wouldn’t invest in a company that was below 105%.
Brett
Essentially, customers always need to be wanting more of the service each year, not less. Seems like a pretty reasonable metric. But to be fair, you know and part of this, the purpose of this podcast is actually not to just focus on you know the top-performing companies.
Because not every company starts as a top-performing company, right? Let’s take one of these metrics like ARR growth: What’s not good? Then if a company isn’t seeing 2x ARR growth, why not? What does that mean to you?
Tom
I mean so if a company isn’t growing that fast it could be for a couple of different reasons. The first reason could be that the market doesn’t want the product which is sort of the worst reason. And the initial hypothesis of the product just isn’t landing. The second reason could be this is a company that’s creating a new category and the market doesn’t yet know it wants the product, but the market will know that it wants the product.
You can imagine a category-creating company like Segment, which is the SaaS company that offers a great tool for marketers and that category didn’t really exist before Segment started. So they had to do a bunch of education. We’re investors in Gremlin, and they do chaos engineering. A lot of people don’t know what that is. That’s why you might see a slower growth rate in a company like that. The third reason might be that you have sort of mismatched your product with your go-to-market (GTM) strategy. This happens less and less, but in the beginning days of SaaS you might see a company with like a $5k, $10k a year product hiring super expensive salespeople to try to close those deals.
Brett
Yep. And they just don’t match and it doesn’t work. And so that’s why you might be the growth being really low. I mean, the first one: People don’t want it. Obviously. But the other two, feel like those could be improved, right? Those people can take action to improve their ARR growth.
Tom
Yeah! A lot of the times the slow growth is more driven by the internal operations, rather than by external market demand. If you’re not selling anything and nobody wants the product, that’s a pretty clear signal that it’s going to be really tough. But I think if you’re getting customers to close and a lot of companies have different data points with that, like one really fast sale cycle and then a whole bunch of really slow sales cycles. Or lots of people were running to take meetings, but no one sort of gets across the finish line or lots of people do a ‘proof of concept’ that no one gets across the finish line. That tends to be more of an internal operations process problem than it does a product problem.
Brett
Got it. So the growth might not be a problem with the market or even your product, but rather how you’re selling it, how you’re delivering it. And what about payback period? Obviously it’s great if you’re getting all your money back in 14 months. But what if it’s taking 3.5 years? What does that say about a company and its product?
Tom
So, if your payback period is 3.5 years, it probably means you’re spending money in the wrong place. Payback period of the cost of customer acquisition has two parts. There’s the cost of all the salespeople in their efforts (traveling expense and all that kind of stuff). Then you have the marketing part which is both the people and then the program spend, like the advertising spend, the event spend.
It’s important to break it down. When you’re looking at this metric and it’s 3.5 years, the first thing you want to do is understand that cost of customer acquisition metric into: what are my sales costs and what are my marketing costs? And you typically see it being something like 60/40 or 50/50 or 60/40, so anywhere between 40 to 60% for each of them. What you see in a lot of companies particularly if they hire a senior marketer too early is the program spend or the amount of money they’re spending on ads and events, or on brand can be very, very high, and not really driving a whole lot of ROI — return on your investment. And so that’s one common issue that you might see.
The second issue might be a sales issue which is, your sales cycles were just really long. So if you’ve got,, a multi-$100k TV product and you’re going after the 2000 largest companies in the world and you’re not closing very many customers in a year because you’re just getting started, you might have a payback period that looks like 3.5 years. And there’s probably actually nothing wrong with the business at that point. If you’re going after J.P. Morgan and Citi, and it’s taking 6 to 9 months, or 12 months, you’re probably spending a lot on sales and marketing because you have account executives who are spinning up those accounts and you have marketers. They’re supporting those sales people with the marketing materials and doing the branding. You just need to get enough scale where the number of customers that you’re closing increases. So let me make that a bit more concrete. Let’s say I’m like a 20 person company and I’ve got five people in sales and marketing and, in aggregate they cost me a million bucks. If I close one customer, that’s worth half a million. Just looking at revenue, my payback period is about two years which lets you know whenever I close the customer for $250k my payback period is four years. That’s more a function of the number of customers than it is the efficiency of my sales team, because if I close three, then my metrics would be way better.
Brett
So what you’re saying is that, particularly for the larger contracts, that payback period at an early stage of a company might be longer, and as the company gets up and running, and gets to scale you know as n starts increasing, you start to see more regularity in that payback period, that it could actually start to shrink. So in that sense, you know, you’re actually looking for some leading or lagging indicators of success.
So two questions: How do you look at companies with very large contracts? (maybe they’re selling to governments) How do you think about that in early-stage? Are there some early indicators that might improve? And then what’s happening with Covid-land or in a Covid world? Are any of these metrics moving around?
Tom
Yeah. So a third of the companies that we invest in at the series A in our B2B practice have no revenue at the time we invest. And so we might invest in a company before it has lots of customers because we have a thesis in that space, or we’ve spoken to a couple of potential buyers and they’re really excited about a company’s product. What that means is when we join the board of a company and it looks like they’re not necessarily managing the business to those metrics, right? So we’re not going to manage it to a certain payback period in the very early days. The more important goal is getting to a place where you have consistency and repeatability in the sales process. And then once you’ve established that, then you can start looking at the metrics and fine tuning.
But a lot of these customers, these businesses (ACV-positive companies) don’t get that many at-bats at the beginning. Just because you’ve got one sales person, they’re probably working 6 to 12 accounts and might close like 15% of them. So maybe two? In a year? And so you just can’t manage the business yet through metrics until they get to like, four or five or six different account executives and maybe $3 to $5 million in ARR. So there it sort of matters less.
Brett
In the beginning, you have to really drill down beneath the metrics and see what’s actually happening.
Tom
Yeah. And so at the board meeting, you ask: “Hey, go through the top three accounts.” What are they saying? What are we learning from those accounts? It’s way more qualitative than it is quantitative.
Brett
It sounds like if I could again summarize, for payback period: You’re basically looking at the spend, breaking it down into its components (whether it’s sales or marketing), looking to see if you’re over investing or not, getting the ROI on either of those categories, and then you always have to be cognizant the beginning that you may not have enough data to actually look at the top line numbers so you get down and see what’s happening on the ground. Then, I guess for our third metric of net dollar retention. What if that’s below your 105? It sounds like customers are actually buying less in the next year. Why might that be happening? Is there any reason, other than, people just don’t like it?
Tom
It’s probably a product problem. It suggests that the product-market fit isn’t there.
Brett
Perfect. Actually helping people know what they should be focused on. It’s actually a sales metric, but it indicates a problem elsewhere in the company. So that was a perfect overview. Now what’s a metric that you consider almost like a secret metric; a metric that you think is really powerful or telling but not yet well known?
Tom
One of the metrics we look at is the pipeline to quota ratio. What this tells us is: Will the company hit its plan next quarter? If you think about the customer funnel, it has got a bunch of different stages. Salesforce was the one that defined the funnel. But if you think about it like a sales qualified lead, sales qualified leads are basically people who are interested in talking to a sales person. And when you create a sales qualified lead you give it a value, right?
You might say: “Hey, Brett’s calling me to buy Redpoint software. I think Brett’s probably willing to spend like $15k a year.” And that’s what the SDRs and AEs do when they initially talk to the customer and put that in the Salesforce or your CRM. And if you take the sum of all those qualified leads, that’s your qualified pipe. And so that’s the first number in this division. And the second number is: I have two account executives, and they are inside account executives, and so their quotas are, let’s say, half a million dollars each, so their total quota for the year is a million. But I only want to look over the next quarter, so the total quota for Q3 is $250k and that’s the sum total of all the AE (account executive) quota. If I take the sales qualified lead total for next quarter and divide by the total quota 1/4 million dollars, I’m going to get a ratio. And that ratio varies by company. The very best companies will close something like 20 to 25% of their sales qualified leads. So one in four people who call and say I’m interested in buying Brett’s product will buy it. Most companies are typically in 10 to 15%, in their very early days. And okay, why does the number matter? Because if I’ve got a product that closes at 10%, goes from sales qualified lead to close at 10%. In order for an account executive to hit their number, they need, whatever 10 opportunities, Right? Let’s just say one opportunity number. But if I’ve got a 25% close rate, then I only need four opportunities.
Brett
Those people are capacity constrained. So is this a measure of the product or fundamentally a measure of how well the product sells once you get it in front of customers?
Tom
Yes. The conversion rate is: how efficient is the company of converting leads into deals? Into closed customers? The reason that we use this ratio is we’re trying to understand: how likely is it that the company hits the plan next quarter? Because if you have a company with a 10% close rate. If you’ve got a pipeline, that’s 10x the number that you’re looking to hit next you’re going to hit. You’re going to hit. And if you’re a company that you know closes 25% and you’ve got four times the pipe, then you’re going to hit the number.
The problem happens when you’ve got a 10% conversion rate, but only a 4x pipeline company. You’re going to miss the number, and that’s why we care. That’s why we’re spending time on that number because we want to figure out when founders come in, they pitch, you know, they give you the aggressive plan, which is exactly what they ought to be doing. And so we come up with our own plan. And we try to figure out if we’re going to hedge a little bit; what is it? What we think it’s actually going to look like. And so that’s why we pay attention to that number. If that number is off, then you should break it down; is this a sales problem? And is my conversion rate really low? Why is that? Is it sales execution of the product? Or is it a pipeline generation problem? Is marketing or sales not generating enough pipe in order to hit the number?
Brett
Yep, that makes a lot of sense. So basically, it sounds like you’re expecting things to be on plan and then if you have more pipe; if you have high conversion and more pipe than you might even come above plan; or if it looks like you don’t have enough, stuff; if you don’t have enough stuff coming, in relative to your ratio of closing it, then you kind of discount the entrepreneur’s plan.
Tom
Yeah, typically most companies were served in the 15 to 20% close rate. That implies that you want your ratio of pipeline to quota to be somewhere in the 4–5ish range at the beginning of the quarter in order to make sure you’re going to hit the quarter. The other thing to consider companies actually add to the pipeline within the quarter, particularly if their sales cycles are shorter than a quarter. They’re going to add some pipeline within the quarter, and that could be 15%. So if you’ve got the 4 to 5x pipe to quota ratio, you know, it’s very likely that you’re going to hit your number unless it’s dominated by one really big deal. And if you’ve got a 1 to 2x, I can almost guarantee you’re going to miss your quarter.
Brett
Yeah, unless your sales guys are literally hypnotists. Exactly. Unless you’ve got Alec Baldwin in there closing for you.
Tom
Yeah. “Always be closing.” — Glengarry. Yeah, for sure.
Brett
Perfect. So that’s a great way of actually looking at your numbers and saying “OK, Is this going to happen or not?” So conversely, What’s the metric that you think is just overrated?
Tom
I don’t know the history but I think it might have come out of a consulting company and it’s measured on a 1 to 10 scale, 10 being the best, 1 being the worst. And the way you calculate it is, you survey let’s say 10 people, and you ask them a question. Which is: “how disappointed would you be if this product were to go away?” Or, “how likely are you to recommend this product on a 1 to 10 scale?” And so you take the proponents, which are, I think, it’s like if you vote 8 and above, and then you subtract the detractors. So the number of people who I think voted three and below. It’s something like that, and you get to a number. Banks are basically like negative 10 which means you’ve got way more detractors than you have proponents. And, you have some companies like Apple, I think in the 50 or 60, and then you’ve got some really exceptional companies with 70 to 80, which means you have 90% of customers who are super enthusiastic and some tiny population of people who don’t like it. And for I want to say the last two years, it’s sort of been a metric that a lot of companies have led with to demonstrate product market fit.
We found a couple of different things. The people who run customer success, the NPS is actually the least important metric to them (the company itself doesn’t care). That’s because the correlation of NPS to net dollar retention isn’t great. What you really care about is, the customer loves my products so much that they’re gonna buy more of it, and they’re gonna tell five of their friends about it. That’s what you care about, because it means more business for you.
Brett
Money talks. And you want to know people like the product look to see if they buy more of it.
Tom
Exactly. And that’s how you measure your teams on: NPS. I was talking to one customer success leader and he was basically saying that NPS is super volatile because if a customer success manager is having a bad day and has a bad interaction with a customer, that customer’s NPS could fall from 7 to 3, just because, maybe they were short with them on email. But the net dollar retention to the customer might still be 140% because they’re not going to get rid of the product because they had one off interaction.
Brett
All right. So maybe the message to entrepreneurs is, you know, don’t read too much into that.
Tom
Yeah, exactly. NDR as a referral rate is a powerful metric. You know, Logo churn is a powerful metric. NPS is sort of like a feel good, almost vanity metric at this point.
Brett
Got it. Getting right to the inside pass. We’ve spent a lot of time talking about metrics. Now for entrepreneurs who are just getting started, people who are still figuring out their first customers: Do you have any best practices, or how they should think about their metrics, or setting up their metrics before they really have the data?
Tom
I’m going to start this with; I’m going to give you sort of a puzzle. So Brett,
You have just been made the general manager of a restaurant, and the restaurant is struggling because the customer service is really bad. And so what you want to do is to put in a series of metrics that will tell you when a customer has had, or even a single metric that will tell you when a customer has had a bad experience right? So what are your options?
Brett
Whether they come back and eat in my restaurant again (the net dollar retention analogy for a restaurant). And I would assume it’s whether, if I’ve sold a meal, people come back and eat again. Maybe even bring the friends to eat more.
Tom
Yeah, That’s a great one. Another one would be tip. If they leave a big tip. That’s a great one. Another one would be if they complement the waiter. Another one might be what kind of review they leave on Yelp. Anyway, there are all these metrics that you could come up with to try to figure out after the fact. How great was the experience for that particular diner in the restaurant? And when you’re getting started, those aren’t that useful, because they’re lagging indicators. You have to wait until the customers have gone through the entire process, or basically left the restaurant, in order for you to figure it out.
If you were really a forward-thinking restaurateur like I know you are, the metric that you would come up with is how full are the water glasses. And that is a metric that, you know, I call a proxy metric because it tells you the level of customer support and attention to detail. The waiter is paying attention to his or her tables. And if you go around the restaurant and if you start measuring the level of the water glasses, at everybody’s table you’re going to know which of the waiters and waitresses are paying lots of attention to their customers and which aren’t.
Brett
I like it. So it’s basically thinking about before higher funnel metrics. Basically leading indicators of the metrics that you want to move.
Tom
That’s it. It’s worth spending the time to figure those out. And the reason is if it takes you, say, 90 days to get from the beginning of a customer’s meal with your product to the end of the customer’s meal to the Yelp review your cycles are going to be 90 days, right? Whereas if you can figure it out halfway through the meal, your cycles are 45 minutes. If you can learn from that cycle in 45 minute increments on your product, whatever that may be, you’re going to be in a much better position because your learning rates are going to be that much higher if you can figure out what those proxy metrics are.
Facebook had this metric of ‘if you sign up to seven friends, you’re a Facebook user for life.’ That’s a proxy metric. So then all the growth marketing was really focused on getting you to get seven friends on Facebook.
Brett
So every business basically has its proxy metrics that will end up being reflected in the financial results, which are ultimately a lagging indicator. And so in terms of, looking for those proxy metrics, that Facebook example is canonical. How do you think they found that?
Tom
Well, you’ve got to do a bunch of analysis. You’ve got to figure out, experiment, and understand. You know, I think the first step is understanding what the customer life cycle journey is. What does the meal look like? There’s a drink, and then there’s an appetizer. And then there’s the entrée, the dessert and the bill. What is that for your business, and then trying to figure out what are the steps that are important at each of these stages, and what are the conversion rates across the stages and you’ve got to do the math. There’s no sort of shortcut to it. Each business is going to be slightly different. But if you can find it, it’s a huge strategic advantage.
Brett
It’s essentially the entrepreneur’s understanding of their proxy metrics is really a proxy for how quickly they’re learning.
Tom
Yeah, exactly. So it could be like if you’re selling a developer product, does this person come to three or more meetups that are tech talks, is this person an open-source contributor to the product? How many different people are using my open-source product? It could be any number of different things, but it’s worth spending that time because then you can focus your growth team or your demand generation team on just driving a truck through that opportunity.
Brett
Yes, and essentially it’s more definition and clarity around, again: the things higher in the funnel that will ultimately translate to dollars and cents. One of the things we’re trying to do is helping entrepreneurs from non-traditional backgrounds to use metrics to their advantage. If they don’t have the network, if they don’t have the warm intro: How could an entrepreneur speak to you about how they present their metrics?
Tom
I think the first is credibility in metrics. So if a venture investor asks you about a metric, if you have the answer, great. If you say I don’t know, that’s even better. But sort of waffling between an answer raises doubts about how much command you have over your business. And so I definitely advise that you either say: “I firmly know” or “I don’t know; but if it’s important to you, I could get back to you”. I wouldn’t be in the middle.
The second thing that’s really impressive is when a founder comes in and says: these are the three or four metrics that matter in my business and why. And this is how I’m driving the company towards maximizing those metrics because it demonstrates a really great management skill of internal alignment. For example, everybody knows that the three OKR (objectives and key results) for this business for this year are ARR, NDR and, payback period.
Whatever it is, it’s different for every company, depending on the stage. But, that’s really important.
The third way that entrepreneurs can really demonstrate understanding is, if you have a weak metric, like your payback period is a little higher than most. If I ask you a question about it and you say: It’s high for the following reasons: “You know it gets the 60 / 40 sales marketing split we’ve had. We just hired three new account executives, they’re all getting ramped and none of them are productive. So 75% of our workforce is at full draw. In other words, full salary. But they’re not producing, and that’s why.” That’s a fantastic answer.
Brett
So basically, having already anticipated your reaction to the numbers, diagnosed the problem and having already started a solution. And are there big red flags, in terms of metrics? I think that you already alluded to one. But any particularly egregious ways that you’ve seen people present metrics that turn you off?
Tom
Everyone has sort of slightly different definitions for these metrics. And so, I think if you don’t understand the definition, or if you don’t understand the metric deeply enough, that’s sort of a red flag because it demonstrates a lack of understanding of the business.
Brett
Awesome. I’ll ask one more question, which is: Can you think of a time, where you know, you passed because the metrics didn’t seem interesting, but the company ended up being a success? What happened there? And what did you learn about your own metrics?
Tom
Yeah, it happens all the time! At the early stage what you really should lean on instead of metrics are important, particularly for, like, smaller ACVs. But the bigger the ACV, the more you should lean on customer references. And if you hear absolutely rabid demand from customers? It doesn’t matter what the numbers really look like today because in the future things will change.
For example, two years ago I met a company and they’re ACVs were like, 1500 bucks. And we passed because the contract values were small and I didn’t really understand the path to increasing them very well. That’s not to say that they needed to be much larger. I mean, for the top five publicly traded SaaS companies by market cap are or by forward multiple are small ACV contracts. So it’s not that big of a deal, but I didn’t really understand the path to increasing those over time, let’s say to $5k? And so I passed. I was looking at the customer references the other day because the business is doing great and you know? Just missed it.
Brett
We’re approaching the end of our time with you. And, you know, we’re really grateful for you dropping so much knowledge on us. But I’ll ask you one more question to take it home. I was talking about startups the whole time, you’re very successful at the fund; and have a great large family; Five kids, which is amazing. I don’t know how you pull it off. What would you say are your top, two or three personal KPIs for living your life? And how did you come up with them? How do you track them? What are your goals and how’s it going?
Tom
For sure. I’ll talk about work OKRs. One is personal return: How am I doing as an investor? How are the firm and are limited partners and all the people downstream doing? That’s something I look at all the time. And I just want to make sure that it’s a good number. The second thing is more personal. Success for me in work is: looking back at the end of my life and having gone through lots of fun and adventurous journeys with people and having a really strong set of relationships. So that’s super important. It’s not something that you can necessarily track, but it’s sort of an approach to people that’s really, really important. The third thing I really care about is this idea that comes from the All Blacks, which is in the New Zealand rugby team. They have this value that’s part of their organization. And they call it “plant trees that you will not see.” And that means investing in the future. Paying it Forward. Yeah, and so that’s really important to me. Both inside the firm and outside.
Brett
So, trees planted! I hope to have a forest. I like it. Well if everyone would adopt that, we’d have a future look forward to. Tom. Thank you! Thank you so much for taking the time. It was really great to get your insight. And, is there any best way for folks to get in touch with you?
Tom
Yeah! So, if you go to tomtunguz.com, that’s the blog. And there’s actually an email link and you can email me directly.
Brett
Excellent. All right, Tom. Hey, thank you. Have a great rest of your day. And talk to you soon.
Tom
This was a pleasure, Brett. Thanks for having me on!
This interview is a part of our Stats for Startups series, brought to you by Charge Ventures and the Kauffmann Foundation. You can find the technical companion to this interview on the Stats for Startups website. If you are an investor with a quantitative take on venture, we’d love to hear from you: hello at Charge dot vc. Thanks for reading!
