Why we’re changing how we calculate metrics

Josh Pigford on January 13, 2016

We just rolled out a change to Buffer’s very public revenue dashboard that resulted in a $25,000 change in their reported MRR. This article provides some backstory in to how we originally calculated metrics and the transition to a greatly improved version of metric calculation we’ve been building over the past year.

Very few things in life are black & white, but we so desperately want them to be. Life’s just easier when the choices are obvious. And that certainly holds true with entrepreneurship.

Every day you’re faced with hundreds, if not thousands, of choices to make, most of which lack obvious answers. You make an educated guess, you move on, and you address any potential problems that arise from said guess.

I wanted to build a service that removed a lot of those decisions you had to make when it came to your business data. Constantly making decisions about how to calculate this, which formula made the most sense for that or figuring out how to get all the data in one place…it can be incredibly overwhelming.

So, in 2013, I started Baremetrics to address that very thing. The premise being that you shouldn’t need to make so many decisions. You should just be able to get the information you need to grow your business and carry on with actually growing your business.

If only it had been that easy…

Endless options

As a founder, you’ve likely spent an inordinate amount of time learning How to Business™. You may even have a degree saying you’re a Master at Businessing. But it turns out all that research and reading doesn’t replace just getting in there and doing it.

In fact, I’d argue that spending too much time learning the minutia actually hinders your ability to grow and build a company.

For example, there are endless articles on how to calculate X metric, why this formula is better than that formula and what the “right” way to calculate it is. After you’ve read all of those things you’re essentially back at square one with a burning heap of conflicting information. Take the Pacific Crest SaaS Company Survey: it outlines nearly 50 different ways that companies calculate retention rates. 50! That’s nearly 50 different formulas for a single metric.

For 99% of founders, it’s an epically terrible use of time to sift through all that information to learn the detailed intricacies of metric reporting and the pros/cons of each calculation formula. There are just a thousand more profitable uses of your time.

I’m on a tangent now, so let’s get back to my original goal with Baremetrics: How do we make decisions so founders don’t have to?

It’s Complicated

I knocked out the first version of Baremetrics in a month, based primarily on my own needs building a couple of SaaS products. Within a few weeks, as more and more businesses started using the service, I started realizing just how different businesses can be.

When you read “SaaS company” you likely think of the traditional set of 3-5 monthly plans with customers that regularly and reliably pay. In practice though, that’s just not how it works. It’s never that clean. For any company. Ever.

Business is messy and subscriptions are insanely complicated to address properly. Customers change their minds, they grow, they shrink, their payments fail, they demand refunds, they want coupons, etc. We currently cover over 30 different possible subscription “states” to address this. Who knew simple subscriptions could vary so much?!?!

All of these things (and thousands of other scenarios) need to be accounted for. But how do you account for them appropriately and usefully? That’s the real question. What’s most useful from a decision-making perspective?

Changes

Over the past 2+ years of serving thousands of businesses, we’ve seen a lot of use cases. And over those years we’ve learned an immense amount about what’s useful and what’s actually needed to run a business and make decisions.

When we initially launched, we based most of our metrics off the idea of a “charge”. If there was a charge and that charge was tied back to a subscription plan, then we’d consider it “Monthly Recurring Revenue”. That worked pretty great initially, and covered a lot of use cases.

But over time we found that method was a bit too volatile for a metric as important as MRR and it was hard to make the connection between “charges” and “subscription activity”. That’s what most founders are actually after because the subscription conveys what their customers were actually doing.

It was during this time that we had a bit of an epiphany: we are not an accounting tool—we’re a business decision making tool.

That’s a very important distinction because the expectations of those tools are entirely different. “Charged-based metrics”, as we called it, was built from the accounting perspective. But we decidedly did not want to be an accounting tool.

So we started a shift that’s been nearly a year in the making. We’ve been moving over to what we call “subscription-based metrics”…which is based on the states of subscriptions rather than the charges themselves.

This lets us build the history of a customer and their subscriptions from beginning to end, telling a more complete story. It lets you dig in to your numbers so when they change, we can show exactly why. And the answer to “why” is where real business insights are found.


Tweet this quote

So why do you care? Well, for most customers, it means their metrics will have a slight shift. Which when you’re talking about “data”, sounds like the worst possible thing on earth, because traditionally “data” is viewed as black & white. It is what it is.

But our goal isn’t simply to provide “data”…it’s to provide decision-making insights. It’s to surface trends, to see progress, to find the driving force behind revenue, to learn what is and isn’t working. And that’s what the shift to subscription-based metrics allow us to do.

To our customers

I know it can be disconcerting to see numbers change, but this shift allows us to not only give you more accurate metrics, but also more actionable insights.

At this point, about 70% of you are actually on the new system already and taking advantage of all the new features this change enables (revenue breakouts, cohorts, trendlines and more). The remaining 30% should be moved over in the coming 2-3 weeks.

As we’ve been rolling this out, we’re hearing time and time again that it’s much more in line with how businesses want to view and use their data. Deeper insights, better views into trends, more answers to the “why’s”.

In addition, this lays the foundation for much faster development, much more stable numbers, and many more useful features that we’re excited to get working on.

If you have any questions about this, we’re always an email away: hello@baremetrics.com.

Josh Pigford

Josh is most famous as the founder of Baremetrics. However, long before Baremetrics and until today, Josh has been a maker, builder, and entrepreneur. His career set off in 2003 building a pair of link directories, ReallyDumbStuff and ReallyFunArcade. Before he sold those for profits, he had already started his next set of projects. As a design major, he began consulting on web design projects. That company eventually morphed into Sabotage Media, which has been the shell company for many of his projects since. Some of his biggest projects before Baremetrics were TrackThePack, Deck Foundry, PopSurvey, and Temper. The pain points he experienced as PopSurvey and Temper took off were the reason he created Baremetrics. Currently, he's dedicated to Maybe, the OS for your personal finances.