What Is a North Star Metric? Which Number Should a Startup Track?

In the spring of 1902, the new sewers of Hanoi filled with rats. The French colonial administration wanted the city to be a showcase of modernity in Asia; beneath it, the pipes had become a motorway for rats that might carry plague. The authorities first hired Vietnamese rat-catchers to go down into the sewers and paid them for every dead rat.

According to the tally sheets the historian Michael G. Vann found in the archives, the numbers grew every week. In June 1902, more than 20,000 rats were recorded killed in a single day. The count kept rising, but the rats did not go away.

So the administration opened the hunt to the whole city: a small bounty for every rat's tail handed in. Thousands of tails arrived at the town hall. A few months later, as Vann tells it, health officials began to notice live rats running around the city with no tails. The hunters were catching rats, cutting off their tails and releasing them back into the sewers; if they bred, that meant more tails. When officials found farms outside the city raising rats for their tails, the scheme was scrapped.

The French wanted fewer rats. What they were counting was tails.

The company that counts tails

Every startup counts tails somewhere. Sign-ups, downloads, page views, followers. These numbers grow, look good in a deck and lift the team's spirits. But they do not tell you whether the business is actually getting better. The lean startup approach calls them vanity metrics: numbers that paint the rosiest picture possible without measuring what moves the business forward.

Failory's guide to measuring product-market fit starts from the same place: the number of registered users proves nothing on its own; what you need to watch is active users. So the question is this: in your business, what is the rat itself?

What is a north star metric?

A north star metric is the single number that best captures the value a customer gets from your product. Everyone on the team makes decisions with that number in mind; every other measure serves it.

Find the number that stands for value

Failory describes a two-step method from Y Combinator partner Gustaf Alströmer. First, pick the action that best shows the value users get. Then track whether that action repeats over time. For Airbnb, that action is not signing up but staying somewhere. Sign-ups are the tail; stays are the rat itself.

Set the frequency to fit your business

People do not go on holiday every week. So in Alströmer's example, what Airbnb should watch is stays per user per year. A messaging app is measured daily, an accounting tool monthly. If the line stays flat over the years, people keep using the product; if it falls every year, they are leaving.

Four measures that check the compass

A single number gives you direction, but it can also mislead. Failory argues that product-market fit shows up in several measures at once.

The share who would be very disappointed

Ask your users: "How would you feel if you could no longer use this product?" If more than 40% answer "very disappointed", the common benchmark says you have found fit. Superhuman's founder Rahul Vohra, writing in First Round Review, says he measured this at 22% in the summer of 2017. By focusing the product on the people who said "very disappointed", the score rose to 33%, and within three quarters to 58%.

The recommendation score

The Net Promoter Score rests on the single question that Bain consultant Fred Reichheld introduced in Harvard Business Review in 2003: "How likely are you to recommend this product to a friend?" You take the share who answer 9 or 10 and subtract the share who answer between 0 and 6. According to Failory, anything between 30 and 70 is a strong result. Even so, surveys are mostly filled in by happy users; do not rely on this alone.

The curve of those who stay

Plot the share of active users over time. If the curve drops for a while and then flattens, the product has a lasting audience. If it keeps falling, it does not. The details of measuring this are in the article on customer churn.

What a customer is worth and what they cost

Divide the total value a customer brings you by the sales and marketing cost of winning them. According to Failory, a ratio of 3 to 1 or higher means healthy growth. Below that, every new customer may be making you a little poorer.

The dog's tail

In English, something that guides you is a "cynosure". The word comes from the Greek Kynosoura: "the dog's tail". It was the ancient Greeks' name for the constellation of the Little Bear.

In the Phaenomena, written in the 3rd century BC, the poet Aratus compares two constellations. Greek sailors find their way by the Great Bear: it is bright, it appears early in the evening, it is hard to miss. The Phoenicians trust Kynosoura instead. It is small and faint, but because it turns closer to the pole, it shows sailors a truer course. By it, Aratus says, the men of Sidon steer the straightest path.

A good metric is rarely the brightest one. In 1975 the economist Charles Goodhart said something similar about monetary policy: any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes. Later writers put it more briefly: when a measure becomes a target, it ceases to be a good measure. The tails of Hanoi had proved it a lifetime earlier.

A final word

In his 1963 book Informal Sociology, the sociologist William Bruce Cameron wrote a sentence that is often wrongly credited to Einstein:

"Not everything that can be counted counts, and not everything that counts can be counted."

The number in the biggest type on your dashboard today: is it the rat, or the tail?

To tell whether the number you measure has real demand behind it, What Is Product-Market Fit? is worth a read.