The ABCs of Defining Product Metrics for your Team

You already know, collecting data about your product’s performance is essential. The right product metrics give you essential insights on your product’s health and potential issues, and let you know where and how you should improve.

That’s why great product managers are always hungry for information. And thanks to technology, we have vast amounts of data that we can use to inform our product decisions. It’s an awesome time to be alive and building great products.

But with so much data readily available, how do you know which product metrics are most important to you?

Building a successful product is not only about tracking the right metrics, but also understanding what influences those metrics.

But what are product metrics anyway?

Metrics, or KPIs, are among the key points in building a product roadmap. They are a quantifiable measure that allows businesses to define and track the success of a product.

Metrics are used by stakeholders, marketers, and the product management team to detect problems, set goals, and make informed decisions. They’re how you know what’s working, what’s not and where you can improve.

How you choose your product metrics depends on your business’ objective – attracting a new customer segment, improving popularity with users, getting ideas for new features, etc.

You need to pay attention to the KPIs that contribute to your goals. Why? Because, first, not all products are the same, and thus, not all the collected data is equally important.

Second, monitoring an ever-growing number of metrics out of fear of missing something might actually cloud your vision and lose track of what really matters. And you don’t want that.

Ok, where do I start?

If you’re only just beginning, these metrics will help you get the necessary information to drive success for your product:

1. Technical performance: no one wants to use a buggy app. So you need to know if your product has any issues, how many, and how long it takes to solve one because this has a direct impact on the quality of your product.
2. Business performance is not easy to determine if your product is free (in this case, you should look at product usage metrics). But if your product generates revenue, that’s the most obvious metric to look at, alongside your margins (how much you have left after you’ve covered your costs). If you need to spend $2 to make $1, that’s obviously not going to be great in the long run, and you should re-evaluate your strategy.
3. Lead management means knowing how many of your leads convert and why. The better you understand why some people convert while others don’t, the easier it will become to get new customers.
4. Engagement & product usage: find out how many weekly active users you have and how much time they spend using your product (average session time), and see how these metrics change over time. Ideally, they should be growing.
5. Retention management: retention goes hand in hand with engagement. If you fail to keep your users engaged, they’ll probably stop using your product. Things to look at are monthly/yearly churn and number of inactive users.
6. Marketing ROI: are your marketing efforts paying off? Do you see an improvement in your performance?
7. Customer ROI: you should aim for a customer acquisition cost as low as possible, and a lifetime value for each customer as high as possible. For example, if each user costs you $5 to get, but they generate $50, you’re on the right track.

By now, you should have an idea of which product metrics you should track. However, this information is only the beginning. As every product is different, so are the metrics you should track.

Choosing the right product metrics – the GAME framework

The GAME (Goals, Actions, Metrics, Evaluations) framework is a 4-step process that can define metrics for any feature or product. Here’s how it works:

Step #1: Define your goals

Setting goals ensures you maintain a purpose-oriented strategic mindset.

They also serve as tangible markers for revisiting and validating directional correctness throughout the metrics development process.

You know, there’s a famous Henry Ford quote going around in the startup industry: “If I had asked people what they wanted, they would have said faster horses”.

However, Sean Sheppard, one of How to Web Conference’s 2019 star speakers, strongly believes that, when you’re launching a new product, “you shouldn’t be thinking of your broader market. You need to be thinking about who’s your right customer right now”. So take this into consideration when defining your goals.

Another benefit of setting goals first is that it forces you to have a top-down approach, which is critical. A bottom-up approach heavily relies on intuition and can lead to analysis paralysis.

There are two types of goals you need to define: user goals and business goals. However, for many products, user and business goals align. And both can be revealed by asking the right questions.

Here’s how you can determine your user goals. Ask yourself:

• How are users benefiting from my product?
• How do they interact with my product?
• How are they feeling when using my product?
• How is my product integrating into my users’ life?

Revealing your business goals, on the other hand, can be done by asking yourself what your tactical or strategic business benefits are. Increase revenue? Lower costs? Become more competitive?

Also, you should define how your business would look like if your product is successful.

Step 2: List your users’ actions

But not all their actions, only the ones that are relevant according to the goals set.

Depending on the goals you’ve set at the previous step, you can choose from different types of questions:

1. Acquisition & activation questions:

• How will prospects learn about my product?
• How can a prospect turn into a user?
• What actions do users have to take to get value from my product?
• At which point is my product solving a real problem for my users?

2. Retention & engagement questions:

• Why do users come back to my product?
• What do users do when they are engaged with/interested in my product?
• What actions provide users with repeat value?
• How often should users take those actions? Daily? Weekly? Monthly?

3. Monetization & revenue questions:

• How do users pay for my product? With money? Or time?
• If my product is free, what would make them pay for it?
• Do users pay me directly, or does someone else pay me when they take action (payments from ads or affiliates)?
• Do they pay me for each action (transaction)? Or do they pay me periodically (subscription)?

Alright, now it’s time for the next step.

Step 3: Define your metrics

You should know this is the hardest step in the framework. You need to turn each desired user action into a measurable and trackable value.

Here are some significant decisions on how to count each action:

• Can the action be directly tracked? (For example, clicks can be measured directly, but views can’t)
• Can you group more actions for an overview, then separate them for later analysis?
• What makes more sense: measuring the overall magnitude of an action or making comparisons using ratios?
• Can you derive more knowledge from the measure intrinsically? Or do you have to rely on heuristics for the metric to be valuable?

Once you’re done defining your product metrics, go through your list again and remove any “vanity metrics”.

Step 4: Evaluate your metrics

Often, you don’t know right away whether a metric is working. The best way to ensure your product metrics are providing you the correct insights is to test and iterate.

You should evaluate the functional usefulness of your metrics. You can check for usefulness by monitoring the metric for any false-positives or false-negatives.

For example, if a metric drops, does it signal a real problem in the product? If an issue surfaces, are your metrics helping you flag this issue? If not, you should reconsider your product metrics.

Similarly, if the set of actions you’ve picked are not reflective of your product, user, or business goals, or they’re not telling the correct story about user behavior, you’ll want to re-evaluate your metrics.

Last but not least, you need to think about your goals. Do your product metrics tie in with business or user success? Are your users happier when your metrics are positive? If not, you might need to rethink your metrics as well.

Bottom line is, you need to be honest with yourself and remember you are looking for your metrics to measure the vital signs of your business, indicators of health and performance.

A practical example – The Instagram feed

Let’s apply the GAME framework step-by-step, so you get a better idea of how it works. First, we need to define our user goals and business goals.

A user goal could be finding out how my friends are doing. From a business perspective, if the user comes back only if he has a reason to (he has a new message, for example), we’ll want to improve engagement.

Next, we need to list the actions we want the user to take, actions that can be used to measure engagement: Login/Open app, Post, View, Scroll, Like, Commen, Share, Save, etc.

The third step would be defining our metrics, and this is where things get tricky. Most of the actions above can be measured directly, using clicks. But not views. For views, we can use a proxy, such as “count viewed if 60% of the feed item is in the device viewport”.

Analyzing the list above, we can see that some actions are more valuable than others. For example, posting is more important than sharing, so actions should be grouped accordingly.

We can also see that defining actions-per-day or actions-per-user is more relevant because each user can generate many actions.

As engagement is challenging to define and measure, it’s up to us to decide when to consider a user is engaged or not.

Putting all this together, our metric for the Instagram feed will look something like this: (Engaged users/Total number of users)*100.

Now, all we need to do is evaluate and validate our metric.

How others do it – examples we can all learn from

Depending on what you decide to prioritize on your strategy roadmap, there are multiple sets of metrics to choose from:

• Customer-oriented metrics: customer retention, NPS, churn rate, and conversions;
• Engagement & features metrics: product usage, feature usage, and number of key user actions per session;
• Business-oriented metrics: customer acquisition cost, monthly recurring revenue, and customer lifetime value.

We’ve said it before: every startup is different, so these specific metrics, even though essential, might not represent the key product metrics for you.

To give you an actionable example, here are the metrics that Appcues, a platform that helps startups grow faster, scale bigger, and retain more users is also looking at:

• Time to value (TTV): the amount of time it takes new users to realize the product’s value. You should aim to reduce the TTV as much as possible by optimizing the onboarding process around the actions that correlate to activation.
• Product-qualified leads (PQLs): leads who have already experienced the product’s value. To find out who your PQLs are, you need to identify your product’s activation event and the actions that indicate users are ready to move on to the next stage. You can do this through user interviews, session recordings, or A/B tests.
• Expansion revenue: also called expansion MRR (monthly recurring revenue), this metric measures the revenue generated from existing customers. It’s recommended that at least 30% of your revenue should be expansion revenue.
• Average revenue per user (ARPU): the amount of money, on average, that you can expect to make from an individual user. ARPU = MRR / number of customers.
• Virality: virality results when your product’s adoption increases exponentially. The virality formula is C(0) * k = number of customers at the end of the period, where k is the number of new users each existing user can successfully convert (k = number of invitations * rate at which invitations convert).
• Network effects: a product with a network effect becomes more valuable to users as more people adopt it. The more people do a certain action regularly, the better the product experience becomes for other users in the long run.

We hope by now you have a clearer idea of what the key product metrics of your business are.

Keep in mind, though, that you will also need to look at traditional marketing and sales metrics. But don’t improve your numbers for numbers’ sake.

Your product is about bringing value and customer satisfaction, so your metrics should reflect just that.

Want to learn more about product metrics? Join us at How to Web Conference 2019 on the 30th & 31st of October.

Get your early bird ticket here: https://www.howtoweb.co/buy-tickets/

Illustration credits: vecteezy.com

0