How to track product engagement: the most important SaaS metric you probably aren’t tracking.
Product Engagement: SaaS Lifeblood
As you’ve heard us say many times before, product engagement is the lifeblood of any SaaS business. The entire SaaS business model is dependent on people consistently using and getting value from your product over time. Your success depends on product engagement.
This is why creating a method for scoring product engagement is essential for every SaaS business. Without a proper way to score the engagement of your users and accounts, it’s pretty much impossible to know what the hell is going on with your business.
- As a Founder/CEO, how many times have you asked, “How are our accounts doing?” or “How’s engagement this month?”, only to be met with blank stares from your team?
- As VP of Product, how many times have you wondered if your recent releases were helping to drive overall engagement, only to realize that you don’t actually have a good way of determining that?
- As a Customer Success Manager, how many times have you wished for a simple way to prioritize your outreach—an easy way to determine which accounts are thriving and which are struggling—only to give up and waste the day sending a bunch of random “just checking in” emails?
- As an AE, how many times have you wished you could easily see which trial accounts are ready for a call and which ones you shouldn’t bother with…only to give up and refresh your Facebook feed a few more times?
Thankfully, all of these issues can be solved with a good product engagement scoring system. By scoring engagement, you can create essential visibility for every part of the organization that relies on it.
This is why Sherlock was created. This post will explain the process of creating a solid engagement score, plus share how this is done in Sherlock.
Four Steps for Creating an Engagement Score for Your SaaS Product
STEP 1: Define engagement for your product
The first step in creating a product engagement score is a strategic one. In this step, you need to think about what engagement specifically means for your product. Why? Because your product is unique.
Active vs. Engaged Users
A quick note about active vs engaged users: they are not the same. Lincoln Murphy, one of my favorite SaaS thought-leaders said it best:
“Very often, I see people using the terms ‘active’ and ‘engaged’ interchangeably when talking about user engagement. I think this is a mistake. These terms are related…but do not mean the same thing. You want “engaged” users…not just active users. A common definition for engagement for many SaaS apps is: number of logins. Seriously. This is a lot more common than I wish it were (I see it ALL THE TIME), but this is how a lot of SaaS providers measure ‘engagement.’ Logging in is a proxy for an active user, but it is certainly not engagement. Someone might login to your product to find the cancel button. Engaged users are those who are logging in and using the features that are driving value for him/her.”
We completely agree with this perspective. We don’t believe in some universal definition of engagement that applies to all products. Each product is unique. Each product has its own nuances which define successful engagement. In short, it’s important to define specifically what it means for someone to be engaged with your product.
For example, a B2B productivity tool could define engagement as a certain number of projects created, tasks completed, team members added, comments left, files uploaded, projects completed, etc.
A social networking application could define engagement as connections made, content posted, posts liked, comments made, and so on.
The point is, your product IS unique. Don’t run away from that fact. Embrace it and create an engagement model based on the unique activities that are important to your product. Start by creating a list of activities that a user can engage in while using your product. Even if you already have a good idea of what your most important activities are, I would suggest a quick survey of your team. Ask everyone a very simple question:
“What is an engaged user of [our product]?”
They should answer with something like, “An engaged user of [our product] does X, Y, Z.”
You will likely receive some very interesting answers. Undoubtedly, some will be very qualitative while others will be more quantitative. None will be “right” (there is no “right”), but these responses will give you a great starting point for defining engagement and maybe help you think about things from a different perspective.
Ultimately, your list of engagement activities could look something like this:
- Logged in
- Added photo
- Shared photo
- Invited friend
- Commented on photo
- Edited photo
- Posted to Facebook
- Posted to Twitter
- Opened email
- Clicked on email
Once you’ve finalized this list (and don’t worry, it can change—you will be iterating and refining this list over time and as your product evolves) you can move on to the next step: tracking product engagement.
STEP 2: Start tracking product engagement activities
I’m assuming that most SaaS teams reading this are already tracking their important product events. But if you’re not, what are you waiting for? Get yourself a Segment account and get it done.
Seriously. No excuses…make it happen.
I mean it. Why are you still reading this? Go… get it done. Then move to Step 3.
STEP 3: Weigh each ‘engagement’ event
Now that you are tracking your important engagement events, the next step is to weigh each event based on its impact or its importance to overall engagement with your product. This is an essential step due to one simple fact:
Not all activity is created equal.
Think about it: taking initiative to invite a new user to your product is a more engaging activity than simply logging in. Choosing to write a long post on a social media site is more engaging than simply liking a post in passing. Creating a project on a task management application is more engaging than marking a single task as completed. And so on.
Because each of these engagement activities carries a different degree of value, you need to weigh these activities accordingly. Create a table that looks like the one below. List your engagement events in the left column, then weigh each event on the right, like so:
You can choose to have a score range of 1–10, 1–100, 1–1000…whatever you prefer. The point is, you should give each event a point value that reflects its value to overall engagement. As a general rule, common, higher-frequency events should have lower point values. Less common, lower-frequency events—those used by power users—should have higher values.
Then, for each one of your users, you should add a column for the number of times they triggered each event over a period of time (for example, the last 7 days):
Next, simply multiply the event weight by the number of events. Your result should look something like this:
The total of all your individual event values will give you a total engagement score for an individual user.
Finally, just run this for each one of your users and you will have the basis for a quantified user engagement score.
But don’t stop there. Next step is to give it all some meaning.
STEP 4: Give it context by normalizing raw scores
If you are going to go through the exercise of creating a product engagement score, it’s essential that it is something that can be used across your organization. And in order for it to be easily operationalized by your various teams, it needs to be easily understood. This means all your score needs to be in a format that is easily understood and consumed by anyone. Let’s face it: simply telling your marketing team that a user has a score of 458 isn’t very helpful. How are they supposed to know if 458 is a good score or a bad score? Is that high or low? However, telling them that a user has a score of 91 on a scale between 1-100 is something they can easily understand.
To get here, you need to apply a normalization formula to your raw engagement scores. In Sherlock, in order to normalize scores, we use a process called Winsorizing (we wins at the 90th percentile threshold) and then apply an exponential function to ensure that differences in the raw scores are more effectively represented.
STEP 5 (BONUS): Apply the scores to make them actionable
Once you have a normalized set of user scores, you have done the hard work. All that is left is using these scores in ways that have direct business application. Here are a few ways to make this scoring model applicable and actionable:
1. RANK YOUR USERS
When every one of your users has an engagement score, that gives you the opportunity to do something amazing: actually rank your users based on their engagement. And this opens up so many opportunities. For example, you can:
- Discover your power users and find out what makes them great.
- Prioritize customer success efforts to drive greater support, identify problem users and growth opportunities.
- Drive more personalized marketing programs.
- Identify great user research targets.
This kind of user ranking can help you understand your users in the context of their actual engagement with your product — incredibly powerful stuff!
2. SCORE AND RANK YOUR ACCOUNTS
With all of your users scored for engagement, you can then aggregate those scores at the account level. In Sherlock, we simply take the average of the active users on an account to calculate an account engagement score. As you can imagine, being able to both track activation metrics and rank your accounts will enable you to:
- Understand the overall health of your business.
- Prioritize sales efforts by knowing which trial/freemium accounts are more likely to convert.
- Prioritize CS efforts by focusing on accounts that are ripe for expansion and those needing some extra support.
- Identify which features are most important to your best accounts.
3. CALCULATE OVERALL SCORE FOR YOUR PRODUCT
By calculating a score for each one of your users, you can also aggregate those scores to create an engagement score for your product as a whole.
By tracking this average score over time, you can determine whether or not the work you are doing on your product is actually driving engagement. This data can also be used at a higher level, helping to support board-level decisions about when to (or whether to) make further investments in the business.
4. COMPARE POPULATIONS OR COHORTS
A user engagement metric becomes tremendously helpful when comparing different populations of your of users. You can compare the engagement of new users vs. older users, users on a free plan vs. those on a paid plan, users with different levels of access, and so on. The opportunities to gain deeper product insights by comparing engagement across various segments are endless.
5. CORRELATE WITH OTHER BUSINESS METRICS
Ultimately, a good user engagement score isn’t just a valuable product metric—rather, it’s an essential business metric for any software business. Comparing the levels of user engagement to other business metrics—sales, retention, growth, LTV, etc.—is a great way to ultimately predict and forecast business progress based on engagement levels.
And if you don’t think user engagement is tied to the value of your business, think again. In 2016, it cost Twitter over $1B in market cap.
Peter Drucker is famous for saying, “You can’t improve what you don’t measure.”
This is certainly true when it comes to user engagement. If product engagement is an absolute key to success for your software business (and it likely is), then you simply must have a way to measure it. You aren’t going to be able to improve unless you do.
Of course, building this system yourself is NOT an easy task! We built Sherlock to not only help teams easily create these engagement models, but to help them operationalize engagement as well.
Sign up for a free trial today and discover the magic of engagement scoring for yourself!