Meaningful Metrics for Product Analytics

Introduction

In the digital age, data is the lifeblood of any business. It can transform a company’s trajectory, inform strategic decisions, and predict customer behavior. But data alone isn’t enough. It’s the application of relevant metrics that can truly drive business growth. When created and measured appropriately, metrics can help illuminate the path to better customer experiences, optimized products, and business success. However, not all metrics are created equal. The key lies in selecting ones that are meaningful, actionable, and tied to your specific business objectives.

In this blog post, we dive into the importance of metrics in product analytics, how to set the right ones, and when to measure and evolve them.

Understanding Quality Metrics

Quality metrics provide actionable insights that are specific to your business. They’re quantifiable, easy to understand, and directly linked to your key performance indicators (KPIs).

For instance, an essential metric is Viewing Time in seconds if you’re a streaming media business like ViX. That heartbeat metric is directly tied to the business goals of driving more watch time and directly impacts revenue. Please check out this case study for a more detailed overview of how ViX teams use Kubit to support and enhance their daily work.

Setting Quality Metrics

Identifying the right metrics is vital for your product’s success. Here are some common categories of metrics to consider:

  • Acquisition: how users find and start using your product. Examples include marketing campaign efficacy, traffic source breakdowns, and customer acquisition cost (CAC).
  • Activation: how effectively your product engages users after they sign up. Examples include time to the first essential action and completion of the onboarding process.
  • Engagement: how active are your users? Examples include the Viewing Time per Session or Likes per Day. Typically, these metrics should have a time factor instead of just simple counts.
  • Conversion: how effectively your product drives users towards desired actions. Examples include free to premium signup, checkout, and referring a friend. Typically a Funnel or Path (Sankey diagram) can provide visibility of the users’ journey.
  • Impact: demonstrates the effect of your product on user behavior. Examples include the impact of new product releases on user behavior and engagement.
  • Retention: how well your product keeps users coming back. Examples include how often and at what rate users are returning and how that retention rate changes over time.
  • Churn: Churn metrics help you understand why users stop using your product. Examples include churn rate, reasons for churn, and average time before churn.

Choosing the right metrics depends on your business type, product, and specific goals. There’s no one-size-fits-all approach, but keeping these categories in mind will guide you toward meaningful metrics that reflect your product’s performance and user behavior.

Measuring and Evolving Metrics

Once you’ve identified the right metrics, the next step is to measure them regularly to understand the baselines and make informed decisions. The frequency of measurement depends on the specific metric and your business needs.

For instance, DAUs (Daily Active Users) might be measured daily, while churn rate or retention might be measured monthly or quarterly. Reviewing and updating your metrics periodically ensures they remain relevant as your product and market evolve.

Also, remember that metrics should be seen as tools for learning and improvement, not just reporting. If a metric is consistently underperforming, use it as an opportunity to investigate, learn, and iterate on your product.

Metrics with Kubit

Kubit stands out in the crowded data analytics space due to its unique ability to seamlessly handle a comprehensive spectrum of data types, including but not limited to online, offline, transactional, operational, and behavioral data. Our warehouse-native approach ensures that organizations have the ability to access, analyze, and assimilate ALL of their data with Zero-ETL. This sets a new standard for creating, measuring, and adjusting metrics, offering unparalleled flexibility and precision. Unlike other solutions that mandate predefined data models in their data silos or limit the scope you can view, Kubit’s platform empowers you to explore every facet of your data and gain deep, actionable insights. This differentiation unlocks improved data-driven decision-making and gives you a competitive edge in today’s data-centric business environment.

Conclusion

Meaningful metrics are the guiding compass in navigating the expansive realm of product analytics. They provide a clear direction, enable informed decisions, and drive business success. By understanding what good metrics look like, how to set them, and when to evolve them, product managers and data analysts can increase their positive impact on business outcomes.

Remember, numbers tell a story. Ensure your metrics tell a story that matters to your business. Happy analyzing!

Explore your data with Multi-Dimensional Data Tables

When analyzing data, analysts often need to compare various measures simultaneously and break them down by different properties and segments. Introducing Kubit’s latest analysis chart, Data Tables, which allows for multi-measure, multi-dimensional analysis in a single view.

Tools like Excel and Sheets have provided this type of data visualization and it works! While you may still want to see data in funnels, lines, bars and pie graphs; it can sometimes be best to see it laid out in a table view.

Our Customers are using Data Tables to understand things like:

  1. Cross Tab Analysis
    • How are User engagement metrics across different user segments and features?
  2. Custom Measures and KPI Analysis
    • Compare custom-defined measures or KPIs across different dimensions
  3. Segmented A/B Testing
    • Analyze user segments by control vs. variant groups
  4. Impact of Marketing Campaigns
    • Show click through rate, conversion rate by user segments and Campaigns all in one report

Getting Started with Data Tables:

  1. Navigate to Report → Data Table.
  2. As you can see from the snapshot below, the end user can easily begin adding new events, saved measures, breakdowns and segments.
  1. Highlighted below is an example of a user selecting 3 saved measures, building 2 measures on the fly and breaking it down by Country (United States, Canada, United Kingdom), Plan Type and Platform.
  1. When executed, the below table will be displayed. Users have the ability to sort, search, adjust columns widths, export to CSV, and view the SQL behind the chart.

Take it for a Ride

Now that you have a high level overview of Kubit’s Data Table, click through the guide below and get a feel for it yourself. If you’re interested in learning more, please reach out to our team.

Click the below GIF to walkthrough the demo.

Work around 5 common Data Quality issues with Kubit

Intro

We already know the perfect data model for product analytics but even with a perfect data model you can get tripped by other data issues on your way to obtaining insights. It often happens that a data issue is uncovered while working on a report in Kubit and it suddenly blocks the task at hand. Unfortunately, data issues typically take time to fix – in the best case scenario as early as the next sprint, often a month or two and in some rare cases the issue cannot be resolved at all. So while at Kubit we advocate for data modeling and data governance best practices, we have also developed a few features to help you work around 5 typical data issues in a self-service fashion while the root cause is being addressed:

  • Missing Data
  • Duplicate Data
  • Ambiguous Data
  • Inconsistent Data
  • Too Much Data

In this blog post we’ll explore how you can leverage these features to save the day whenever a data issue tries to stop you from getting your work done!

1. Incomplete Data

Very often we have some building blocks in our data but we don’t quite have the value we want to filter by. For example, we may have a timestamp property generated when a user installs our app, but for our report we want to measure the percentage of daily active users who installed our app within 7 days. Or we might want to filter by session duration but this information is not available when each event is triggered and must be computed afterwards. Or we may even want to extract the user’s device platform from a user-agent header.

Whenever this is the case you can reach out to the Kubit team to define what we call a “virtual property” which will be computed on the fly on top of your existing data. To continue our first example, let’s call the virtual property Install Days and it will be based on a timestamp column named  install_date. Now we can think of our virtual property in SQL like this:

datediff(day,install_date,event_date)

However, it looks like and is used as any other property within Kubit which makes our analysis very simple –  we get the amount of unique users who are active and filter by  Install Days <= 7, then divide that by the total number of daily active users like this:

2. Duplicate Data

Duplicate Data is always a pain to deal with and in the context of product insights we usually see it in the form of duplicate events. You can already leverage Kubit’s zero-ETL integration to do as much data scrubbing as you need. The results of your work will be immediately available in Kubit without any extra effort required. However, we often get asked to try and resolve some duplication on the fly – maybe the team who can fix the issue is overloaded, or there is some third-party responsible for the events generation – in both cases the process to resolve the issue will take any time between a lot and never.

Again, “virtual property” can come to the rescue, as we can generate a virtual property based on some criteria on only one of a set of duplicate events so we can distinguish it from the rest. Let’s consider the following example – imagine we have 5 purchase events for the same user, all for the same purchase but  at different timestamps:


user_id

event_name

purchase_id

event_date

purchase_amount

a7cb92df1c87c07fd

completed purchase

20041876

2023-10-23 15:23:11

$18.78

a7cb92df1c87c07fd

completed purchase

20041876

2023-10-23 17:05:47

$18.78

a7cb92df1c87c07fd

completed purchase

20041876

2023-10-24 10:32:03

$18.78

a7cb92df1c87c07fd

completed purchase

20041876

2023-10-25 22:11:59

$18.78

In this case, if we want to find the number of unique users who made a purchase, the duplication is not really a problem. But if we want to count the number of purchase events or aggregate the purchase_amounts, then our results will be way off.

How does Kubit fix this?

We can advise on the best solution, but one example is to assign a boolean property Deduped with a value true on the first of a sequence of duplicate events. Kubit can easily select the first duplicate event in a time range using some SQL along those lines:

CASE row_number = 1 ROW_NUMBER() OVER(PARTITION BY user_id, purchase_id ORDER BY event_date ASC NULLS LAST) AS row_number

And once we have the first event of the sequence we can assign the virtual property. So now we can aggregate without any adverse effects caused by the event duplication:

3. Ambiguous Data

What if 2 events in our dataset are easy to confuse with one another? Perhaps the naming is not ideal and people often make mistakes when they need to use them for a report. Let’s say we see 2 Signup events in Kubit – Sign Up and sign_up

But what is the difference between the two? Maybe one is a front-end event and the other is a back-end event, but the names don’t reflect that. There is a quick fix you can make yourself in Kubit to make the difference between the two events much clearer. You can simply go to Dictionary -> Event and  Rename from the Context menu for both events to give them more appropriate names, e.g. Sign Up (server) and Sign Up (client), and a nice description:

4. Inconsistent Data

This is true especially for multi-platform apps. As soon as you start instrumenting on multiple platforms inevitably from time to time there will be discrepancies between the implementations which can result in any of the following issues:

  • the same event comes back with a different name from one or more platforms
  • property name is different on one or more platforms compared to the others
  • property value mismatch between platforms

4.1 Same event, different name

Let’s say we have the same event coming back from different platforms in 3 different flavors – FavorFavourites and Favorites.

Such a situation can be extremely frustrating as you now have to go talk to multiple teams responsible for each instrumentation, align with their release schedules, prioritize the fix and wait for it to go live so you can go back and finish your work. This is one of the reasons why we developed Virtual Events as a way to group and filter raw level events to create new entities which have exactly the meaning we want them to.

It’s super easy to create a Virtual Event, anywhere in Kubit where you have an Event Group an a Filter you can save that combination like this:

And then the Virtual Event will simply appear in any event drop-down with the rest of the regular events, so you can use it for all types of reports:

4.2 Property name mismatch

Let’s say we have a streaming platform and for all the streaming events we have a property called Stream Type. However, a typo was made when implementing the instrumentation on Android and the property is called Stream type instead. Now, for the purposes of our reports we want to treat these two as one and the same, so that our metrics don’t get skewed.

To fix this in the data warehouse properly we would need to:

  1. correct the Android instrumentation in a new app version
  2. go back in our historical data and fix the property name retrospectively

And we still haven’t solved the issue completely – what about all the people who are using older app versions and don’t have the instrumentation fix? They will keep generating data using the inconsistent property name. Turns out a simple typo will be causing us trouble for a long time in our reporting.

There’s 2 solutions which Kubit can provide to help you work around such issues:

  1. You can create Named Filters using both property names and save them for reuse
  2. The Kubit team can easily make such configurations as to treat both properties as one and the same

Let’s explore option #1. In this case we have 2 properties which are actually the same – Plan Type and PlanType. So whenever we want to filter by one of them we actually need to filter by both in order to ensure our filter is applied correctly: 

To help prevent mistakes you can then save a Named Filter which others can re-use. Also helps you save time by not having to create the same filter over and over again:

Once the filter is saved you can use it anywhere in Kubit:

4.3 Property value mismatch

This typically wreaks havoc in our report when we group by that property. For instance, a simple typo in a property value will lead to our report containing 2 groups of the same thing instead of 1 as in the example below:

To overcome issues like this on the spot you can use Kubit’s Binning feature:

Using the Value Binning option you can define custom Groups – in this case we want to merge back Winback Campaign and Winback Campaing into one group and then we want to leave  Group Othersturned off so all the other groups remain as they were:

Congratulations, you’ve successfully removed the extra group from your report:

5. Too Much Data

What if our perfect data model contains more event types than we need for our analytical purposes? Or we have an event which is still noisy and in the process of being fixed, so we want to prevent people from relying on it in their reports?

The Dictionary feature in Kubit keeps track of all your terms – Events and Fields coming from the raw data and also concepts defined by you such as Measure and Cohort. Dictionary also allows you to easily disable an event, which means it will no longer be available for selection in any event drop-down in Kubit. All you have to do is go to Dictionary -> Event and then hit Disable from the context menu of the event you want to hide:

Note that in the case where you are dealing with a noisy event you can easily enable it once the underlying issues with the event generation have been resolved.

Outro

We just explored 5 ways to overcome common data quality issues in Kubit and get to your insights on time. The best part is that all of these solutions are dynamic and the mapping happens at runtime so you can take action immediately. You don’t ever need to invest in complex ETL jobs to update and backfill data. This also gives you the ability to test some hypotheses with real data with live product insights.

At Kubit, we want our customers to have the best possible experience, so please, do let us know what else you would like to get from Kubit to tackle data quality issues!