n my last few articles, I explored the basics of instrumentation and the challenges we encountered when implementing it. I also shared some thoughts on what we'd do differently next time.

Now, let's focus on the first step: designing Key Performance Indicators (KPIs).

I call it "designing" because it's a bit of an art form, but don't worry—it's not as complicated as it sounds. As a product analyst, I've worked with many Product Managers (PMs) and seen almost everything regarding KPIs.

Sometimes, you get these super-efficient, data-savvy PMs who show up with a list of metrics. Great, right? Sometimes, that list has 10 or 20 metrics, and we must figure out how to whittle it down to 3 or 4 that matter.

Other times, we're caught off guard when a product suddenly goes live, and we're left scrambling to dig through random logs, trying to piece together how to track the feature's performance.

Let me tell you, there's nothing quite like when a team realizes, after launch, that they need to figure out how to measure whether their shiny new feature is succeeding.

This is why it's so important to discuss KPI design. It's not just about picking numbers; it's about setting ourselves up to understand and demonstrate the value of what we're building.

The Real Deal on Why Good KPIs Matter

When we rush KPI design (or worse, treat it as an afterthought), we're setting ourselves up for a world of chaos:

• Overtracking: We track everything under the sun because we're still determining what matters. This leads to overwhelming information that's difficult to sift through and analyze effectively.
• Undertracking: On the flip side, sometimes we don't track enough. We launch with minimal instrumentation and realize too late that we're missing crucial insights. It's like trying to navigate with half a map.
• Strained relationships: Nothing like constantly changing tracking requirements strains the analyst-engineer relationship. "Hey, remember that event we said was crucial? Yeah, we don't need it anymore. But can we add these ten new ones?"
• Analysis Paralysis: It is incredibly challenging to make sense of data when we need an apparent reason for collecting it. Without a solid foundation for why specific metrics were chosen, drawing meaningful conclusions or creating insightful reports becomes nearly impossible.

But it's not all doom and gloom. When done right, solid KPI design can be a game-changer. It provides clarity, guides decision-making, and helps everyone from the C-suite to the newest intern understand what success looks like.

The KPI Ownership Debate: Why Analysts Often Take the Wheel

Here's a truth bomb: while product managers often think KPI design is their territory, I've taken the lead more times than I can count. Why? It's more of a natural evolution based on a few key factors:

1. Data Fluency: We analysts live and breathe data structures and metrics. We know the difference between good and vanity metrics and can spot a misleading correlation from a mile away.
2. Feasibility Factor: There's often a gap between what teams want to track and what's possible. We're like the reality check – yes, it would be great to know the exact moment a user has an "aha!" experience, but unless we're mind readers, we might need to find a proxy metric.
3. Cross-Functional Liason: We're in a unique position in the product world. We talk to engineers about data schemas, designers about user flows, and PMs about business impact. This bird' s-eye view is crucial for designing KPIs that make sense across the board.

But here's the kicker—it's not about analysts vs. product managers. The real magic happens when we collaborate. PMs bring a strategic vision and deep user understanding. We bring analytical firepower and data expertise. When these forces combine, you get meaningful and measurable KPIs.

Designing KPIs for Oura's Daytime Stress Feature: A Real-World Example

So, you're a new PM at Oura, and you've been handed the reins of the new daytime stress feature. Exciting stuff! But before you dive into the nitty-gritty of development, let's talk about how you will measure success. After all, you don't want to be that PM who launches a feature and then has no idea if it's working, right? Let's walk through this process step by step.

Step 1: Mapping Your Feature in the Product Universe

Understanding where your feature fits in the grand scheme of things is crucial. It's not just about knowing your place; it's about seeing how your work contributes to the bigger picture. Let's walk through how to do this using examples from Oura and Uber.

How to Map Your Feature:

1. Identify the Company's Product Lines: These are major offerings, often aligned with different market segments or user needs.
2. Break Down Product Areas: Identify the main functional areas or user goals the products serve within each product line.
3. Locate Your Feature: Determine which product area your feature belongs to and how it contributes to its goals.
4. Draw the Connections: Create a hierarchical map from the company level to your specific feature.

Let's see this in action with two different companies:

Example 1: Oura's Daytime Stress Feature

Oura Product Lines

• Hardware
  • Oura Ring (primary product)
  • Software
    • Oura App
    • Oura Cloud
  • Enterprise Solutions
    • Oura for Business

Oura Product Areas

• Sleep Tracking
  • Sleep Score
  • Sleep Stages Analysis
  • Activity Monitoring
    • Activity Score
    • Physical Activity Tracking
  • Readiness Assessment
    • Readiness Score
    • Recovery Tracking
  • Stress Management
    • Daytime Stress
    • Resilience
  • Women's Health
    • Menstrual Cycle Tracking
  • Heart Health
    • Heart Rate Monitoring
    • Heart Rate Variability (HRV) Analysis

Now, let's map the new daytime stress feature:

• Oura (Company)
  • Software (Product Line)
    • Oura App (Product)
      • Stress Management (Product Area)
        • Daytime Stress (Your Feature)

Why This Mapping Matters

Understanding this structure is valuable for several reasons:

1. Strategic Alignment: You can see how your feature contributes to Oura's stress management capabilities, one of their key product areas. This helps you align your feature's goals with broader company objectives.
2. Cross-Functional Collaboration and User Experience Considerations: You might need to work closely with teams responsible for related areas like Readiness Assessment or Heart Health, as stress levels often correlate with these metrics.
3. Future Integrations: With Oura's focus on third-party integrations, you might consider how your stress feature could integrate with partners like Headspace for stress-reduction content.

Now, let's take another example to solidify this.

Example 2: Uber's UberX Share Feature

Now let's look at Uber's structure:

Product Lines:

1. Ride-Hailing
2. Food Delivery (Uber Eats)
3. Freight

Product Areas (for Ride-Hailing):

1. Rider Experience
2. Driver Experience
3. Matching Algorithm
4. Pricing
5. Safety

Mapping the UberX Share feature:

• Uber (Company)
  • Ride-Hailing (Product Line)
    • Rider Experience (Product Area)
      • Ride Options (Sub-Area)
        • UberX Share (Feature)

By going through this mapping process, you set yourself up for success in designing meaningful KPIs and ensuring your feature aligns with and contributes to your company's broader goals.

Step 2: Figuring Out What Makes Your Boss Tick

After mapping your feature's place in the product ecosystem, the next crucial step is understanding and aligning with your supervisor's key metrics. This isn't about playing office politics; it's about ensuring your work contributes meaningfully to broader organizational goals.

Why Care About Your Supervisor's Metrics?

1. Strategic Alignment: Your supervisor's metrics often reflect higher-level company objectives. By aligning your KPIs with these, you ensure your feature contributes to what the company values most.
2. Resource Allocation: When you demonstrate how your feature impacts metrics your supervisor cares about, you're more likely to secure resources and support for your initiatives.
3. Avoiding Silos: Understanding broader metrics helps you see beyond your immediate feature and consider how it interacts with other product areas.

Example: Oura's Stress Management Supervisor Metrics

For the head of Stress Management at Oura, key metrics might include:

1. User Retention Rate: Percentage of users who continue using stress management features over time.
2. Daily Active Users (DAU): Number of unique users engaging with stress-related features daily.
3. User-Reported Stress Reduction: Average change in self-reported stress levels for users with stress management features.
4. Feature Adoption Rate: Percentage of Oura users who have activated and use stress management features.
5. Correlation with Overall Health Scores: How stress management feature usage relates to overall Readiness and Sleep scores improvements.

Remember, the goal isn't to replace your feature-specific KPIs but to complement them with metrics that show a broader impact. This balanced approach ensures you're driving feature and overall product success.

Step 3: Defining Your Target Audience, User Value, and Business Impact

After mapping and aligning your feature with your supervisor's metrics, the next crucial step is clearly defining your target audience, measuring user value, and determining how your feature impacts the business. This step is vital for several reasons:

Why Is This Step Important?

1. Focused Development: By clearly defining your target audience, you ensure you build features that address real user needs, not just assumptions.
2. Measurable Success: Understanding user value helps you create metrics that matter to your users, not just vanity metrics that look good in reports.
3. Strategic Alignment: Defining business impact ensures your feature contributes to overall company goals, helping you justify resources and support.
4. Avoid Misalignment: Without this step, you risk creating features that users don't need or don't contribute to business objectives.

Now, let's break down how to do this effectively:

Defining Your Target Audience

Your target audience is not just "all users." It's a specific subset that your feature is designed to serve. Consider these factors:

1. Product use case (e.g., business vs. consumer)
2. User demographics (age, occupation, etc.)
3. Level of product engagement (power users vs. casual users)
4. Level of product experience (new vs. returning users)
5. Customer type (free vs. premium, creators vs. consumers)

Oura Example:

For the daytime stress feature, the target audience might be:

• Health-conscious professionals aged 25-45
• Both new and existing Oura users
• Primarily premium users, but also available to free tier
• Users who check their Oura app at least 3 times a week

Defining Your Target Size

Consider your target audience size in two ways:

1. Percentage of your current active user base
2. Potential expansion into new market segments

For Oura, this might be:

1. 60% of current Oura users (those most engaged with health tracking)
2. Potential to attract new users specifically interested in stress management

Segment Your Audience

Identify relevant segments within your target audience. For Oura, this could include:

• Geographic: Urban vs. rural users
• Occupation: High-stress vs. low-stress professions
• Usage patterns: Daily checkers vs. weekly reviewers

Measuring User Value

Understanding and measuring user value is at the heart of successful product development. But what exactly is user value, and how do we approach it?

What is User Value?

User value is a user's benefit or satisfaction with your product or feature. It's why they choose to use your product over alternatives or not use anything. User value is about solving real problems, fulfilling needs, or providing meaningful experiences for your users.

1. Value Metrics: Develop metrics that measure how much you solve the user problem. Often, it’s challenging to have a direct value metric so that we can choose proxy metrics. These could include:
   • Engagement Metrics: How often and how deeply are users interacting with your feature? Example: Daily active users of the stress feature, average time spent on stress insights.
   • Outcome Metrics: Are users achieving their desired results? Example: reduced reported stress levels over time and improved sleep quality correlated with stress management.
   • Satisfaction Metrics: How do users feel about your feature? Example: User ratings of the stress feature, Net Promoter Score for stress management capabilities.
   • Comparative Metrics: How does your feature perform against alternatives? Example: Retention rate of users who engage with stress features vs. those who don't.

Understanding Business Value

While user value is critical, it's equally important to understand and measure the business value of your feature. Business value ensures your product satisfies users and contributes to the company's goals and sustainability.

What is Business Value?

Business value refers to the tangible and intangible benefits a feature or product brings to the company. It's about how your work contributes to the organization's objectives, whether revenue growth, market expansion, competitive differentiation, or operational efficiency.

How to Measure Business Value

To effectively measure business value, consider these aspects:

1. Revenue Metrics:
   • Direct revenue generated by the feature
   • Impact on average revenue per user (ARPU)
   • Influence on conversion rates (e.g., from free to premium)
2. User Growth and Retention:
   • New user acquisition attributed to the feature
   • Impact on user retention rates
   • Effect on customer lifetime value (CLV)
3. Operational Efficiency:
   • Cost savings from automated processes
   • Reduction in customer support tickets
   • Improvements in development or deployment efficiency
4. Market Position:
   • Share of voice in the market
   • Competitive win rates
   • Brand sentiment improvements

Balancing User and Business Value

Finding the sweet spot where user and business value align is the key to sustainable product development. For Oura's stress feature, this might mean focusing on metrics that show both user stress reduction (user value) and increased app engagement or subscription renewals (business value).

When defining how your feature creates business value, ask yourself: "How does solving the customer problem create value for the business?"

For Oura, this might be articulated as: "By helping users effectively manage their daily stress, we increase engagement with the Oura ring and app, improve overall user satisfaction, and drive retention. This feature also differentiates Oura in the competitive wearables market, potentially attracting new users specifically interested in stress management. We expect this to positively impact our daily active users, premium subscription conversions, and long-term customer retention rates."

Remember, the most successful features are those that create a virtuous cycle: they provide significant user value, which in turn drives business value, allowing the company to invest more in creating user value, and so on.

By considering user and business value in your KPI design, you ensure that your feature contributes to this positive cycle and drives sustainable growth for your product and company.

Step 4: Picking Your Metrics (Without Losing Your Mind)

Now that we've mapped our feature in the product ecosystem, aligned with our supervisor's goals, and defined our target audience, user value, and business impact, it's time to bring it all together by quantitatively designing our KPIs. This step is crucial because it allows us to directly link our work to the outcomes our supervisors care about and measure our impact effectively. But before we dive in, a couple of warnings:

1. Don't fall into the "one metric to rule them all" trap. While it's tempting to boil everything down to a single number, life (and product management) is rarely that simple.
2. Beware of vanity metrics. Sure, "number of times users check their stress" might sound impressive in a meeting, but does it really mean your feature is helping people?

Instead, aim for a balanced scorecard. Here's what you might include:

Core Metrics to Consider

Let's break down the key metrics you should consider for your scorecard:

Adoption/Acquisition Metric

This metric answers the question, "How well are we reaching our target audience?"

• Definition: The number or percentage of target users who start using your feature.
• Calculation: (Number of users who have used the feature) / (Total number of users in the target audience)
• Example for Oura's stress feature: Percentage of Oura users who have activated and used the daytime stress tracking at least once.

Why it matters: Adoption indicates how effectively you're surfacing your feature and providing initial value to users.

Retention Metric

This metric helps answer the question, "How well are we solving the relevant user problem over time?"

To define a retention metric, consider three components:

a) Frequency: How often should users engage with the feature?

b) Core action: What specific action indicates feature use?

c) Target audience: Which users should we include?

• Definition: The percentage of adopted users who continue using your feature over a specified time period.
• Calculation: (Number of users who used the feature in the current period) / (Number of users who used the feature in the previous period)
• Example for Oura: The percentage of users who check their stress levels at least three times a week four weeks after their first use.

Why it matters: Retention shows the ongoing value users are getting from your feature.

User Value Metric

This metric aims to answer the question, "What's the best proxy for measuring the value our feature provides to the end user?"

Often, engagement metrics serve as good proxies for user value.

Consider these types:

a) Frequency: How often users engage with your feature

b) Intensity: How deeply users engage (time spent, actions taken)

c) Breadth: The range of feature functionalities used

• Example for Oura: Average number of stress-reduction recommendations followed per week, or improvement in reported stress levels over time.

Why it matters: This metric helps you understand if your feature is truly solving the user's problem and providing value.

Business Impact Metric

This metric answers, "How does our feature contribute to the key outcomes our supervisor cares about?"

• Definition: The metric(s) from your supervisor's scorecard that your feature most directly impacts.
• Example for Oura: If your supervisor cares about overall app engagement, your metric might be the increase in daily active users attributable to the stress feature.

Why it matters: It directly links your work to higher-level business objectives, demonstrating your feature's strategic importance.

For each of these, pick one or two concrete metrics. For example:

• Adoption: Percentage of Oura users who've activated the stress feature
• Stickiness: Percentage of users still using it after a month
• Value: Average reported stress level over time for active users
• Business Impact: Retention rate for users who regularly use the stress feature vs those who don't

Putting It All Together: The Oura Example

Let's walk through the entire process for Oura's daytime stress feature, combining both qualitative and quantitative aspects:

Product Area Mapping

• Company: Oura
• Product Line: Software (Oura App)
• Product Area: Stress Management
• Feature: Daytime Stress Tracking

Supervisor's Key Metrics

• Overall user retention rate
• Daily active users
• User-reported health improvements
• Premium subscription conversion rate

Target Audience

• Health-conscious professionals aged 25-45
• Both new and existing Oura users
• Primarily premium users, but also available to free tier
• Users who check their Oura app at least three times a week

Target Size

• 60% of current Oura users (those most engaged with health tracking)
• Potential to attract new users specifically interested in stress management

User Value Definition

"Helping users understand and manage their daily stress levels to improve overall well-being and productivity."

Core user actions:

• Checking daytime stress feature daily
• Engaging with stress-reduction recommendations. Example: Guidance & Tips
• Checking the daytime stress trends to understand if there is a change in stress levels after following the recommendations
• Correlating stress levels with other health metrics

Business Impact Definition

"By helping users manage their daily stress effectively, we increase engagement with the Oura ring and app, improve overall user satisfaction, and drive retention. This feature also differentiates Oura in the competitive wearables market, potentially attracting new users specifically interested in stress management. We expect this to positively impact our daily active users, premium subscription conversions, and long-term customer retention rates."

Quantitative KPIs

Now, based on all the above qualitative information, we can define our quantitative KPIs:

1. Adoption:
   • Metric: Percentage of target Oura users who click and drill down on daytime stress tracking within the first week of the feature launch
   • Target: 40% activation rate
2. Retention:
   • Metric: Percentage of adopters who click on daytime stress tracking at least three times/week, four weeks after first use
   • Target: 70% retention rate
3. User Value:
   • Metric: Average improvement in user-reported stress levels after one month of feature use. This could be achieved through an in-app survey, app experimentation, or comparing the initial resilient score with the score after four weeks.
   • Target: 15% reduction in reported stress levels
4. Engagement\ User Value
   • Metric: Average number of stress-reduction recommendations clicked per week
   • Target: At least three recommendations followed per week
5. Business Impact:
   • Metric: Increase in overall app engagement (daily active users) attributable to stress feature
   • Target: 10% increase in daily active users among stress feature adopters
6. Sense Check

Looking at these metrics together, we can ask ourselves: "If all these metrics hit their targets over the next year, would we confidently say the daytime stress feature is a success?"

The answer should be yes, because:

• We're reaching a significant portion of our target audience (Adoption)
• Users are finding ongoing value in the feature (Retention)
• The feature is helping users achieve their goals (User Value)
• The feature is contributing to overall product success (Business Impact)

This comprehensive scorecard ties together our understanding of the product's place in Oura's ecosystem, the target audience, the user and business value propositions, and quantifiable metrics that indicate success.

It provides a clear roadmap for measuring the feature's performance and impact while aligning with higher-level business objectives.

Remember, these metrics and targets should be reviewed and adjusted regularly based on actual performance data and evolving business priorities.

‍