Raquel M Smith

How to Determine KPIs for Your Product using Vision-Oriented KPIs

Effective KPIs are sometimes akin to the pot of gold at the end of the rainbow – full of promise but difficult to find. Moreover, it can be tough figuring out how to articulate a vision, set a strategy, and determine which KPIs were essential to measure the health of the product. You can take a guess at which metrics are important, but then you run the risk of missing something or, worse, measuring the wrong thing.

To add to the difficulty, it’s hard to set a product strategy without knowing how healthy the product is in all of its areas. You may have an objective to grow sales by X%, but without a defined set of KPIs it’s hard to know what areas could really use improvement. An Importance vs. Satisfaction survey can really help and customer interviews are pots of gold in themselves, but you needed some metrics to round out the picture. The question becomes: which metrics?

We’re currently leading an effort at Dozuki to define our key performance indicators. We sell documentation and training software for manufacturing companies. It’s a fairly broad product with no shortage of metrics that I could possibly measure. We wanted to make sure the metrics were meaningful, though. Measuring something that doesn’t really make a difference to our customer is pointless at best and a distracting target for improvement at worst.

Because it’s difficult to come up with KPIs in the situation described above, I was struggling a bit. So, I came up with a framework for defining KPIs based on the product vision. Let’s see how it works.

What is a KPI?

This has been answered in quite a number of other places, so I won’t go too in-depth. In my words from a product perspective, it’s a metric that describes the health of the product in that specific area. It is something that can be used to:

  1. Help identify company objectives
  2. Track the progress toward those objectives
  3. Ensure that product changes elsewhere don’t impact product health in another area

How can KPIs be used to both identify and track company objectives?

Good question. This was something we really struggled with – we can’t come up with an objective unless we know which areas of the product are hindering the business (eg. KPI indicates lack of health). And according to online sources we can’t set a KPI unless we have an objective! It was a catch-22. We needed to find a way out – and that way out was our product vision.

Using Product Vision to Uncover KPIs

When you’re stuck in a conundrum where you need KPIs to help set company objectives, but you need those company objectives to determine KPIs, you need to get out of the loop. Something else needs to be at play to help you determine which KPIs matter to your company.

At Dozuki, we know what we’re making, and we know why we’re making it. This vision is something that’s felt pretty viscerally when talking to our customers. But no one had taken the time to write out that vision in a way that was actionable. Attempts had been made, but nothing that I felt was something that we were able to rally around.

After feeling my way around in the dark for 4 months, I decided to take action and come up with a product vision that really worked for us. What was cool is that the process I discovered resulted not only in an actionable product vision for Dozuki, but a set of important KPIs as well. I’m calling the results of this process Vision-Oriented KPIs.

Vision-Oriented KPIs are directly tied to your product vision. They help you see progress toward reaching your product vision, and they make sure that improvements to one KPI don’t degrade the performance of another.

How to determine KPIs using the Vision-Oriented KPIs process

There are four steps to this process:

  1. Outline your product’s value proposition
  2. Identify activities required to realize that value proposition
  3. Describe properties of those activities
  4. Identify meaningful ways to measure those properties

Step 1: Outline your product’s value proposition

Plenty has been written about how to determine your company’s or product’s value proposition. Use whatever framework you’d prefer, but make sure you come out of it knowing what your company offers that is different from what your customers are currently using.

Heads up: your company’s value proposition might be different from what you’d expect. For instance, we had a number of meetings about product vision before I started developing this framework and in all those meetings everyone talked about continuous improvement – companies use our software so that they could collect feedback from their operators about processes and continuously improve them. Awesome! Sounds great to me.

But it wasn’t until we started digging down to get to what the “core” of our product was that the CEO said something along the lines of, “No, what companies really want out of us is for their people to utilize the documentation. Improvement comes next – but making it easy for employees to use documentation is the crucial first step.”

By focusing so much on continuous improvement, we could have missed out on the real core value proposition – that we make it easy and desirable for employees to utilize documentation so that they can then move on to improving it. Documentation that isn’t used doesn’t get improved.

Make sure that when you’re identifying your product’s value proposition you’re focusing on the very core benefit(s) and value(s) that your product provides.

Step 2: Identify activities required to realize that value proposition

This is an important step because it helps you map out a path the user must take in order to receive value from your product. You can do this with story mapping and whatever agile technique you want to use. Moral of the story is this: know what basic actions a user must take before your value proposition can be realized by the end-user.

For instance, at Dozuki, our value proposition is that of enabling easy usage of documentation. In order to make it easy to use documentation, that documentation has to be:

  • Created
  • Managed
  • Found by the end-user
  • Usable

We also found that there were a couple other essential activities to our product that didn’t directly contribute to document use, but indirectly contributed. For example, someone needs to manage the platform and the users on the platform so that the documentation can be created and used by those users.

You don’t have to go too crazy outlining everything the product needs to do. We’re thinking big-picture here. If you have more than 6-7 prerequisites to realizing your value proposition, you might need to think about simplifying. Each activity is a hindrance to achieving value – if there are too many hindrances, your customers will never get to the value and you won’t realize your business goals.

Step 3: Describe properties of those activities

Next, it’s time to put some adjectives to use. For each of the activities identified in Step 2, write a sentence with 2-4 adjectives that describe the experience the user should have with that activity.

Think about all the things a user might want to do to achieve that activity. Then think about what that experience should be like for them. This is a best-case situation; it’s what you are striving for and what you wish with all your heart your product embodied. These are lofty ambitions, wildest dreams.

You might get sad as you go through this list as you realize how you are failing your users at providing this transcendent experience. If that’s the case, then you’re on the right track. If you’re simply describing things you already do, you’re not trying hard enough.

Some example statements we made for Dozuki are:

  • Creating & editing documentation is easy, flexible, and fast.
  • Document management is transparent, flexible, and encompassing.

For each of these descriptors, also write down what it means to embody and achieve that descriptor. For instance, under the creating & editing documentation section we wrote down:

What does it mean to be fast?

  • Limited mouse usage required
  • Intuiting required actions and doing them automatically
  • Stellar UI on any device

The idea here is to rack your brain to come up with fundamental properties of your product that help your users achieve the value your company promises.

Step 4: Identify meaningful ways to measure those properties

Finally! The KPIs we’ve all been dreaming of. The KPIs that aren’t just fluff, but that are directly measuring the health of your product as described by your product vision. Your Vision-Oriented KPIs.

At this point you should now have:

  • A defined value proposition
  • A list of activities required to achieve that value
  • Properties that describe the experience of those actions

Now, you’ll take those properties and think about how you can measure them. This is why these Vision-Oriented KPIs are so cool – they tie directly to your product’s value proposition – the reason why your product exists in the first place.

Take each property (eg, easy, fast, and flexible for creating & editing documentation), look at what it means to embody that, and think of ways you can measure progress in that direction. Some metrics will be easy to come up with; others will be more difficult.

For instance, when we were thinking about how to measure fast document creation, we decided that important metrics for this property were 1) the number of clicks required to edit a guide step, and 2) the amount of time it takes to go from initial draft to first publish.

Here’s what that section of our vision document looked like:

  • Creating & editing documentation is easy, flexible, and fast
    • What does it mean to be fast?
      1. Limited mouse usage required
      2. Intuiting required actions and doing them automatically
      3. Stellar UI on any device
    • How can we measure fast creation & editing?
      1. Average time from creating first draft to first publish
      2. Number of clicks

As a bonus (a kind of sad bonus, at that) we also listed ways in which we are failing to embody that property. I won’t list those out here 😉but it’s a healthy exercise to do.

Some properties will re-use metrics that you identified for other properties. For instance, measuring easy document creation is nearly the same as measuring fast document creation. This is okay.

And now, suddenly, you have a few product metrics that you need to be measuring. So cool!

What about the “standard” KPIs like churn, etc?

Sure. Those are very important. Those are listed out in numerous places across the internet, but they don’t pertain to the specifics of your business and the product you are creating. This article was about determining KPIs for your product.

Measuring KPIs is not a replacement for product strategy

As you start to get data on these you’ll see some that might need improvement, some that look pretty alright, and some that are in dire need of work. You might also start collecting these numbers, but then wonder what’s more important to users – is it fast creation, or document management transparency? Where should we focus our efforts first?

You might have a KPI that looks like it’s in sore need of improvement. But, will improving it really make the most difference to your customer? Or will improving this other one that isn’t as bad actually matter more?

These are questions that KPIs can’t answer. KPIs look at distinct properties of your product to determine their health in an isolated world. Product strategy, on the other hand, brings all your product properties together into the proverbial melting pot and determines how to move the business forward. KPIs contribute to this strategy by illuminating areas of the product that might need work, but they are not the whole picture. Use them wisely.

  • About
  • Projects
  • Bookshelf
  • Blog
  • Github@raquelmsmith
  • LinkedIn@raquelmsmith
  • Emailhello@raquelmsmith.com