Share:

We can’t know if performance improves unless we know where it started. That’s why we set a KPI baseline. Here’s how to set your KPI baseline the right way!

Because most people lack an understanding of natural variability, they will set baselines for their KPIs the wrong way:

• average of last 12 months, which is wrong because over a 12 month period, there can be many shifts and changes in performance that are different to where performance is now
• last month’s performance, which is wrong because a single performance measure value is too subject to random variability to be representative of the KPI’s true level of performance
• using the target for the KPI, which is wrong because the target is where we want performance to be, but a baseline is where performance is now

Everything varies, including performance. The trick to setting a sensible and useful KPI baseline is to use enough performance measure values to calculate it, and not use any more than you need. The minimum number of performance measure values you’ll need is 5. Sometimes you need more than that, if the KPI’s variation is a little chaotic.

And the method to set your KPI baseline will depend on the maturity of you KPI. In some cases you can use historic data to set the baseline. But in other cases, you might need to collect some data for a while before you can set it.

Let’s explore how to set a KPI baseline for three common scenarios:

1. a brand-spanking new KPI with no data yet
2. a mature KPI with a lot of historical data
3. a mature KPI with a seasonal or cyclical pattern

If you have a brand-spanking new KPI with no data yet…

A new KPI for my business is PuMP Pilot Started, and it’s a simple count of the number of members in our PuMP Community that have started to implement PuMP after their training, on a simple pilot project.

I’ve only just finished setting up the new PuMP Community with a dashboard for members to log their PuMP Pilot progress, so I don’t really have much data yet to set a baseline.

At the moment, I have a spreadsheet set up, where I’m logging each member and the date on which they started their PuMP Pilot (data emailed to me from the PuMP Community dashboard).

But because I don’t really have enough data yet, I don’t know whether to measure this weekly or monthly. I’d want several pilots started each week, to make it worth measuring weekly. Then when I have five week’s of data, I will average the weekly totals to create the baseline.

TIP: Don’t create a baseline until you have a good feel for the frequency of calculation for your measure, and you have enough data for five measure values. Then average those measure values to set your baseline. You need at least five measure values for a valid baseline.

If you have a mature KPI with a lot of historical data…

The PuMP Blueprint Workshop has been running since 2006, but in 2007 I started to measure it’s Net Promoter Score, or NPS. Even if I focus just on the public workshops, there is a lot of historical data for this measure now.

If I hadn’t done it already (and of course, I have), I’d set up a baseline for the PuMP workshop’s NPS by averaging the first five values in my time series. Sure, that was years ago, but you can see how that historic baseline immediately highlights a subsequent improvement in 2010 (using the signal interpretation of XmR charts):

So a new baseline can be set from the start of that improvement, and it helps to see future changes. Can you see the new change in NPS after updating the baseline?

I don’t know the cause the for dip in performance that started in late 2013. I’m not worried because it was temporary, and also because an NPS above 50% is still considered excellent. (That doesn’t mean I’m not still working to improve NPS – but only when it’s a priority compared with improving other KPIs.)

TIP: Don’t be afraid to create historic baselines. When you do have a lot of historical data, you can start your baseline in the past, and use it to retrospectively see how performance has changed since that point in time.

If you have a mature KPI with a seasonal or cyclical pattern…

Seasonal patterns in our KPIs can be caused by lots of factors other than Mother Nature, like school holidays, financial or management reporting rhythms, shift work or maintenance schedules.

This KPI is Peak Day Water Usage, and clearly it has a seasonal pattern:

Before you set a baseline for a KPI that has a seasonal pattern, you must de-seasonalise it! Otherwise your baseline won’t be useful for any other part of the seasonal cycle than the one it was calculated in. Here is Peak Day Water Usage in its de-seasonalised form:

After we de-seasonlise Peak Day Water Usage, the current baseline can now be set by taking the average of the first 5 points that define our starting performance level:

And now you can see, quantitatively by the difference in the baselines, how much this KPI improved in mid-2009:

TIP: It makes no sense to set a baseline for a seasonal KPI, because you’ll be at risk of mistaking a seasonal change for a real change in performance. It’s easy to de-seasonalise your KPI before you set a baseline.

If you have a KPI that has chaotic up and down shifts or trends.

Chaotic performance measures or KPIs are those that vary unpredictably. They might be flat for a while then swing upwards, take a wild swing down again, stabilise for a bit then start swinging again. These KPIs are the hardest to set baselines for.

A great example of this kind of measure is the US Trade Deficit. Dr. Bill McNeese demonstrates how to set and use baselines as constructively as possible with chaotic KPIs like that one. His main point is that there is no one right way to do it. But read between the lines and notice how Bill shows the importance of thinking it through and making a call based on what seems sensible.

TIP: While it’s harder to set baselines for chaotic KPIs, it’s still worth trying, and only changing the baseline when the measure has stabilised for long enough.

Looking for evidence of improvement from the baseline?

In my world, a baseline is simply a quantitative value that describes current performance. When performance changes, for good or bad, the baseline changes. Your definition might be a bit different, and it doesn’t matter. What matters is that we understand, objectively and quantitatively, where performance was in the past, where it is now, and do the improvement projects that will shift it to where we want to be.

We can’t know if performance improves unless we know where it started. That’s why we set a KPI baseline.
[tweet this]

DISCUSSION:

How are KPI baselines set in your organisation? Are they sensible and helpful in driving performance improvement?