Share:

There is a lot of doubt about how to correctly set a performance measure’s baseline using XmR charts. Remove your doubt by following this process.

Because routine variability is a natural part of any performance measure, we need a way to quantify the baseline of performance, the current as-is level of performance. Individual measure values are subject to routine variation so they won’t work. What does work as a performance baseline is the central line in the XmR chart.

But there is always doubt about exactly how many measure values to use to calculate the baseline, or central line, and when it should be updated as more measure values become available over time. Marina, a Measure Up reader, perfectly sums up this doubt in her email to me:

“In the past I heard people say that the central line is calculated from the first five points in the data set and only adjusted if there is a shift in the process – but this would assume that those first five points are representative of the process. Others say that the central line changes as you add data points – but then the result of applying the XmR signal rules could potentially change from one period to the next, if the central line shifts up or down. What is the right answer? Is it calculated at the beginning and left as a constant until it is manually adjusted to respond to a change in the process or does it evolve as data points are added?”

The doubt is, in part, because there are no hard-and-fast rules to follow. The trouble is, it depends. And the trouble is, it doesn’t matter as much as we might think it does.

Remove most of your doubt by following the steps I take, when I’m setting a baseline for a new performance measure. (Note, I’m not going to teach you the baseline calculation method, since it’s all here in how to build an XmR chart for your KPI.)

Step 1: Does the measure have less than 5 values so far?

If the answer is yes, we don’t have enough measure values to work out the baseline performance. We have two options here:

• Option 1: Check if we’re measuring as frequently as we can. If our measure is annual, and we have less than 5 years of annual values, then perhaps we can measure at a higher cadence, and instantly we’ll have more measure values (and can move on to Step 2).
• Option 2: Wait until we have five measure values. Patience is a virtue that will stop us from drawing fictional and false conclusions from our measure. There will always be a critical minimum amount of data needed before useful information is possible.

If the answer is no, we have more than five measure values already, then we can go on to Step 2…

Step 2: How many measure values should we use to calculate the baseline?

We probably shouldn’t say “should”. Should implies rules, and with setting baselines, it’s more about guidelines. But, probably like you, I make faster decisions when the guidelines are clear. Here we go…

Does the measure have between 5 and 10 values so far?

If we only have 5 to 10 measure value points, we can simply calculate the central line and process limits from the first five of those, or all of them. The sensitivity of 5 versus 10 versus more points to set our first central line isn’t nearly as high as people think. XmR calculations and signal rules are robust enough that we won’t miss a signal that’s really there.

Does your measure have stable variation?

If we have 15 or more measure value points, it’s useful to look at the measure in a line chart first. This way, we can get a feel for how much natural variation it has, in general. How many points we use for the baseline calculation depends on whether the variation is chaotic, or predictable.

If the variation seems chaotic and unpredictable, we have two options, depending on how bad the chaos is:

• Option 1: If the chaos is quite bad, a baseline won’t mean anything. The chaos means we don’t have much or any control over the thing we’re measuring. So we’d be wiser to focus on getting performance under some level of control before we worry about getting a baseline of performance.
• Option 2: If the variation is not quite chaotic but is quite large, it’s better to use more than five points to set the baseline. In this case, we can use at least 10 points, and up to 15 or even 20, to calculate our first baseline.

If we have predictable variation (even if we suspect a signal or two in there), then it’s just fine to start with the first five measure values to calculate our baseline.

With a baseline calculated, using a roughly appropriate number of measure values, we can go on to Step 3…

Step 3: When do we change the baseline calculation?

There is one incorrect reason and two correct reasons to change a baseline calculation for a performance measure:

1. INCORRECT: New measure values have been added to the chart, as time goes by. Simply because we have new measure values does not mean the original baseline is wrong or inaccurate. The whole idea of having a baseline is to have something set, against which to compare future measure values to determine if they are behaving the same (that is, performance isn’t changing), or if they are behaving differently (that is, performance might be changing).
2. CORRECT: The first baseline just doesn’t look right. If, say, we have 20 measure values that don’t really suggest any change over time. But when the baseline is calculated from the first five only, those five just don’t quite represent the set of 20. In this case, we could recalculate the baseline using the first 10 or 15 measure values instead.
3. CORRECT: The measure has a signal of change. If new measure values start behaving differently to the baseline, it might mean a signal is emerging (or has emerged). When we get a signal, and in XmR language it would be a short run or long run, then it means performance has changed, and the new level of performance needs its own baseline. So, leaving the old baseline there, we calculate the next baseline using at least five of the first points from the start of the signal.

If we don’t have either of the above two correct reasons to change a baseline, we don’t change it. We leave it there until our measure tells us it’s no longer behaving the same way any more.

***

It’s easy to over-engineer something technical, like the XmR chart. But it just wastes time and energy. The outcome is that we can pick up signals of change in our performance measure, quickly and reliably. We don’t need oceans of data to achieve that.

XmR charts are robust enough that we use as few as 5 measure values to set a baseline that can reveal real signals of change. [tweet this]