My interview with Donald Wheeler is a very practical exploration of how we can better interpret signals from our performance measures and KPI’s.

Listen to Stacey’s interview with Donald Wheeler here:

Read the transcript here:

Hi everyone. I’m Stacey Barr, performance measure specialist from www.staceybarr.com and creator of the PuMP methodology. Welcome to the Measure Up podcast. My guest today is Dr Donald Wheeler. And possibly without knowing it, he is one of my most valued mentors and teachers.

Dr Wheeler is the author of a landmark book that my manager actually gave me back in the mid 1990’s when I still worked in the corporate world and I was a measurement consultant. Now Don’s book rocked my world.

The book is called “Understanding Variation: the Key to Managing Chaos” and it’s a book about a very powerful statistical technique called XmR charts – capital X, little m, capital R. And this technique, in my opinion, is the most powerful way that we can monitor our Performance Measures and KPI’s.

Now Dr Wheeler is an American author, a statistician and expert in quality control. And Don has been teaching people how to use their data more effectively for over 40 years. For 21 years, he was a student and colleague of Dr. Deming – who we know as “the Father of Quality” – and is a recipient of the Deming Medal. Don is a Fellow of the American Statistical Association, also a Fellow of the American Society for Quality. He has conducted over 1000 seminars in seventeen countries on five continents. He is author or co-author of 25 books and over 200 articles. And among these incredible achievements and contributions, Don is responsible for bringing XmR charts to life in modern-day management.

Consistent with Don’s very approachable and understandable speaking and writing style – he’s a statistician who knows how to speak to management and to the non-statistically literate – my interview with him today is a very practical exploration of how we can better interpret signals from our performance measures and KPI’s. So Don, thanks so much for accepting my invitation to share a tiny bit of your vast knowledge with our Measure Up readers and listeners.

Don: Well Stacey, thank you for the invitation!

Stacey: Excellent. So Don, we both know that people who work in management rely very heavily on comparing this month to last month (or the same month last year, or comparing this month to a monthly target) what they are trying to do is to assess if performance is getting better or worse. So why do you think this is so? Why do we do these month to month comparisons?

Don: Month-to-month and year-to-year comparisons allow us to place some perspective on the most recent figures from the accountants. They are a natural way of trying to create a context for the current values. In this regard there is nothing wrong with these comparisons. In fact, I find that I essentially make a point-to-point comparison when I compute moving averages for the monthly values that I get from our work. So the comparison is one that we would naturally use even if the accountants did not push it in front of us each month.

However, as you suggest in your question, the point of interest is not actually in the numbers themselves, but in what the numbers can tell us about the underlying process and how it is performing over time. This is where we often get into trouble with these point-to-point comparisons.

Stacey: So, if so many people are doing this Don, all around the world, and have done it for so long, how can it be wrong? What are the risks of this way of looking at our data?

Don: Well, point-to-point comparisons are natural, they are intuitive, but they are slightly naive. It is our interpretation of these comparisons that gets us in trouble. We usually interpret all the changes as signals that need to be explained. After all, two points always make a trend. The numbers go up and that is a signal. The numbers go down and that is a signal. The numbers stay the same and that is a signal too, because we were expecting them to go up or to go down, you know. Then everything becomes a signal and every signal needs to be explained. Unfortunately, these explanations only need to be plausible, they do not need to have any real contact with reality.

In the mathematical world two plus two is always equal to four. But in this world, in everyday life, two plus two is only equal to four on the average. When you think about this you know that from your own experience that there is variation in everything we do and variation in everything we measure. This variation creates uncertainty (or noise) in all of the numbers we compute, and this noise complicates the interpretation of simple point-to-point comparisons.

So how do we deal with variation? The first step is to understand that variation comes in two flavors. There is routine variation and there is exceptional variation. Routine variation will naturally occur even when the underlying process is unchanging. Exceptional variation will only be present when there has been a change in the underlying process. Since the purpose of analysis is insight regarding the underlying process, we will need to separate the routine variation from the exceptional variation. After all, as Aristotle taught us, in order to discover the causes that affect a process we need to study those points where the process changes.

So this idea has been around forever, so in business and industry the fundamental question is “Has a change occurred in the underlying process?” Exceptional variation will signal a change in the process, and by explaining the exceptional variation we gain insight into how our process works, we can discover how those cause and effect relationships that influence our processes and products, and we can learn how to operate our processes more effectively in the future.

On the other hand, seeking to explain routine variation is like seeking to explain noise. Here there is nothing to be learned. Any “explanations” we come up with are likely to be nothing more than superstitious nonsense. The underlying process is unchanging are we are misleading ourselves when we interpret routine variation as a signal of a change.

The fact that variation comes in two flavors means that there are two correct ways to interpret our data and two errors we can make when interpreting our data. The errors occur when we miss a signal that the process has changed or when we interpret noise as a signal. Therefore, as soon as we understand that variation comes in two flavors, and as soon as we understand what these two flavors mean in terms of the underlying process, we will want and need to learn how to differentiate between routine variation and exceptional variation.

So, while simple point-to-point comparisons may allow us to occasionally detect a real signal, they will more often result in false alarms as we run around like Chicken Little crying “The sky is falling! The sky is falling!” as we misinterpret noise as signals of change.

Stacey: Don, I see this Chicken Little behavior all the time, from the Board Room all the way to the shop floor and that’s a big reason why I am so keen to have people discover and experience XmR charts, which is the method I prefer for monitoring performance measures. So I announce to everyone that I know that you are the guru when it comes to XmR charts. So maybe you could give us a brief anatomy of an XmR Chart and what measures it can work for?

Don: The XmR chart is the simplest form of data analysis possible. It allows you to separate the exceptional variation from the routine variation, so you can pay attention to the signals of process changes and ignore the noise that occurs when the process remains unchanged.

The XmR chart begins with the basic time series graph where the data are plotted in time-order sequence. We do this because some of the most important information in our data is tied up in the time-order sequence. How are things changing over time?

We typically use the average for some baseline period as our central line for the X chart. This central line give us a reference for detecting trends or cycles over time.

We use the differences between successive values within the baseline period to capture the short term variation. We used this short-term variation to filter out the noise in the data. These successive differences are commonly known as “moving ranges.” Once we have computed these moving ranges we typically use the average moving range to compute our “three-sigma” limits for the X chart.

The three-sigma limits for the X chart are centered on the central line for X. They are found by adding and subtracting a constant to and from this central line. This constant is typically found by multiplying the average moving range by the scaling factor of 2.660.

When we have done this, these three-sigma limits will filter out 99% to 100% of the routine variation in the X values. Once we have filtered out 99% to 100% of the routine variation anything left over is a potential signal. So when a point falls outside these limits we classify them as a signal of exceptional variation and go look for an explanation of what changed and why it changed. If no points fall outside the limits, then we have nothing but routine variation and no explanations are needed.

All sorts of data can be used with the XmR chart. I tell my Students all the time, that the only limitation on this technique is your imagination. This approach makes no assumptions about your data. It is completely empirical. I have worked with all kinds of organizations in both the service sector and in manufacturing, and while I have occasionally been surprised by the way my clients have used the XmR chart, I have yet to find an area where this technique did not provide more insight than the traditional point-to-point comparisons.

Stacey: Don that is such a true statement. I teach XmR charts or a very brief introduction to them, in my courses and after those courses people are always sending me their XmR charts for all kinds of measures and they are just so excited by the signals and the story that they can now see in their measures that they never even knew where there. It’s fabulous. It’s just a revolutionary tool and I love your opening statement was about ‘it’s the most simple analysis technique’ and that’s exactly true. And it just makes it so accessible to a lot of people. So, can we explore then, how exactly is an XmR Chart better than month-to-month comparisons, than trend lines, or moving or running averages?

Don: Well, the essence of doing business is making predictions. Predictions require knowledge. XmR charts provide us with knowledge regarding the past behavior of our processes. If a process has been operated predictably in the past, then the past can be used as a reasonable guide for the foreseeable future. However, if a process is changing it will display exceptional variation and the past can no longer be used as a reasonable guide to the future. So, by characterizing the past behavior of our processes, we can begin to make reasonable predictions. This is just so intuitive that I have even had Bank Managers figure this out in 15 minutes, so you know, anybody can do it.

Point-to-point comparisons treat all differences as if they were signals, this results in a lot of confusion, and a lot of frustration, and a lot of chaos. With point to point comparisons we get explanations for why the numbers go up, or why the numbers went down. But these explanations do not have to be based on actual cause and effect relationships, they only need to be plausible enough for the gullible to believe. So we get a lot of explanations, but we get very few predictions.

Unless we have evidence that the process is actually changing in such a way that we can say that there really is a trend, any trend line created from the data will be nothing but wishful thinking. Once more, this requires that we filter out the noise before we start interpreting the data as containing trends or signals.

Moving averages, well that’s a different kind of tool, very useful in summarizing the underlying changes in data that are full of noise. I use them regularly for the monthly accounting data. However, in this use I am not trying to characterize the process but I am simply describing the past using report-card data. So, moving averages serve a slightly different purpose than XmR charts. They do not attempt to do anything other than to average out the month-to-month noise so we can see the long-term trends and cycles of the past. This can be very informative. I had one man, a CEO, show me some moving averages he was keeping and how he avoided shutting the plant down because he could see the trend developing it got to be a crisis.

You see management requires predictions. Predictions require knowledge and to gain knowledge we have to separate the signals from the noise. And this is the one technique that lets you do that. The other techniques are all are aimed at different things.

Stacey: That’s an important point, Don. I think it’s very easy to use statistical techniques in incorrect applications where they are really not, the technique really has not been designed for the question we are trying to answer with our data.

Don: You see, most of what you learn in your college classes on statistics, is in experimental data that are collected and going to be analysed one time and what we are talking about are data that are sequential, they’re continuing. We need a sequential data analysis technique so most of what you learn in your stat class really rarely applies in practice.

Stacey: What a shame!

Don: There are different kinds of data that are routinely used in business and industry.

Stacey: Absolutely. I think not a lot of people are aware of that. So Don, I studied statistics to a post graduate level. You are clearly very, very experienced and have a lot of wisdom and skill in statistical analysis. But do you think that XmR charts are going to be too hard for some people or too technical for managers or business analysts or strategy professionals to understand?

Don: I have had people who claim to be math-phobes use the charts successfully. I have had 11 and 12 year old kids create and use the charts. As outlined above, all that is needed is the ability to compute and use averages and differences. In one very successful case the team leader did not have a high-school diploma, yet they tripled their physical output, doubled their profit margin, and lowered the cost to their customer all without any capital expenditures.

The secret to carrying out a good analysis is not the complexity of the technique. It is not about having the right piece of software. The purpose of analysis is insight, and the best analysis is the simplest analysis that gives an insight. So, one of the key elements of the XmR chart is the fact that it displays the data in a visual format.

The running record on the X chart shows the actual process performance, and every point has to sink or swim on its own. Nothing can hide.

The three sigma limits on the X chart show what the process has the potential to do. They define the actual potential of a predictable process and they approximate the hypothetical potential of an unpredictable process. Thus, these limits define what the process will do, or what it can be made to do.

By combining the process performance on the same graph with the process potential, the X chart allows us to make a judgment regarding our process behavior. If the performance is consistent with the process potential, then we can say our process is operating up to its full potential. But when a point goes outside the three-sigma limits we are no longer operating it to our full potential, we have a signal of a process change, and by study of these points we can begin to understand the causes of process changes and then take action to prevent those changes from happening in the future.

So, much of the power of the XmR chart is in the simple graph that it uses. This graph tells you what is happening in your process, it helps you to understand what your process can be made to do, and it gives you a way to operate your process up to its full potential.

Stacey: Excellent. Now there is a little bit of a learning curve with the XmR charts. It is not something that you just open up on an Excel spreadsheet and start doing. You have to acquire some knowledge to do it. If an 11 or 12 year old can do it, anybody can, that’s for sure.

Don: They may need a little guidance because their thinking is still a little on the arithmetic level. The instruction to the arithmetic to the process behind it is a little bit of a chore.

Stacey: Indeed. But I have taught people this and they have just learned it on their first go. So as long as the instructions are clear, which your instructions are clear in your books, then absolutely it can be done. So what’s the payoff then Don, for taking the time to learn XmR charts and set them up in the way people report and monitor their measures?

Don: Well, one of my surprises when I left university and went in to practice was that I started hearing things that I did not expect to hear, like “the meetings don’t last as long”, ”we are communicating better than ever”, “these are honest men’s tools”. I have had clients tell me that the use of XmR charts has turned the company around, that they have gone from red ink to black ink, and they have saved millions by being able to reasonably predict performance and capability. The charts have been used to ramp up staffing and then to ramp it back down again as the needs changed in such a way that the service didn’t suffer.

While most of my clients have been in manufacturing, I have seen process behavior charts used successfully in departments of revenue, in banks, hospitals, insurance companies, and telecom companies. The only limitation as I have said before, is your imagination. To stimulate your imagination I have written several books that are full of examples and case histories of how others have used the charts.

Stacey: I am going to provide links to these books, in actual fact I’ve got copies of them sitting on my desk right now, but I will provide links for where people can get these books in the show notes for our listeners to check out. A final question, though, Don: When we are using XmR charts, are there any pitfalls we need really need to be aware of?

Don: Yes. The technique is easy to use but you need to be careful because virtually all of the software will allow you to compute the limits incorrectly. Since this problem makes a profound difference in the usefulness of the charts, you need a good reference book to define and outline the correct way of computing the limits. Now, the easy way to check your software is to use an example from a reputable text and verify the limits given by the software. In doing this you need to use an example that has points outside the limits. Those of you who don’t have signals, given the incorrect methods of calculating limits can give you reasonably approximate limits. But if you have signals in your data set, you’ll spot difference. If the software does not match the textbook limits, the software is incorrect and is to be avoided.

Now since there are only a limited number of ways the limits can be correctly computed you need to learn and use the correct ways. For an XmR chart you have only three options for the central line: Most commonly we use the average for a baseline period as our central line; occasionally we might choose to use the median for some baseline period as our central line; and very occasionally, when we have a process where the average is easily adjusted, we might use a target value as the central line for the X chart. In this latter case points outside the limits would indicate that the process was being operated off-target.

For XmR charts there are only two correct ways to compute the three-sigma distance for the X chart. This is where almost all of the software will give you the option of doing it wrong. Either you multiply the average moving range by 2.660, or you multiply the median moving range by 3.145. Any other computation that not equivalent to one of those two is incorrect. Other computations will hide the signals and mislead you.

Finally, when you choose a baseline period it is important that you pick a period where no known changes have occurred in your process. We are trying to compute limits that tell the story of what is happening in the process. Your baseline period will define the process potential and it will be the period against which future performance will be judged. So while it may have as few as four points, we generally prefer to have 12 to 24 points in a baseline period if possible. Having baselines with more than 50 values will seldom be of much interest. After all, we are trying to understand the current process, not what was happening back in 1999. This morning I literally got a data set that went back to 2000 and all the way up to 2014. It had 3600 some odd data points on it and just one set of limits. That was just this morning!

Stacey: That’s incredible. When you were providing some coaching for me Don, when I was creating the content of the XmR charts segment in my workshop, I remember you saying that anything more than 3 or 4 years old is ancient history in business. Do you remember that?

Don: Yeah, well in some places last week is ancient history. The difficulty is the context again. When we learn and use this tool well, it can have a profound effect on your organization.

Stacey: Great! So there is a little bit to learn to create and use these charts well, but getting this know-how isn’t hard, it is not time-consuming. Don, I have found that I can teach an introduction to this, including getting people going through their first example of creating an XmR chart and understanding the basic theory of it in less than 90 minutes. So they leave the workshop and they start creating their first XmR charts. And like I mentioned before, they will email them to me and get all excited about the story that they can now see. But while people can learn about in the introduction from my workshops, I always recommend that they go learn more from your work Don, from your books. So what are some of your recommendations, for what people should be reading to get a deeper understanding of this technique?

Don: When I am talking to people in the service sector and administrative and managerial type data, I recommend they start with the book you mentioned earlier “Understanding Variation, the Key to Managing Chaos”. That has 32 examples in it and they are all real. Sometimes the data have been disguised but they are all real stories. And this is the management overview, it’s the big picture without a lot of the details to get in the way. There is a sequel to that called ’”20 Things You Need to Know”. And these are 20 short chapters, appendices really, to understanding variation and they explain the pitfalls and explanations about how the charts work and answer frequently asked questions and kind of steer you away from tangents that can be problematic. But when you really get serious about getting down to this, there is a full sized text book called “Making Sense of Data: SPC for the Service Sector” and that has 130 examples and case histories on how to use these charts successfully. Including things like seasonally adjusting the data or de-seasonlising it. Trends and doing trend lines, charts from around trend lines. Just all the kind of questions that come up. As well as using some of the simpler tools and so that one is kind of the third one that really gives the nuts and bolts, that is where you got to for that.

Stacey: And like I said I have all of those books plus “Understanding Statistical Process Control” sitting here on my desk and I refer to them all the time. They are great books. So everyone, you can find Don over at his website – spcpress.com. He runs training courses regularly. Don than you so much, thank you so much for sharing your time and very profound knowledge with us. I really believe that getting more managers and decision-makers using XmR charts is going to radically is going to change the performance of the service sector, as much as it has done in the manufacturing sector over the past decades.

Don: Yes it does totally change your perspective once you start doing this. This has been fun. Thank you very much.

Stacey: It has been my pleasure. Thank you everyone for listening in today. Head on over to measureupblog.com for the transcript of this interview with Dr Donald Wheeler, and links to Don’s great resources and his website. This is Stacey Barr from www.staceybarr.com and Dr Donald Wheeler, signing off. Happy XmR charting!