Share:

Finding the right KPIs to measure policy advice is a big challenge. But it’s not because policy is immeasurable. It’s because we don’t define policy success clearly enough.

Every government agency has to provide policy advice of some sort to their Ministers. Government policy, like any, should be well formed and well implemented so it can improve the Ministers’ decision making.

Australia’s Protective Security Policy Framework (PSPF), as an example, guides government agencies in how to protect their people and information and assets. It’s a complex policy to navigate and to implement. But it’s important to make it work. In most cases, if a policy was created, it was created because what it provides guidance on really matters.

Policy advice therefore matters, so it’s important to measure and important to continually improve. But where do you start?

Start with the customer of the policy advice.

For most government agencies, the most obvious direct customers of policy advice are Ministers. So many policy teams default to a measure of “Minister Satisfaction” to evidence the performance of their advice. But it’s too trivial.

There are other important customers of policy advice, since there are many users of the policy. For the PSPF, the customers include users in a large number of government agencies.

So a great place to start to measure something complex and intangible like policy advice is to list the customers that need that advice. You might even go a step further and find out what the most important attributes are for them, when they seek policy advice. Some will be obvious, some might be surprising:

• Timeliness of the advice
• Ability to understand the advice
• Ease of translating the advice into action
• The advice actually helping

An example of how to do this is a simple two-stage customer survey process that helps you measure what really matters to the customers of your service, in addition to measuring overall satisfaction.

Get clear about the policy advice process.

Very few people have a clear flowchart of their business processes. Or if they do, it’s simply too convoluted to be insightful.

We want to understand what aspects of our policy advice should be improved, in order to better service those who need our advice. So we need a flowchart of the policy advice process that is as simple as possible but not trivial.

With the flow of steps as the backbone, we can flesh out a cause-effect process model with:

• the policy advice outcomes most important to customers (see above)
• the tangible or observable outputs that lead to those outcomes, like a report or a conversation or specific recommendations
• the important in-process results, that currently aren’t performing well enough to deliver the outputs and outcomes that matter, such as understanding the customer’s real challenge, or quickly co-ordinating the experts needed to provide a piece of advice
• and the inputs, or qualities of those inputs, that have a strong bearing on the in-process results that matter, which might include enough policy experts, or information about answers to all previous policy questions

(I do realise this list is back-to-front relative to what most people do, but I like to start with the end in mind. This is so I can isolate what really does matter, rather than lose my way in an overwhelmingly comprehensive map of everything that could matter.)

For an example of how to map a process to identify the bits that matter most to customers, see this case study on customer-driven KPIs for a billing process.

Write result statements that describe how it should be.

Good measures are born from clear and measurable results statements. They’re kind of like goals, but we don’t worry about including targets and timeframes. We just want a statement that describes clearly what we want to have, as though we already have it.

We’d create results statements for each of the policy advice outcomes, outputs, in-process results and inputs that matter most. For example:

• Customers value their overall experience of policy advice
• Customers get their policy advice when they need it
• Customers can easily take action from the policy advice
• Policy advice reports are easy for customers to understand
• Access to policy experts saves us time

The key to writing measurable results statements is language. Avoid weasel words, don’t mistake actions for results, and keep them simple. Read more about how to do this well, with these three tests of the measurability of your results.

When you know what matters, and express it as results, you’re ready to measure.

Choosing measures is something people try to do way too soon. They ask “what are the KPIs for policy advice?” But they have no clear idea of what aspects of policy advice are worth measuring. They’re hoping for a quick fix, so they can say they’ve jumped through the KPI hoop.

If you really do want your policy advice to be better, stronger, faster, then you’ll want to follow a more deliberate approach to measuring it. If you take the time to understand the specific results that your customers want and your process needs to perform well at, then you’ll end up with the right measures.

Simply take those specific policy advice results, and put them through this measure design template:

• Policy Advice Overall Value: Average rating customers provide on a 10-point scale of the overall value of their policy advice experience with us.
• Policy Advice Timeliness: The average number of days past the date customers required policy advice that the policy advice was received by the customer.
• Policy Advice Actionability: Average rating customers provide on a 10-point scale of the ease with which they could take action from the policy advice they received.
• Policy Advice Clarity: The average number of minutes spent with customers to help them understand the written policy advice provided to them.
• Policy Expert Availability: The average number of days spent waiting for a policy expert to be available to assist with providing policy advice to customers.

To meaningfully measure complex, intangible things like policy advice, we must do the work to define success simply, specifically, and starting with our customers.
[tweet this]

DISCUSSION:

Do you notice how the process for making policy advice easier to measure is also a process for making anything complex or intangible easier to measure?