Share:

Governments need outcome measures as evidence to prove success. But there are five challenges to overcome, first.

The increasing demand for greater transparency of, and accountability for, the impact from how public funds are spent requires that governments measure their outcomes, not just their activities. That’s because the results we want from government aren’t that they can prove they’ve spent taxpayers’ money on services and programs. It’s that they can prove how our way of life is improved as an outcome of those services and programs.

The first step toward meaningful outcome measures in government is to understand the challenges that make outcome measurement so hard for government.

Challenge 1: Government outcomes are often intangible.

Inherently, social outcomes are not as tangible as outcomes of non-government organisations.

Most government agencies exist to influence social stability, justice or welfare; not to produce widgets. This, however, is not the reason that outcomes are hard to measure. If an outcome matters, we should be able to describe it clearly enough to observe, in some way, if it’s happening or not.

But how most government agencies describe their outcomes is not with clear language that describes anything observable. This example illustrates the typical style of writing used in government mission, vision and goal statements:

“Ensure that our counter-terrorism arrangements are resilient, collaborative, consistent and proportionate, both nationally and internationally.”

The common practice that limits our ability to meaningfully measure intangible outcomes is using vague weasel words to articulate those outcomes.

To succeed at finding meaningful measures of intangible outcomes, we cannot start with questions like “How could we measure this?” Rather, we make it easier and more meaningful if we start with questions like “How would we recognise if this was happening or not?”

Challenge 2: Government outcomes are often long-term.

Many social outcomes take years to change and cannot be meaningfully measured weekly or monthly.

If it’s true that government agencies exist to influence social stability, justice and welfare, then it can’t be ignored that such things change over longer timeframes. Some outcomes can take many years to change.

Most government outcomes can’t be measured as frequently as sales quotas, or revenue growth, or tonnes of coal exported. Like this outcome:

“Contribute to the improvement of the extent, condition and connectivity of Australia’s unique biodiversity and natural resources, consistent with national and international obligations, through protection of habitats and mitigation of threats to threatened species and ecological communities.”

The common practice that limits our ability to meaningfully measure long-term outcomes is limiting our measures to our planning horizons.

To succeed at finding meaningful measures of long-term outcomes, we can measure both the outcome itself and the results of our activities. We can develop useful lead indicators to monitor more regularly, and thereby project changes in our outcome measures.

Challenge 3: Government outcomes are systemically complex.

Government departments and agencies can struggle to isolate their contribution to social outcomes.

The role of government is more to influence than to directly control social outcomes. Government organisations often rely on collaboration with other public, private and nonprofit sector organisations for an outcome to be achieved.

Government departments and agencies can struggle to isolate their contribution toward outcomes they can only partially influence, or ‘facilitate’. For example:

“Facilitate jobs growth through policies and programs that promote fair, productive and safe workplaces, and facilitate the growth of small business.”

The common practice that limits our ability to measure systemically complex outcomes is defining accountability as ‘hitting targets’. No-one will accept being judged by a target they don’t have control of achieving. So, it’s no wonder that the above outcome is measured by such an operational activity measure like Percentage of claims processed within 16 weeks of receipt of an effective claim.

To succeed at finding meaningful measures of systemically complex outcomes, we need to build and test a causal map of how we influence the outcomes, and measure both the outcomes and the drivers we can more directly control.

Challenge 4: Government agencies are execution-driven.

Government agencies have stronger focus on executing their mission than on the performance of achieving it.

Humans generally find it easier to think about action than results. Actions are tangible, controllable, and immediate. Results are less so, since they’re the product after the actions.

In government, many results are – as we’ve seen– intangible, not directly controllable, and not at all immediate. It’s harder for people to connect with them, and so their performance measures are often just more actions:

• “Protect the safety, wellbeing, and interests of… aged care consumers through regulatory activities.”
• “Undertake quality audits and assessment contacts of [aged care] providers.”
• “Build organisational and clinical governance capability of [aged care providers] through targeted education and engagement.”

The common practice that limits how much focus is put on agency outcomes, compared to execution, is a confusion between performance management and program management.

To succeed at measuring outcomes meaningfully but without losing sight of execution, we need to make the time to set the outcomes that provide the context for execution, then use a causal map to build the relationships between and among agency activities and agency outcomes.

Challenge 5: Government agencies lack measurement capability.

Government agencies traditionally have not prioritised internal strategic performance management capability.

It is still not widely enough appreciated that measurement needs a methodology or proper approach. Current common practices leave out essential steps and proven techniques to make outcome measurement work well. Management, leadership and strategy training largely fails to teach good measurement practice.

It’s therefore understandable that so many government departments and agencies have poor measures. More useful measures of outcomes, and their drivers, will only come about when the measurement approach, skills and implementation are allowed to improve.

The common practice that limits our ability to develop internal strategic measurement capability is the lure of quick fixes to save time.

If a framework looks like it will save time, it can be too tempting to ignore. But simplistic frameworks like OKRs, when used on their own, typically lead to more trivial activity measures:

“Objective:
Foster partnerships with relevant stakeholders to amplify advocacy efforts.

Key Results:
Identify and engage with 5 strategic partners…
Develop joint advocacy campaigns or initiatives with 3 partner organizations…
Establish effective communication channels with 5 key stakeholders…”

Frameworks like OKRs can work, but they need the support of a complete performance measurement methodology to underpin them.

To succeed at finding the outcome measures that are right for our department or agency, we need to invest in internal measurement and performance management capability and treat it like the vital management process that it is.

The roadmap to meaningfully measure government outcomes.

The challenges of measuring government outcomes cannot be solved with current practices. We need a new outcome measurement roadmap.

This roadmap is more deliberate than it is difficult. It does have more steps than simplistic KPI frameworks, but that’s why it works:

1. Invest in the first wave of internal measurement capability.
2. Start with a fresh understanding of why we need to measure.
3. Measure the outcomes and strategic goals first.
4. Align the organisation by building a cause-effect map.
5. Encourage and hold the space for ownership and buy-in.
6. Use measures to drive continuous improvement toward targets.
7. Reflect and learn, then go back to Step 1.

To get the details of this roadmap, plus links to dozens of helpful resources, please download the full White Paper “How to Measure Government Outcomes” and feel welcomed to share with your colleagues.