Share:

Given that research has shown performance measurement doesn’t improve government performance, should we give up on measuring performance?

In June this year, the Australia and New Zealand School of Government (ANZOG) published some alarming findings from an AJPA research paper by Smith, Halligan and Mir, about the efficacy of performance measurement in the public sector:

The paper concludes that the failure to find evidence that performance measurement improves performance supports the view that performance measurement does not lead to improvement. – ANZSOG

The researchers concluded that the primary reason for government agencies to undertake performance measurement is “external review by organisations such as audit offices”. And that performance measurement only leads to improvement when the government agency has a “culture that supports quantitative thinking”.

Set aside the irony that any organisation could demonstrate improvement without measurement, could this really be the truth: that government should give up on performance measurement altogether?

But there *is* evidence that performance measurement improves government performance…

It’s hard to imagine that every government organisation should stop measuring its performance in response to this research. A partial study of performance measurement in only 9 government organisations in Australia is not enough weight of evidence.

Our PuMP Community has a growing collection of case studies that provide evidence to the contrary, two of which are publicly available (outside our PuMP Community) at the time of writing this:

• Reducing the cost of administrative waste in federal government
• Improving resource allocation in local government

What separates government organisations in our PuMP Community from government organisations like the 9 in the Smith, Halligan and Mir research is the approach that was taken to performance measurement.

The research design omits a vital component: measurement approach.

Every research design has limitations; it’s impossible to design a perfect study that takes account of every contributing factor. The Smith, Halligan and Mir research omitted two contributing factors that we’ve found are vital to the success of performance measurement in any organisation:

1. The performance measurement methodologies used by the organisation to develop, implement and use performance measures; and
2. The definition and qualities of excellent performance measures, compared to the traditional misunderstandings of what performance measures are.

These factors have far more impact on the usefulness of performance measurement for improvement than the jurisdiction and type of organisation. For example, this similar research studied the performance measures of 50 public sector and non-profit organisations in Australia. This research found two factors that directly impact the likelihood of performance measurement resulting in improved performance:

• Only 14% of organisations have clear and specific strategic goals in their strategic plans. Without clear and specific goals, meaningful measures are impossible to find.
• Only 6% of organisations have real measures that are direct evidence of their strategic goals. The rest had measures that were simple milestones or activities, and not direct quantitative evidence of their goal achievement.

Performance measurement is like any tool: if it’s broken or poorly built, it will never work.

If external audit is the driver, there are consequences.

The Smith, Halligan and Mir research also concluded that external audit is a significant driver of government organisations undertaking performance measurement. But are they driving the right measurement behaviours?

Many government organisations fashion their performance measurement approaches on the guidance and audit criteria developed by audit offices. But this guidance lacks the depth and rigour of a proper measurement methodology. This
guidance focuses, as it should, on qualities of good measures and not instructional how-to.

Audit offices are not the expert source of guidance and advice on how to go about performance measurement. But the consequences of organisations fashioning their performance
measurement approach based on audit requirements means shortcuts are taken and struggles continue to stand in the way of measurement leading to improvement.

What should we do about performance measurement in government?

It’s not a question of whether or not performance measurement is appropriate for government. Every organisation – public, private or non-profit – needs evidence to make decisions about performance. The important question is:

Why is performance measurement not working for enough government organisations?

From everything that our PuMP team has learned over the past 20 years, there are three significant parts to the answer:

1. Be deliberate in the performance measurement approaches we take, and adopt proven methodologies rather than the latest fads or traditional shortcuts.
2. Shift the culture of measurement away from accountability and blame, and toward collaborative learning and continuous performance improvement.
3. Insist that audit offices use an appropriate audit framework based on proven performance measurement methodology developed by performance measurement experts.

Performance measurement doesn’t lead to public sector improvement? Of course it won’t if they continue to use a flawed performance measurement approach.

Government will never improve performance through measurement without a proper measurement approach, and a proper measurement audit framework. [tweet this]