Share:

Despite the existence of government performance measurement audit processes, performance measurement in government isn’t improving. This is why…

For a long time now, governments have been trying to get a stronger alignment between agency outcomes and the budgets they are allocated to produce those outcomes. And so the purpose of auditing performance measurement in these government agencies is to create more accountability for this, and provide feedback on how to improve it.

There are many governments around the world that provide performance frameworks to guide how their agencies should measure and manage performance. Searching for the details of the overarching model and criteria of such frameworks, however, is very difficult. Each country has a confusing labyrinth of webpages and pdfs that relate to performance auditing. But at least some examples of government performance frameworks include:

• Australia’s Public Governance, Performance and Accountability (PGPA) Act
• United Kingdom’s performance framework for its Single Department Plans
• Canada has an Audit of Departmental Performance Measurement process
• United States’ Government Performance and Results Act (GPRA)

Despite the existence of these performance frameworks, and the audits that can follow, government agencies are still not demonstrating excellence in performance measurement, nor the achievement of their outcomes. How can this be?

Government audit frameworks have limitations.

Audit offices are experts in auditing, not performance measurement. Couple that with the requirement governments so often have to be impartial to any proprietary method. The result is that their performance frameworks list criteria or requirements that can be vague, incomplete, prescriptive and somewhat random. For example:

• “include measures of an entity’s outputs, efficiency and effectiveness” – one of six requirements for performance indicators from the PGPA
• “a process is in place to establish indicators and targets” – one of five audit areas used in Canada’s Audit of Department Performance Measurement
• “balanced, giving a picture of what the organisation is doing, covering all significant areas of work” – representing the ‘B’ in the UK government’s FABRIC model of evaluative criteria

On top of this, each governments’ frameworks emphasise a different set of criteria or requirements. Which country has it right, and how wrong is every other country?

Shouldn’t there be one complete framework of performance management that guides government agencies to measure how well they provide the outcomes they exist, and are funded by taxpayers, to deliver?

The criteria of a performance framework needs a framework.

It’s not enough to give a set of unholistic guidelines as a performance framework. To improve government agency performance, we need a fundamental shift in what performance and its measurement really means. And audit frameworks and processes have the opportunity to facilitate this bigger shift.

Business excellence frameworks around the world – like ABEF, the Baldrige Award and EFQM – have done the thinking about how to build evaluation frameworks that stand the test of time, that really do guide best practice. And they do this without being vague or prescriptive.

One part of their evaluation framework is the concept of ADRI (borrowed from the ABEF). ADRI stands for Approach, Deployment, Results and Improvement. To evaluate an organisation against the business excellence framework, we look for evidence of these four parts, against each category of the framework.

Business excellence frameworks include categories that directly relate to performance measurement, like these in the ABEF:

• Category 5: Information and Knowledge
• Category 7: Results and Sustainable Performance

Therefore, in evaluating government agencies against a performance framework, we can learn from these business excellence frameworks to get more rigour and depth.

For the purpose of discussion, I present to you an example of how government performance audit frameworks could be designed.

Let me stress: I am providing examples only, and this is most definitely not a complete set of criteria for auditing performance measurement!

Approach – is there a robust performance measurement method?

ADRI’s Approach criteria would evaluate if the agency has a deliberate and robust method for its performance measurement process.

For anything to be auditable, it has to have a methodology. So, for performance measurement to be auditable, there needs to be a performance measurement methodology. What’s the point in auditing random KPI behaviour that changes from organisation to organisation?

A performance measurement audit framework would then have specific criteria relating to the approach adopted for performance measurement. For example:

• There is an approach for the selection, implementation and use of performance measures throughout the agency.
• The approach ensures the agency can create the performance measures that evidence its outcomes and the drivers of those outcomes.
• The approach guides how performance measure calculation, data requirements, interpretation and reporting requirements are decided and documented.
• The approach makes it clear how performance measures are reported and interpreted to identify performance gaps that need to be closed.

Deployment – is the performance measurement method consistently implemented?

ADRI’s Deployment criteria would evaluate how consistently the agency is implementing its performance measurement approach.

In particular, the implementation of the approach would consider its consistency over time, the breadth of its use across all organisational functions and processes, and the depth of its use from the executive team to front-line teams.

A performance measurement audit framework would also have specific criteria relating to the deployment of the performance measurement approach. For example:

• The right people are involved directly in developing the performance measures they will use.
• All performance measures developed are aligned to the agency’s strategic goals and purpose (mission and vision).
• The approach is implemented as part of the routine work of strategy development, performance reporting, decision making, and performance improvement.

Results – how well is performance measurement working?

ADRI’s Results criteria would evaluate how well the agency’s performance measures are helping to improve agency performance.

At the risk of making your brain bend back onto itself, here we are taking about measures of performance measurement. What is the evidence of performance measurement working well? If we agree that the role of measurement in government is to help reach the outcomes promised with the budget provided, then we have a starting point.

A performance measurement audit framework also needs specific criteria relating to the results of the performance measurement approach. For example:

• Performance measures are trusted and used by their owners.
• Performance measures show progress toward targets, or targets are met.
• Unintended consequences resulting from use of the performance measures are managed or mitigated.
• The return gained from the performance measurement approach (e.g. amount of performance improvement) is greater than the cost of performance
  measurement (e.g. data, reporting).

Improvement – how does performance measurement get improved?

ADRI’s Improvement criteria would evaluate how well the agency has evolved and improved its approach to performance measurement, based on reflection on how well it works (ADRI’s Results).

Do they ever reflect on how well it’s working? What is the agency learning and improving about their performance measurement approach? If they see it’s not working well enough, to they find the causes and improve their approach? What’s the evidence that changes made to its performance measurement approach actually changed the results of how well it works?

A performance measurement audit framework finally needs specific criteria relating to the improvement of the performance measurement approach. For example:

• The results of the performance measurement approach are routinely reviewed to identify where it works well and where it needs improvement.
• The performance measurement approach is only changed based on evidence of where it is not working well enough.

***

Do you also believe we need to take a serious look at how our government organisations are being encouraged, by way of audit, to measure their outcomes? What’s happening now isn’t working. I wrote this article for discussion. I don’t have the answers, and I am keen for you to share your thoughts.

Shouldn’t there be one complete performance framework to audit how well any government agency on the planet provides the outcomes they exist, and are funded by taxpayers, to deliver? [tweet this]