Performance and outcomes in regulation: How do you define and measure them?

Many regulatory leaders are increasingly being asked to define and demonstrate their performance and outcomes, but this is not a simple task. It can feel like a drain on already stretched resources, expose the regulator to scrutiny, and deliver limited value if not done well. These concerns are valid. Yet, there is growing pressure on regulators to clearly articulate what they are achieving through their interventions and how they hold themselves to high standards of performance.

This kind of clarity underpins public trust and has tangible implications for policy development, investment decisions, and the ability of regulators, government agencies, entities and other stakeholders to collaborate effectively on shared goals.

While many regulators struggle with this work, the reason is straightforward: it’s hard. But it’s also worth doing well. 

This article offers practical guidance to help regulators build performance and outcome measurement frameworks with confidence, so that the effort invested delivers real value.

Regulator performance refers to how effectively a regulator carries out its mandate. In Australia, regulators are often required to report on their performance in line with legislative obligations and published guidance. Historically, this reporting has leaned heavily on activity metrics, such as the number of licenses issued or inspections conducted. While these indicators are easy to quantify, they rarely capture the full picture of regulatory effectiveness.

In recent years, however, there’s been a shift toward more meaningful ways of conceptualising and reporting performance. This includes examining how regulators approach their work, whether they adopt a lifecycle or stewardship mindset, how well they use data and risk to guide decisions, and the extent to which they collaborate and engage with stakeholders.

For example:

• Commonwealth regulators report under RMG128, which outlines three principles of best practice: continuous improvement and building trust; risk-based and data-driven decision-making; and collaboration and engagement.
• Queensland regulators, where applicable, report against model practices that include proportionality, meaningful consultation, support for compliance, continuous improvement, and transparency.

Outcomes are the real-world effects of regulation, what changes because of the regulator’s work. These might include safer communities, cleaner environments, fairer markets, or more resilient systems. Unlike performance, outcomes are typically longer-term and often lie outside the regulator’s direct control. They depend on the behaviour of regulated entities, broader system dynamics, and external conditions.

This complexity makes outcomes harder to define and measure. But that doesn’t make them any less important. In fact, it’s precisely because they’re difficult that they deserve thoughtful attention. 

Developing robust performance and outcome frameworks is a strategic exercise as much as a technical one. It requires structure, clarity of purpose, and a dose of relevant expertise. 

Too often, frameworks are built around what data happens to be available. This is a classic case of the tail wagging the dog. Instead, begin by asking: Who are the audiences for these insights, and how will they use the information?

Ministers and Parliament may need high-level indicators that speak to public value and system-wide impact. Operational leaders will want data that informs tactical decisions. Regulated entities may seek transparency and fairness. The public may be looking for reassurance and trust.

These needs are not static, but are rather shaped by context:

• The maturity of the regulatory scheme: Is it stable, or undergoing reform?
• Recent incidents: Have there been high-profile failures or risks?
• Market dynamics: Is technology or practice evolving rapidly?
• Political landscape: Are there intersecting policy initiatives or pressures?

Understanding these audiences and their informational needs is not a nice-to-have. It’s foundational, and ensures that the framework delivers value and do not become just another compliance exercise.  

Once the purpose and audiences of your performance and outcomes framework is clear, developing a program logic is a smart next step. Program logics are structured visual or narrative tools that help clarify how regulatory interventions are expected to deliver impact. They map the logical flow from context, inputs and activities through to outputs and outcomes, making visible the causal pathways that link what a regulator does to the changes it aims to achieve.

This graphic shows the basic components of a program logic and a series of prompts a regulator could ask themselves to create version that is fit-for-purpose:

Once the logic is in place, it can become the foundation for surfacing insights across key domains, such has appropriateness, efficiency, reach and quality, and effectiveness. You can see what this looks like below.

Questions that can be posed under each domain include:

1. Appropriateness

• Are our objectives aligned with our legislative mandate and the public interest?
• Are we applying a risk-based approach to prioritise interventions?
• Is accountability for decisions and actions clearly demonstrated?

2. Efficiency

• Are we using resources in a way that minimises unnecessary burden?
• Are we leveraging data, technology, and partnerships to improve efficiency?
• Are we using feedback and performance data to drive improvement?

3. Reach and Quality

• Are we engaging stakeholders effectively?
• Is there consistency in how rules are applied and decisions made?
• Do we have mechanisms to assess the quality and fairness of our interactions?
• Are we building trust through clear communication and responsiveness?

4. Effectiveness

• Are we achieving the intended outcomes (e.g. improved safety, reduced harm, increased compliance)?
• Are there unintended consequences we need to understand?

Are our efforts contributing to broader system-level goals (e.g. innovation, sustainability, equity)?

Selecting the right indicators is both an art and a discipline. It’s not just about what data is availabl, but about what insights will be meaningful, credible, and actionable. 

A balanced mix of lead and lag indicators – those that predict future performance and those that measure actual outcomes – is often most effective. Combining quantitative metrics with qualitative insights also helps capture the nuance of regulatory work.

Below are examples of indicators aligned to the four key domains of performance and outcomes:

Turning indicators into insight demands the right analytical approach. Executives need to be confident that the analysis behind performance and outcome frameworks is not only technically sound but strategically useful. In many cases, simple inferential statistics – such as trend analysis, correlation, or variance across cohorts – can reveal patterns that help explain what’s working and where attention is needed. But in more complex regulatory environments, especially where multiple actors and external factors influence outcomes, contribution analysis may be necessary to build a credible narrative about impact.

Equally important is how these insights are communicated. Data alone rarely drives action. Stories do. That’s why storytelling through data is essential. Interactive dashboards, infographics, and tailored visualisations can bring numbers to life in ways that resonate with different audiences, from Ministers and regulated entities to internal teams and the public. The goal is not just to report, but to engage, inform, and influence. When done well, this kind of communication builds trust, sharpens strategy, and supports better decision-making.

Defining and measuring performance and outcomes in regulation is undeniably complex, but it’s also essential. When done well, it enables regulators to lead with clarity, communicate with credibility, and make decisions grounded in evidence. It strengthens public trust, supports strategic alignment, and positions the regulator as a confident contributor to broader system goals. The frameworks, tools, and techniques outlined in this article are not just for compliance, but also for leadership. By investing in thoughtful design, meaningful indicators, and compelling communication, regulatory leaders can turn measurement into a strategic asset that drives improvement, informs policy, and demonstrates impact.

Get in touch to discuss how we can help you to develop and roll out a strategically valuable performance and outcomes measurement framework

Connect with Lauren Ware and Chloe Sheptooha on LinkedIn.