Outcomes over outputs: How community service providers can better measure the true impact of their work

If you ask the staff at a community service provider why they come to work in the morning, they’re more likely to say “because I’m making a difference” than “because I’m trying to maximise service volume”.

However, the organisations they work for often define their success based on the quantity and quality of their services rather than on the long-term effects of those services on clients and the community. The reasons for this focus – on outputs rather than outcomes – tell you a lot about the sector.

The allure of outputs is easy to understand. Reporting on outputs is often a condition of funding, as the quality and volume of services are used as a proxy for impact and value-for-money. Outputs are also easier to quantify than outcomes, saving staff time and effort in collecting complex data, providing a clear metric to track progress, and minimising concerns about data bias and inaccuracy.

Additionally, outputs offer more immediate data than outcomes, which is particularly useful when writing the annual report or when funders want to know how their money is being spent.

All of this would be fine if measuring outputs provided a comprehensive picture of a service provider’s impact. Unfortunately, outputs don’t tell the whole story. 

If service providers only measure outputs, they risk mistaking doing more for having an impact. They further risk delivering services that clients do not want or need, delivering one-size-fits-all services that do not suit all clients, and overlooking the need for adjacent or additional services.

Outcome data counters these risks by allowing providers to ask whether they are meaningfully supporting clients to achieve their goals. Measuring outcomes in addition to outputs helps facilitate a client-centred approach to understanding impact, viewing success in terms of clients’ experiences and needs rather than the provider’s activities.

An outcomes framework is a tool that helps organisations define and measure outcomes by linking what they do with what they want to achieve. The process of developing an outcomes framework facilitates valuable discussions about an organisation’s values, its vision for the future, and the outcomes its clients care about most. 

However, the process of defining outcomes and implementing the resultant framework can be uncomfortable. It requires staff to define often abstract concepts – like hope, connection and prosperity – and embed new ways of thinking, which can feel overwhelming. The process can also uncover ill-suited or low-impact services, which may necessitate reforming or discontinuing services that staff feel an attachment to.

This is when leaders must step up.

There are a number of challenges commonly faced by the sector in implementing change. While staff often have deep subject matter expertise, they may not have much experience developing and delivering an outcomes framework. This can create hesitance or even resistance to change and make it harder for leaders to build buy-in. Similarly, staff often struggle to find the time for strategic thinking, particularly when the outcomes framework must compete against a dozen other projects already underway. This can leave the outcomes framework perpetually languishing in the ‘too-hard basket’. And there is always – always – the matter of funding, especially in a sector that typically devotes most of its resources to direct service delivery.

But forward-thinking organisational leaders have a number of tools at their disposal to address these barriers to change, and most community organisations have inherent strengths that they can leverage to that end.

Staff in community organisations often feel a strong sense of purpose in coming to work, and leaders can build buy-in by actively involving them in the design process, ensuring that the future defined by the outcomes framework reflects their motivations. Community services providers also often have a strong brand because they exist to meet a defined need in the community. A need that, when met, can generate considerable good will. Both brand recognition and community support can be leveraged to build commitment to the outcomes framework.

We recently worked with Spectrum Migrant Resource Centre, which was facing these same challenges. Spectrum wanted to better understand its clients and improve the way it drove change internally, across the sector, and in the community.

An off-the-shelf framework was not appropriate due to the diversity of Spectrum’s clients and services, which necessitated flexibility in how the organisation collected client data. The outcomes had to be applicable to clients at different stages of their life and migration journeys, as well as be accessible to people from different cultural, linguistic, and demographic backgrounds. 

While staff were initially nervous about developing the outcomes framework, with many noting that this was a new way of thinking for them and that that weren’t sure where to start, we established a set of core questions to guide the thinking and communicate how we would develop the framework with them. The table below outlines the key questions that we worked through with the staff.

In addition to the expertise of its staff, we leveraged Spectrum’s other core strengths. These included:

• Branding: We attended a community town hall event where we asked Spectrum clients to tell us their goals and what Spectrum means to them. This was a key early input into the outcomes. We then used support group meetings to ask follow-up questions and better understand clients’ lived experiences.
• Staff motivation: We iteratively co-designed, tested and refined the draft outcomes with staff through a series of workshops. This process gave staff the freedom to share ideas without judgement and gave us the opportunity to evolve our thinking quickly, to arrive at a set of outcomes that spoke to staff.
• Clear use case: We worked with staff and leaders to define the expected use of the outcomes framework and its benefit to the organisation and clients. This helped position the framework as a core element of the organisation’s future operating model, rather than a discrete and peripheral academic exercise.
• Capacity: We developed the outcomes framework over three rapid 2-week design sprints. This working rhythm created momentum and helped us avoid the delays Spectrum faced on other strategic projects, as staff realised that the project had a fixed timeline.
• Funding: Spectrum engaged Nous through the Community Partnership Scheme, a program that offers subsidised consulting services to not-for-profit and community organisations for projects that contribute to positive social, cultural, economic or environmental outcomes. This helped minimise the cost of developing the framework for Spectrum.

We also helped Spectrum leaders lay the foundations for effective implementation. This included establishing executive sponsors for each stream of work, who would be accountable for reporting on progress and coordinating staff to deliver implementation activities, and developing a draft timeline with deadlines for each stage of implementation. This would help the organisation’s leaders hold themselves to account.

Additionally, we designed a set of example survey and discussion questions to bring the outcomes framework to life, so that staff could see how the indicators worked in practice and gain confidence to do it themselves. This process helped us develop the outcomes framework below:

The outcomes framework was welcomed by staff, who felt that it properly reflected their own values, while the executive appreciated the ways in which it challenged them to hold themselves accountable, raising the bar for the organisation as a whole. 

Ideally, all this could be easily replicated across the community services sector. In reality, however, many organisations simply don’t have the resources to undertake this kind of project.

Governments and funding bodies have an important role to play here. They must change their thinking and expectations of providers so that the latter can collect outcomes data without overburdening staff or needing to demonstrate immediate results. Indeed, there is an opportunity for governments and the sector to come together to articulate what the sector is trying to achieve with and for its clients. A national discussion could serve as a turning point for how funding is allocated and what services receive it.

As we have seen, some organisations are already turning their attention towards the actual impact they have. However, more work is needed, and more providers need to deliver it. The most significant benefit of an outcomes framework is the way it allows organisations like Spectrum to understand, measure, and communicate their impact in terms that reflect the motivations of staff and the experiences of clients. Get it right and organisational leaders and funders will understand what staff have always known: a successful community service doesn’t ask “What did we do?” or even “How well did we do it?” It asks: “Did we make a difference?” 

Get in touch to discuss how an outcomes framework could help your organisation measure its true impact.

Connect with Paul Kennedy and Stephen Christopoulos on LinkedIn.

Prepared with input from Nikita Weickhardt and Tim Tucker.