Consulting 800 stakeholders to evaluate a sensitive government health program

The Department of Health and Aged Care administers health policy in Australia, including psychosocial support programs for people living with severe mental illness.

For people living with severe mental illness, quality psychosocial support can be vital to their recovery and wellbeing. The department funds psychosocial support programs for this group (complementing other programs for this group) and wanted to understand more about its programs’ effectiveness in supporting people with severe mental illness who are not receiving supports under the National Disability Insurance Scheme.

The department asked us to evaluate the programs and make recommendations on the future directions for psychosocial support services. We were asked to assess the implementation, impact and cost-effectiveness of programs, including the degree to which the services are meeting the needs of people with lived experience, their families and carers, and to what extent the programs were informed by evidence.

Our evaluation was built around seven key lines of enquiry: implementation, intended outcomes, the needs of consumers as well as carers and families, integration, cost-effectiveness, evidence-informed policy, and the future.

To get an accurate understanding, it was essential to draw on qualitative and quantitative information. We used a mixed-methods approach, triangulating multiple sources of data – including literature and policy, extensive consultations and data analytics – to assess the programs’ impact.

The literature review considered international and domestic policy and evidence, including global best practice in psychosocial programs and services.

Our consultations involved interviews and focus groups with around 800 stakeholders from federal and state government, all primary health networks (PHNs), clients and carer peak bodies, service providers and delivery organisations, and consumers and carers. We partnered with experts from the University of Sydney, who led the consultations with consumers, their families and carers, using a trauma-informed approach and involving evaluators with lived experience.

This infographic, part of the final report, shows the breadth of consumers consulted:

To tap into program data, we prepared service maps for all 31 PHNs and triangulated quantitative data to produce the best possible understanding of how the programs were implemented in each jurisdiction and their impact.

Our report, released by the Department of Health and Aged Care in April 2022, provided 18 recommendations.

We also provided PHN implementation profiles, a report on the voices of lived experience (from our partners at the University of Sydney) and a comprehensive literature review.

The findings and recommendations are informing decisions on the future of the programs and future directions for psychosocial support services.