An evidence-based, insightful and well written study can substantially influence public debate, public policy and funding arrangements.
‘If you can’t measure it, you can’t manage it’ is a common aphorism. In a public policy setting, a sense of scale, often expressed in monetary terms, can be crucial to raise the profile of an issue, shift public perceptions and obtain the attention of policy makers. Economic studies that aim to estimate the costs of public policy challenges are commonplace, but too often lack sufficient rigour to be credible and have the desired impact on policy makers and public debate. An evidence-based, insightful and well written study can substantially influence public debate, public policy and funding arrangements.
This information paper draws on Nous Group’s experience to provide three key insights to help organisations better conceptualise costing studies:
A logic tree is a visual representation of the component parts of a problem. A good logic tree is the best foundation for an evidence-based and insightful costing study.
The multi-faceted nature of public policy issues means that costs can typically be grouped in a number of ways, such as cost type, by who pays, by life stage, and so on (see below). Ideally, the objective of the exercise should guide which tree (or trees) is used. If the objective is to best understand the relative size of different expenditure components, the first tree makes sense. If the objective is to understand the relative funders, the second tree should be used. Practically, the decision will often be influenced by the availability of data and how it is disaggregated.
The key test of a good logic tree is ‘MECE’ – each vertical branch should be mutually exclusive and collectively exhaustive. This approach ensures all costs are considered and avoids double counting.
Be clear about what is being costed
Public policy challenges are complex. Terms often mean different things to different people. Data sources may use different definitions. The audience will also bring their own preconceptions. A precise definition of what is being costed (and what is excluded) helps to overcome such challenges.
Decide whether to include direct and indirect costs
Governments, especially in a time of constrained budgets, are primarily interested in direct costs. Direct costs are the expenditures incurred to produce or deliver a particular good or service. For example, direct costs to support a person with a disability may include costs related to health care, accommodation support and job support.
Direct costs are distinguished from indirect costs – these are broader individual, social and economic costs. Indirect costs include pain and suffering experienced by individuals and loss of potential income. Indirect costs may be substantial, but are typically much harder to measure and link to government budgets. Studies that highlight substantial indirect costs (or indirect cost savings) can also lose credibility. For a costing study to be effective and influential, it is important to choose the right cost categories.
Having determined the appropriate categories of costs, be careful in determining what specific costs to include
Complex public policy challenges will typically have multiple causes. In estimating expenditure, a careful consideration of cause and effect is crucial. Estimating prison expenditure due to mental illness is a good example. A significant proportion of people in correctional facilities suffer from mental illness, though mental illness could be unrelated to their incarceration or a result of incarceration. To correctly estimate expenditure due to mental illness, it is necessary to estimate the proportion of people whose incarceration is a consequence of their mental illness.
Make sure all assumptions are justifiable
In complex areas, with imperfect data capture, assumptions are often necessary to estimate expenditure. The proportion of prisoners whose incarceration is a consequence of mental illness is a good example. Assumptions must be evidenced-based and clearly stated.
Take a conservative approach
As public perceptions are crucial, it pays to take a prudent approach to construct expenditure estimates. A more defensible, albeit smaller, estimate of cost is preferable to a larger headline-grabbing number built on shaky foundations.
