Woman making a payment on her phone

Using a theory of change to evaluate regulation of services

Our Work | Case Study

3 Minute Read

RELATED TOPICS

Share Case Study

The Department for Business, Energy and Industry Strategy (BEIS) was a UK Government department. (It is now known as the Department for Business and Trade.)

Department wanted to identify potential improvements to regulations

BEIS oversees several regulations, including the Provision of Services Regulations (PoSRs), which aim to reduce the bureaucracy associated with applying to deliver services in the UK.

The PoSRs mandate that competent authorities that authorise the delivery of services (such as local authorities or professional bodies) minimise the barriers to service delivery. The regulations place duties on businesses to share certain information with their customers to improve the UK consumer experience. The regulations were introduced in 2009 as part of the EU Services Directive.

BEIS asked us to evaluate the PoSRs to understand their impact and effectiveness and identify areas for potential improvements.

We developed a detailed theory of change

It had been more than a decade since PoSRs were introduced and there was no meaningful baseline data to draw from, making it difficult to understand the historical impact of the regulations. This required the evaluation team to focus on the current effectiveness and impact of the PoSRs in order to make recommendations about their future.

We developed a detailed theory of change: a description of how and why a desired change is expected to happen in a particular context. Our theory of change mapped the intended inputs, activities and outcomes to assess the extent to which the regulations were being implemented as intended, and the associated benefits.

Drawing on key evaluation questions to test our theory of change, we engaged 128 businesses as well as staff in 46 competent authorities across the UK via surveys and interviews. The consultations aimed to understand the extent to which stakeholders were aware of – and could comply with – the PoSRs. They also sought to understand the extent to which the PoSRs increased the ease of doing business in the UK and any ideas for improvements.

The government is taking forward the recommendations

We made recommendations relating to the content of the regulations and strategies for raising awareness of them. The government is considering these recommendations as part of a broader review to ensure the regulations remain fit for purpose. Our report is available here.

What you can learn from this project

You can robustly assess policy effectiveness years after commencement without baseline data, but understanding the costs and benefits over time will be challenging.

If there is no theory of change developed at the start of a policy, policymakers will need to establish an agreed theory of change at the outset of the evaluation.

Where regulations apply to a large population, you will need to go the extra mile to access a wide range of stakeholders.