Two woman working with data at a computer

Analytics in the public sector: Five actions to close the gap between data and policy decisions

Our Thinking | insight



6 Minute Read


Share insight


Almost every public sector leader has enormous volumes of data potentially available in their organisation, from internal transactional data to official statistics and alternative data sources from social media or the private sector.

It is easy to be overwhelmed by this flow of data. It can be challenging to organise large volumes of data from a range of sources in a meaningful and accessible way and tougher still to derive useful insights that can drive change.

But the expectations many people have of governments are vast and growing. They have seen how the corporate world uses data and this has shaped their understanding of what is possible for public sector organisations. Expectations among citizens and decision-makers have risen further with the rapidly changing information needs during the COVID-19 pandemic.

In our experience, there are five key things decision-makers want from data and analytics teams in the public sector (notably, none are buzzwords such as “artificial intelligence” or “machine learning”).

They want improved management and access for analytical use of data; better access to data through dashboards and interactive decision-support tools that are comprehensive and updated frequently; accuracy, transparency, and reproducibility in the production of insights for executive audiences and the public; more insightful use of statistical modelling for causal inference, forecasting and scenario exploration; and new, quicker, cheaper and quality-controlled official statistics.

Delivering each of these is challenging but achievable if organisations equip themselves to make best use of data.

Actions can set public sector organisations on the right path

Based on our experience working with public sector organisations, there are five actions every organisation should take to develop a modern analytics capability. These specific skills, processes and governance techniques do not spontaneously develop in an organisation but need to be cultivated through recruitment and training.

1. Assess your current team maturity and identify growth areas based on your priorities

Success in data analytics is impossible without the right skills in the team. Many analytics teams come together by centralising dispersed analysts from across the organisation who bring a mix of skills, practices and levels of capability.

To develop skills and practices it is essential to understand the current capabilities, then identify areas where growth is needed based on your organisation’s priorities. To support this process, Nous has developed a Public Sector Analytics Team Maturity Framework, which can form the basis for detailed diagnostics and roadmaps:

Elements from both sides of each tension are needed in the long run. The challenge is to work out when to act on each part. This requires a deep understanding of the needs and priorities of your stakeholders.

2. Build a team with the right skills, processes and governance techniques

Responsibility for driving improvement in the use of data typically rests with an enterprise-wide analytics team. Organisations that do not yet have this function need to consider developing it, and those that do have it need to give it the enablers to succeed.

The role of the analytics team in the public sector is to link the work of ICT specialists with the work of policy and program staff. This involves being the conduit to connect parts of the organisation that might otherwise be isolated from one another.

There are several ICT capabilities the analytics team needs, including powerful modern analytics tools; project design, delivery and governance disciplines; online collaboration, roadmaps, issues and bug tracking; source code version control; and agile project methodologies.

In addition, the analytics team also needs the best of policy and program capabilities, including responsiveness to stakeholders and to the policy environment; subject matter expertise; flexibility; pace and intensity; and transparency and accountability.

3. Foster a professional and disciplined workflow that learns from software engineering

Becoming a modern analytics team inevitably means acquiring and using the right tools for the job. Commonly used tools, such as Microsoft Excel and specialised statistical software SPSS, are not suitable to achieve the required levels of reproducibility, transparency and scalability. Data science teams around the world are converging on ways of managing code, data and artefacts.

The lessons are unequivocal on good practice to scale up analytics. It needs to combine solid data management, subject-matter expertise, coding and statistical skills with the lessons of software engineering. These lessons include the importance of code version control, properly designed data models, integrated testing, tracking of bugs and issues, project management, coding standards and user involvement in design.

For example, at Nous, for our data-intensive projects we use tools and processes that reflect best practice:

This toolchain has been put into operation in our use of our proprietary Data Analytics Warehouse for Nous (DAWN), which we use to rapidly develop insights for our clients on projects.

4. Build interactive tools to present data with real impact

Time-poor decision-makers, particularly those without a background in data and analytics, are unlikely to wade through confusing and technical presentation of information. The growth in data interactivity from leading publishers such as The New York Times, The Guardian and FiveThirtyEight have also raised expectations among decision-makers and the public for interactivity and data storytelling.

Fortunately, there has been rapid growth in modern tools that can quickly create high-quality, customised, interactive web applications to deliver insights. These tools include Power BI and Tableau – which enable analysts without coding experience to develop interactive dashboards – and packages such as Shiny – which extend the capability of R beyond statistical programming to develop interactive web apps without needing HTML, CSS or JavaScript.

These tools enable analysts to rapidly prototype web applications that directly link to the underlying statistical models and data. Shiny was used by Victoria’s Department of Health and Human Services to develop a Mystery Case Tracker. This screenshot (courtesy of the ABC) neatly demonstrates some killer features of tools such as Shiny: integrating powerful features of R such as network analysis, support for geospatial visualisation, and quick development of web apps.

5. Leaders need to act now

Demand for data will only grow during the social and economic recovery from COVID-19, so leaders need to act now. An experienced partner can be vital in helping you develop a roadmap to develop a modern analytics capability.

As a partner with extensive experience in data strategy and capability, Nous is ready to support you to help to realise the full potential of the data within your reach.

Get in touch to discuss how we can help you to develop your data capabilities.

Prepared with input from Martin Burgess.

Connect with David Diviny on LinkedIn.