Living evidence and AI: New opportunities for policymakers

Imagine a world where policymakers have instant access to accessible summaries of the consensus of current research, where decisions are backed by continuously updated evidence, all powered by the speed of artificial intelligence. The potential convergence of AI capabilities with the growing demand for evidence-based policymaking presents a unique opportunity to transform how we synthesise and use research evidence.

The current policy ecosystem is characterised by speed: the speed in which advice needs to be given –which can sometimes come down to hours – and the speed at which evidence needs to be sourced. That these are often at odds with one another, due in part to a lack of resources, is not without consequences. It can lead to policies that lack any evidence base, which, at best, are ineffective and, at worst, actively damage people’s lives.

Recent developments in Australia and globally indicate an emerging consensus on the potential of AI to enhance living evidence reviews, making this an exciting time for policymakers and researchers.

In recent years, the Australian government has demonstrated a strong commitment to evidence-based policymaking through initiatives like the Australian Centre for Evaluation. The Centre collaborates with government departments to conduct rigorous evaluations, including randomised trials, aiming to inform policy decisions with solid evidence.

However, producing and maintaining up-to-date evidence syntheses remains a challenge across various policy areas. Systematic reviews, which are often considered the backbone of evidence synthesis, are inherently static. They provide a snapshot of research at a specific point in time, which means they may quickly become outdated due to the rapid pace of new research.

Consider the example of infectious disease epidemiology. In 2015, an outbreak of Zika virus arose in South America. A systematic review begun in May 2016 found and synthesised more than 700 papers on the topic. But, by the time the review came out in January 2017, a further 1400 papers had been published and could not be included. This rendered the review outdated before it had even arrived.

Living evidence reviews represent a significant improvement over traditional systematic review. Unlike static reviews, living evidence reviews are continually updated as new research becomes available, ensuring that policymakers always have access to the latest evidence. This ability to keep information current is crucial in fast-moving policy areas, where timely and accurate data can lead to more effective decisions.

Assistant Minister Andrew Leigh recently emphasised the promising role that living evidence reviews might play in evidence-based policymaking. “It’s not enough to produce good evidence,” he told the UK Evaluation Task Force in London. “We also need to ensure that it is accessible to policymakers. […] [T]here is a recognition that the best practice involves living evidence reviews, continuously updated as new studies are published.”

International initiatives are also helping to shape the future of evidence synthesis. For example, the Wellcome Trust has committed £45 million over five years to develop tools that accelerate living evidence synthesis, while the Economic and Social Research Council (ESRC) has announced £11.5 million in funding to build infrastructure for AI-driven evidence synthesis. These funding initiatives reflect a growing recognition of the need for sophisticated tools to manage vast and constantly expanding bodies of research literature.

David Halpern and Deelan Maru, in their recent report A Blueprint for Better International Collaboration on Evidence, emphasised the importance of living evidence reviews and called for more international collaboration on evidence collection and utilisation. They also noted the “potentially ‘transformative’ role [AI] could play, both in the synthesis of evidence, and in driving up adoption of evidence through how evidence is presented.” AI's ability to streamline processes, identify connections across studies, and update evidence in near real-time is crucial to building an ecosystem that meets the needs of policymakers worldwide.

AI can play an important role in evidence synthesis, transforming the way research and data are analysed and utilised.

AI enables rapid processing, analysing vast amounts of research literature much faster than humans can, significantly reducing the time required for comprehensive reviews. AI-driven systems can also continuously update evidence reviews as soon as new studies are published, ensuring that the reviews remain current and reflective of the latest scientific findings.

Furthermore, AI excels in pattern recognition, identifying patterns and connections across studies that might not be immediately obvious to human reviewers, thus providing novel, perhaps counterintuitive insights into complex policy issues. (An AI method developed by Markus J. Buehler, the McAfee Professor of Engineering at MIT, which blends generative AI with graph-based computational tools, was recently used uncover unexpected formal similarities between biological tissue and, of all things, Beethoven's Symphony No. 9'.)

By automating parts of the synthesis process, AI also enhances the accessibility of evidence to policymakers. This automation allows for the distillation of vast quantities of information into concise and current summaries, which are crucial for informed decision-making in a fast-paced policy environment.

The future of evidence synthesis lies in combining human expertise with AI capabilities. As Assistant Minister Leigh pointed out, Australia has decades of experience in identifying evidence gaps and rigorously testing policy interventions. Integrating AI into this process could significantly accelerate our ability to deliver timely, comprehensive evidence to decision-makers.

However, to fully realise this potential, we need to address several challenges. Ensuring that AI-generated syntheses maintain the high quality and reliability of traditional methods is crucial. Similarly, policymakers and the public must trust the evidence they use. Transparency in AI processes is essential for building this trust.

This is the challenge of designing such a system. The design must prioritise transparency in how AI processes evidence, ensuring that policymakers can understand and trust the underlying mechanisms. This means developing interfaces that explain the decision-making paths, as well as the criteria used to include or exclude specific studies. Building mechanisms for expert human oversight, where domain specialists review AI-generated syntheses for accuracy and relevance, can play an important role in building trust. Additionally, learning from open-science processes, such as transparent methodologies and shared data practices, can further enhance the credibility and accessibility of the system. International collaboration, involving shared standards and collaboration, can also strengthen the system's reliability and reduce duplication. The goal is to create a system where AI works with human expertise, with clear accountability at every stage, allowing policymakers to rely on the evidence provided.

By investing strategically in AI-powered tools and fostering international collaboration, we can create a more responsive and effective evidence ecosystem, one that meets the needs of decision-makers and ultimately improves outcomes for the communities they serve.

Get in touch to discuss how your organisation can embrace the potential of AI.

Connect with David Diviny, Will Prothero, and Virginia Wong on LinkedIn.