Putting AI at the Heart of the UK Government Research and Data Infrastructure.
The Evolution of Government Research
The United Kingdom stands at a unique crossroads as an international research leader, across many domains, but particularly those that shape government research and official data collection. With a rapidly evolving artificial intelligence (AI) landscape, ever-greater government engagement with cutting-edge technologies, and robust ethical and usage frameworks being introduced, a new window of opportunity has opened to embed AI at the centre of the nation’s research and statistics collection ecosystem.
Clear signals showing the need for change
Considering the current public sector research landscape, this is not simply a technical proposition, but also a policy imperative, where methods of data collection are increasingly primed for change. As an example of this, traditional research approaches — long the backbone of evidence-led decision-making — are now under increasing pressure. As highlighted in a recent Financial Times article, statistics agencies are struggling to deploy large-scale surveys that are robust, rich in insight, and consistently reliable year-on-year[1]. The UK’s Office for National Statistics (ONS) is no exception, which has led to political challenge and, in some cases, the removal of official statistical classification. The associated challenges are not insignificant — rising costs, declining response rates, and respondent fatigue all point to a system in need of augmentation, if not transformation.
A second FT piece draws further attention to a particularly urgent case — the difficulty of collecting trade data in services[2]. As a services-led economy, the UK relies heavily on accurate, timely data about cross-border services activity — from legal and financial services to software and consulting. Yet this data is notoriously hard to capture using conventional methods, relying heavily on primary research, which is limited in both granularity and responsiveness. Based on recent findings, this suggests that the value of existing research methods is increasingly constrained. If the UK is to maintain its global competitiveness and develop informed trade and investment policies, it must adopt new approaches to understanding the most important facets of the economy. The opportunity for new data sources to be harnessed is immediate.
A Bold, AI-Centric Research Approach
It is within these use cases, and more widely, that the role for AI-centric research methods is both clear and exciting. Advances in language understanding, observed learning, structured data gathering and deep reading techniques tailored to specific government research contexts offer transformative potential. These technologies can extract, synthesise, and analyse enormous volumes of data from the open web and other digital sources with unprecedented speed and precision. In doing so, they unlock entirely new modes of understanding — methods that are scalable, customisable, and repeatable. Indeed, the presence of official guidelines outlining the practical use of AI for Government purposes implies a growing recognition of its role[3]. Here, the emphasis is on leveraging the power of technology to bolster insight, efficiency and transparency, whilst following ethical principles and acknowledging limitations — as should absolutely be the case.
Building on this platform, and our experience of delivering AI data-led assignments for Governments globally, there are several areas where we believe open web sources and curated AI language models can be deployed to deliver new forms of economic insight:
· AI can be used to track the development of emerging sectors — such as Synthetic Biology, Quantum Computing, Green Finance and Advanced Materials — all of which are poorly captured by traditional statistical classifications.
· It can help policymakers understand how technologies are adopted and diffused across different industries, clusters and regions, providing insight at a much larger scale and a richness of data not possible before.
· At the company level, AI methods can identify trade-related activity from its digital footprint, capturing international exposure, global linkages, and perhaps most interestingly, the trading of economically valuable services.
· They can also uncover information about supply chains and commercial partnerships, to better understand relationships and embeddedness, by reading corporate websites, filings, and market intelligence.
· Perhaps most compellingly, they can offer more holistic assessments of company growth — looking not just at headcount or turnover, but at innovation output, research activities, hiring patterns, regulatory engagement, and even reputational awards.
Example UK Government studies which use and advocate AI research methods and data collection techniques
These are all examples of areas that have long challenged economists and statisticians, but where AI-powered analysis of open data sources can offer a step change. In some cases there is traction already — sector research being a prime example. We have deployed our AI technologies on hundreds of assignments, in the UK and globally, for the likes of the UK Departments of Science, Technology and Innovation/Business and Trade, the European Commission and Australian Government, to understand presence, scale, competitiveness, beyond existing limitations. These have breached new ground in terms of insight and granularity, informing policy, helping secure investment and to evaluate landmark programmes.
As a tech-led research capability, we see this as a genuine opportunity for the UK government, ONS, and other public sector institutions, to lead the world in frontier data collection. AI doesn’t just supplement traditional research — it reshapes what’s possible. Of course, this is not to dismiss the value of conventional methods. Primary research approaches, remain essential tools for understanding causality, sentiment, and the nuanced impacts of change across sectors, and wider society. They offer rich layers of context that are not always available from observed or secondary data. But they must now be part of a blended model — one that integrates AI-powered approaches into our national data infrastructure.
Activating the Vision
At Glass.AI, we believe deeply in this vision. Our technologies are already helping public sector organisations unlock new kinds of insight from the open web. We deliver this in a way that makes key components accessible to the user — an understanding of the technology, offering interaction with the data and highlighting the provenance of outputs. We hope to be part of the next wave of innovation in official statistics and policy research, working in partnership with institutions like the ONS, to develop AI-powered methods that complement and enhance traditional techniques.
Indeed, the ONS’s recent publication of its economics statistics plan (including survey improvements) highlights an acknowledgement that change is coming and opportunities to include new data inputs alongside existing methods of statistic collection[4]. This could be a fantastic platform to propel public sector data collection across the UK.
[1] Roula Khalaf, Statistics wrestles with the rising probability of dud data, Financial Times, May 2025, https://www.ft.com/content/1c30ccb5-ca05-4d49-a995-1c5a5540ef57
[2] Delphin Strauss, UK services trade data ‘of limited value’ says statistics regulator, Financial Times, May 2025, https://www.ft.com/content/fcc0db9f-8118-4130-965a-9a9530545733
[3] UK Government Digital Service (2025), Artificial Intelligence Playbook for the UK Government, Available from: https://www.gov.uk/government/publications/ai-playbook-for-the-uk-government/artificial-intelligence-playbook-for-the-uk-government-html
[4] Office for National Statistics(2025), Restoring Confidence, Improving Quality: Our plans for economic statistics and ONS surveys, Available from: https://blog.ons.gov.uk/2025/06/26/restoring-confidence-improving-quality-our-plans-for-economic-statistics-and-ons-surveys/