Governments can learn from Silicon Valley as they tackle AI and data science

A new study from the Oxford Internet Institute (part of the University of Oxford) and the London School of Economics examines how governments have gradually adopted Silicon Valley's way of working.

This shift offers valuable lessons as the new UK Labor government looks to embrace AI and data science in public services. However, the study also highlights ongoing challenges, such as outdated IT systems, reliance on conservative tech providers, and the dominance of big tech companies.

The study, titled "The Political Economy of Digital Government: How Silicon Valley Firms Drove the Adoption of Data-Science and Artificial Intelligence in Public Management," was published in the journal Public Money & Management.

It shows how governments in the U.S., U.K., and Australia were slow to modernize public services due to their dependence on traditional large-scale computer firms, known as "systems integrators." Public officials also hesitated to adopt Silicon Valley methods, which held back progress for decades. The study suggests that to better harness AI, the public sector must avoid repeating these past mistakes.

Professor Helen Margetts from the Oxford Internet Institute explains, "Our research looks at how government contracts for information systems have evolved over time. For years, public officials, working with highly traditional global tech firms, mistakenly believed that Silicon Valley's methods didn't apply to the public sector.

"But this mindset has now changed, driven by the growth of massive platform companies, the adoption of Silicon Valley technologies like cloud computing and AI, and the realization that these companies are often better at cybersecurity and managing sensitive information."

The rise of Silicon Valley technologies and cloud services has reduced the influence of traditional systems integrators, embedding these new tools across government departments.

Patrick Dunleavy, Emeritus Professor of Political Science and Public Policy at the London School of Economics, adds, "Over the next decade, Silicon Valley and platform companies' technologies will shape how governments handle data-intensive tasks. Digital managers within the government will play a crucial role in either speeding up or slowing down cultural change and in developing the expertise needed to manage diverse technology contracts."

While these technologies offer significant benefits for improving public services and the online experience of citizens, the researchers urge government officials to proceed carefully and ensure that these firms are held accountable.

Professor Margetts concludes, "A major challenge for the new government will be building the expertise and capacity needed to fully leverage AI and address the challenges it brings. They'll also need to effectively manage their relationships with both traditional tech providers and Silicon Valley giants while regulating the technology market. Our research offers valuable insights to help them learn from past mistakes."