AI risks: From existential fear to practical governance

Risks have been at the centre of AI debates for a long time. They reflect our hopes, fears, and uncertainties about a technology that is already changing how we work, learn, and communicate.

Today, AI risks can be understood through three broad categories: existential risks, existing risks, and exclusion risks. Each category operates on a different timeline. Existential risks remain more distant and uncertain, but they continue to dominate public imagination. Exclusion risks are likely to deepen over the coming years. Existing risks are already visible. By following how AI risks are discussed, we can see how policy priorities are shifting.

 Diagram, Venn Diagram, Disk
Survey of AI risks (2024, Jovan Kurbalija)

Existential risks: The fear of losing control

Existential risks are the most dramatic and most attractive in public debate. They concern the possibility that future super-intelligent AI systems could move from being human tools to autonomous powers beyond human control. In this scenario, AI would no longer serve humanity; it could become a force that shapes or even endangers human survival.

These risks are often compared to nuclear war, pandemics, or climate catastrophe. As they remain ‘unknown unknowns’, it is difficult to envisage what form they might take, when they might emerge, or how likely they are. This uncertainty makes governance especially difficult.

Yet uncertainty is not a reason for inaction. Climate governance shows that societies can prepare for deep uncertainty through scenario-building, precautionary thinking, early-warning systems, and international cooperation. The same approach is needed for AI. Existential risk should not paralyse policy, but it should encourage humility, preparedness, and long-term thinking.

Existing risks: The harms already here

Existing risks are part of our reality: job displacement, data protection breaches, misuse of copyrighted material, algorithmic discrimination, loss of human agency, cybersecurity threats, and the mass production of synthetic text, audio, images, and video.

AI is also creating new challenges in education. Students and teachers are struggling to define what counts as learning, authorship, cheating, or legitimate assistance. In public communication, synthetic content can flood the information space, increasing confusion and weakening trust.

The good news is that many of these risks do not require entirely new legal systems. Existing instruments can already do much of the work: privacy law, consumer protection, anti-discrimination rules, labour regulation, intellectual-property law, and cybersecurity frameworks.

The challenge is implementation. In some cases, new AI-specific mechanisms will be needed, especially for transparency, auditing, accountability, and risk assessment. But the starting point should be practical: use what already exists, identify the gaps, and build targeted instruments where necessary.

Exclusion risks: The danger of AI monopolies

Between immediate harms and distant existential threats lies a third category: exclusion risk. This is the risk that AI will concentrate economic, technological, and knowledge power in the hands of a few companies and countries.

Advanced AI depends on three scarce resources: data, computing power, and specialised know-how. Today, these resources are heavily concentrated. A few companies control the leading models, the infrastructure needed to train them, and many of the data streams that improve them. This concentration could shape markets, public policy, education, culture, and everyday life.

The danger is not only economic monopoly. It is also a knowledge monopoly. If a few actors control the tools through which societies search, write, translate, analyse, and decide, they will gain influence over how knowledge itself is produced and distributed.

This risk, which will sharpen in the coming years, will create sharper divisions between a few AI-makers and many AI-takers, dependent on systems designed elsewhere, trained on data they do not control, and aligned with values they did not help define.

The policy tools are already known: competition law, antitrust enforcement, data governance, open standards, public-interest infrastructure, intellectual property safeguards, and investment in local AI capacity. What is missing is often a political priority. Exclusion risk must be recognised before concentration becomes irreversible.

From Bletchley to New Delhi: The evolution of AI risk debates

The evolution of AI risks can be followed through the agenda of AI summits held over the last few years. The first AI Safety Summit at Bletchley Park placed strong emphasis on frontier AI and catastrophic or existential risks. This focus was understandable. Governments needed to acknowledge that advanced AI could pose risks beyond those addressed by ordinary technology regulation.

Subsequent summits broadened the agenda. Seoul placed greater emphasis on safe, innovative, and inclusive AI. Paris shifted the language from safety to action, highlighting implementation, inclusion, sustainability, and economic opportunity. New Delhi continued this movement by focusing on the impact of AI, especially on inclusion, development, and practical deployment.

This does not mean that risks have disappeared from the agenda. Rather, the discussion has become more balanced. The initial fear of existential danger has been joined by a more practical concern with present harms and future inequalities.

Towards a holistic approach to AI risks

Existential, existing, and exclusion risks must be addressed simultaneously with an adjusted focus as per technological developments.

For existing risks, governments should deploy and update current legal instruments. Data protection authorities, consumer-protection agencies, courts, regulators, schools, and employers already have roles to play. The main task is to make these institutions AI-ready.

For exclusion risks, governments should act before AI monopolies become structurally embedded. This means stronger competition policy, better data governance, support for open and interoperable AI ecosystems, and investment in public and local AI capacities.

For existential risks, governments and societies need precautionary governance. This includes scenario planning, scientific assessment, international cooperation, testing regimes, and mechanisms to pause or constrain systems that may pose unacceptable risks.

IGF 2023: Grasping AI while walking in the steps of Kyoto philosophers

The Internet Governance Forum (IGF) 2023 convenes in Kyoto, the historical capital of Japan. With its long tradition of philosophical studies, the city provides a fitting venue for debate on AI, which increasingly centres around questions of ethics, epistemology, and the essence of human existence. The work of the Kyoto School of Philosophy on bridging Western and Asian thinking traditions is gaining renewed relevance in the AI era. In particular, the writings of Nishida Kitaro, father of Japanese modern philosophy, shed light on questions such as human-centered AI, ethics, and the duality between humans and machines. 

Nishida Kitaro, in the best tradition of peripatetic walking philosophy, routinely walked the Philosopher’s Path in Kyoto alone. Yesterday, I traced his paths while trying to experience the genius loci of this unique and historic place.

 Person, Walking, Clothing, Coat, Path, Accessories, Glasses, Footwear, Shoe, Backpack, Bag, Architecture, Building, Outdoors, Shelter, Plant, Vegetation, Tree, Walkway, Garden, Nature

On the Philosopher’s Path in Kyoto

Here are a few of Nishida Kitaro’s ideas that could help us navigate our AI future:

Humanism

Nishida’s work is deeply rooted in understanding the human condition. This perspective serves as a vital reminder that AI should be designed to enhance human capabilities and improve the human condition, rather than diminish or replace human faculties.

Self-Awareness and Place

Nishida delved deeply into metaphysical notions of being and non-being, the self and the world. As the debate on artificial generative intelligence advances, Nishida’s work could offer valuable insights into the contentious issues of machine consciousness and self-awareness. It begs the question: what would it mean for a machine to be ‘aware,’ and how would this awareness correlate with human notions of self and consciousness?

Complexity

Nishida paid significant attention to the complexities inherent in both logic and epistemology. His work could serve as a foundational base for developing algorithms that can better understand and adapt to the complexities of human society.

Interconnectedness

Nishida’s philosophy is critical of dualistic perspectives that often influence our understanding of humans versus machines. He would likely argue that humans and machines are fundamentally interlinked. In this interconnected arena, beyond traditional dualistic frameworks (AI vs humans, good vs bad), we should formulate new approaches to AI.

 Book, Publication, Person, Reading, Adult, Male, Man, Novel, Face, Head, Accessories, Glasses, Kitarō Nishida

Nishido Kitara, founder of the Kyoto School of Philosophy

Absolute Nothingness

Nishida anchors his philosophy in absolute nothingness, which resonates strongly with Buddhism, Daoism, and other Asian thinking traditions that nurtured the concept of ‘zero’, which has shaped mathematics and, ultimately, our digital world. Nishida’s notion of ‘absolute nothingness’ could be applied to understand the emptiness or lack of inherent essence in data, algorithms, or even AI itself.

Contradictions and Dialogue

Contradictions are an innate part of human existence and societal structures. For Nishida, these contradictions should be acknowledged rather than considered aberrations. Furthermore, these contradictions can be addressed through a dialectic approach, considering human language, emotions, and contextual elements. The governance of AI certainly involves many such contradictions, and Nishida’s philosophy could guide regulators in making the necessary trade-offs.

Ethics

Nishida’s work aims to bridge Eastern and Western ethics, which will be one of the critical issues of AI governance. He considers ethics in the wider socio-cultural milieus that shape individual decisions and choices. Ethical action, in his framework, comes from a deep sense of interconnectedness and mutual responsibility. 

Nishida Kitaro would advise AI developers to move beyond just codifying ethical decision-making as a static set of rules. Instead, AI should be developed to adapt and evolve within the ethical frameworks of the communities they serve, considering cultural, social, and human complexities. 

Conclusion

As the IGF 2023 unfolds in the philosophical heartland of Kyoto, it’s impossible to overlook the enriching influence of Nishida Kitaro and the Kyoto School. The juxtaposition is serendipitous: a modern forum grappling with the most cutting-edge technologies in a city steeped in ancient wisdom. 

While the world is accelerating into an increasingly AI-driven future, Kitaro’s work helps outline a comprehensive ethical, epistemological, and metaphysical framework for understanding not just AI but also the complex interplay between humans and technology. In doing so, Nishida’s thinking challenges us to envision a future where AI is not an existential threat or a mere tool but an extension and reflection of our collective quest for meaning. 

A Philospher’s Walk in the steps of Nishida Kitaro could inspire new ideas for addressing AI and our digital future. 

Read more on Nishida Kitaro’s work on the Stanford Encyclopedia of Philosophy.