Technocracy: Addicted to Risk

Human civilisation has become extraordinarily good at identifying dangers that it appears increasingly unable to stop producing. Climate change is measured in exquisite detail while the economic machinery intensifying it continues to expand. Cybersecurity grows because digital systems become more pervasive, interconnected and vulnerable. Military technologies generate countermeasures that generate new weapons in return. Artificial intelligence provides the clearest contemporary example: researchers, executives and governments openly discuss catastrophic and even existential risks while simultaneously accelerating development because slowing down might allow somebody else to get there first. None of this requires people to be stupid or secretly suicidal. Each decision can be reasonable within its immediate circumstances. The problem appears when those decisions join together. We are not failing to recognise the danger. We have constructed a civilisation in which recognising the danger has become another reason to continue producing it.

The language of progress conceals this because technological capability rarely remains merely a capability. Possibility becomes infrastructure, infrastructure creates dependency, and dependency makes retreat increasingly difficult. What begins as an advantage becomes something competitors must adopt, institutions must accommodate and eventually ordinary life cannot easily function without. Risk then becomes productive. Climate instability creates markets for adaptation. Cyber insecurity expands cybersecurity. Better weapons demand better defences, which demand better weapons. AI risk generates alignment research, regulation, surveillance, defensive systems and further demands for increasingly capable AI. The warning is not ignored. It is absorbed. Danger becomes information, investment, employment, competition and justification for another turn of the machinery. We increasingly solve the consequences of technological development by extending the technological conditions that produced them.

Artificial intelligence sits directly inside this contradiction. We worry about controlling AI while building AI because it offers unprecedented capacities to predict, classify, persuade, administer and control human beings and the systems through which they live. Capital and technocratic power keep moving toward the promise that enough data, enough computation and enough prediction will finally make uncertainty manageable. The problem is that every increase in control changes the world it is trying to control. New systems alter markets, behaviour, institutions, security conditions and strategic competition; those changes create fresh uncertainty, fresh vulnerabilities and fresh reasons for intervention. Complete control is not merely difficult. It is structurally self-defeating. Every successful intervention changes the conditions under which the next intervention becomes necessary. Risk persists because attempts to eliminate it generate new dependencies, vulnerabilities and uncertainties. The closer the system gets to closure, the more conditions it creates that prevent closure.