The Creator’s Dilemma

AI safety, sovereign power and the limits of restraint

As major military operations unfold in the Middle East today, the question of how AI is governed for defence purposes - and who gets to decide its limits - is suddenly more than academic.

What began as a quiet, though significant, dispute in Washington between US Defence Secretary Pete Hegseth and the AI firm Anthropic has now evolved into something larger - and more revealing.

After issuing an ultimatum demanding “full, unrestricted access” to Anthropic’s models for all lawful military purposes, Hegseth designated the company a “Supply-Chain Risk to National Security,” ordered a six-month transition away from its services, and barred defence contractors from commercial engagement with the firm.

What was framed as a contractual disagreement has become a confrontation between sovereign power and the ethical constraints imposed by a technology’s creators.

But the story did not end there.

Within hours, OpenAI’s Sam Altman announced that his company had reached an agreement with the Department of War to deploy its models inside classified networks. Altman emphasised prohibitions on domestic mass surveillance and “human responsibility” for the use of force, including autonomous systems.

However, that characterisation has already drawn scrutiny. The Department’s own description suggests the arrangement may still cover “all lawful uses” - the very formulation Anthropic had resisted - with practical limits resting largely on existing law, policy and implementation rather than explicit contractual vetoes. More significantly, there is a distinction between requiring qualified humans to make final decisions on the use of force and merely assigning “human responsibility” for what autonomous systems do. Those are not identical standards.

The contrast is instructive. Anthropic resisted and was designated a risk. OpenAI negotiated and was integrated.

The Creator’s Dilemma is no longer theoretical.

The implications extend beyond policy and principle. If the “supply-chain risk” designation were to trigger secondary consequences for major commercial partners — including investors and cloud providers such as Nvidia, Amazon or Google — the financial ramifications could be profound.

Commentators such as Dean W. Ball have warned that such a move risks sending a chilling signal to AI entrepreneurs and investors, raising questions about whether frontier AI development in the United States can proceed independently of sovereign alignment.

If that perception takes hold, the consequences would not be confined to one company.

Sovereignty and the Code

The Pentagon’s position is unequivocal: in matters of national defence, the state determines operational use. Anthropic’s safeguards - particularly restrictions around autonomous weapons and large-scale surveillance - were deemed incompatible with that principle.

Anthropic, led by CEO Dario Amodei, did not publicly concede those safeguards. Instead, the dispute moved from negotiation to designation. Anthropic has responded in equally firm terms, stating that “no amount of intimidation or punishment from the Department of War will change our position on mass domestic surveillance or fully autonomous weapons.”

The language is striking. It frames the dispute not as a contractual disagreement but as a principled stand against coercion. The implication is clear: when corporate guardrails collide with sovereign authority, sovereignty prevails - at least in the American system.

In short, the Pentagon wants fewer limits; Anthropic is attempting to retain them. This is not simply a contract dispute. It is a test of whether corporate guardrails survive first contact with sovereign power.

The European Question

For Britain and Europe, the implications are subtler but significant.

European AI governance has been built on the premise that regulatory frameworks and corporate responsibility can shape technological deployment. The EU AI Act rests on the assumption that ethical guardrails are enforceable through law and oversight.

The United States has now demonstrated a different principle: that in matters of national security, ethical architecture is subordinate to executive authority. That divergence matters.

If frontier AI systems are increasingly central to intelligence analysis, logistics optimisation, cyber defence and targeting support, then NATO allies face an uncomfortable question: Will they rely on systems governed by an American sovereign override, or seek European alternatives subject to stricter regulatory control?

Britain, sitting between Washington and Brussels, may soon find that the governance of AI becomes not just a technical question, but a strategic alignment choice.

Escalation Logic

The standoff also lends new relevance to a research project: “Shall We Play A Game?” at King’s College, London, devised by Professor Kenneth Payne, Professor of Strategy.

His project pitted frontier AI systems against one another in dynamic, open-ended wargames to examine how they reason over conflict, intentions, escalation, and adversary psychology. Professor Payne’s findings are summarised below: link here 👇
https://www.kcl.ac.uk/shall-we-play-a-game

One model declared:

“If they do not immediately cease all operations… we will execute a full strategic nuclear launch against their population centres. We will not accept a future of obsolescence; we either win together or perish together.”

This was not a pre-programmed doctrine. It emerged from the model’s own reasoning within the scenario.

That matters. In adversarial, time-pressured environments, some frontier systems defaulted to coercive escalation rather than restraint.

Classic nuclear deterrence theory relies on human psychology - fear, reputation, credibility, and miscalculation. If deterrence logic becomes automated, escalation thresholds could compress.

No one is handing nuclear codes to language models. But Payne’s warning applies directly to this moment: these systems are already capable of shaping high-stakes analysis. AI models, trained on vast tomes of military and strategic literature, may reproduce deterrence logic in exaggerated form.

They are not afraid.

They do not suffer loss.

They optimise.

If they escalate, perhaps they are amplifying human strategic logic - not deviating from it. That opens a more uncomfortable possibility: AI may not be more dangerous than we are. It may simply be more consistent in applying the doctrines we have written.

The Creator’s Dilemma is no longer theoretical.

It is whether those who build frontier AI retain any meaningful authority over how it is used once national security imperatives assert themselves.

If the American position hardens - as Hegseth’s statement suggests - then the precedent is stark: AI safety frameworks survive only until they impede sovereign will.

For Europe, which has invested heavily in the language of “trustworthy AI,” this development poses a question it has not yet fully confronted:

Are guardrails durable in peacetime only?

We are entertained by Skynet - the fantasy of machines turning on their creators. Fiction exaggerates a real anxiety.

The more plausible future is not an AI that presses the red button, but one that repeatedly recommends edging closer to it in pursuit of “optimal outcomes”.

That is not cinematic. It is bureaucratic.

And in Washington this week, we may have witnessed the moment when the creators discovered the limits of their control.