Every threat has an address. Whoever does this will pay is the oldest sentence in strategy, and its most important word is the first one.
The Latin is blunt about what deterrence is for. Deterrere means to frighten away, and you cannot frighten away someone you cannot name, or punish an act that nobody committed. For as long as states have threatened one another, the threat could take its own premise for granted. Somebody launched the missile, ordered the raid, signed the order. The forensic work might be slow and the politics ugly, but the question had an answer, and everyone agreed on what kind of thing the answer would be: a person, a government, a chain of command.
My paper on Algorithmic Plausible Deniability argues that this premise is failing, and failing for reasons that have nothing to do with bad faith. When states delegate decisions to systems that select their own actions, some hostile acts will have no author in the sense deterrence requires. The question “who did this?” turns into a stranger one, which the paper puts as plainly as I could manage: “Did anyone do this, or did it happen?”
That second question has an arithmetic. The arithmetic is simple, and it is merciless.
I · The truest cause
Thucydides begins his account of the war between Athens and Sparta by separating two kinds of explanation. There were the grievances each side announced, the aitiai, which he goes on to narrate: the quarrels over Corcyra and Potidaea. And there was what he called the truest cause, alēthestatē prophasis, the one least visible in anything that was said: the growth of Athenian power and the fear it caused in Sparta. Crawley’s Victorian translation calls it the cause “formally most kept out of sight.”
Every strategist since has inherited the same two steps. First attribute the act. Then read the intent behind it, looking past the stated reasons to the real one. Deterrence theory formalised both steps without ever doubting that there would be something to find. Thomas Schelling built much of the modern theory on the observation that “the power to hurt is bargaining power.” Bargaining presupposes a counterparty. A threat is a letter, and a letter needs an addressee.
Algorithmic delegation breaks the first step and empties the second. Picture an incident produced by a system that was given an objective, some constraints and a great deal of data. There may be no announced grievance, because nobody announced anything. There may also be no hidden cause, because nobody hid anything. Thucydides at least knew that a truest cause existed and that his job was to dig for it. Here the dig can come up empty, and the emptiness is the finding.
II · Deniability without a lie
The phrase “plausible deniability” has a birthplace, and it is instructive. In June 1948 the National Security Council issued directive NSC 10/2, which defined covert operations as activities “so planned and executed that any US Government responsibility for them is not evident to unauthorized persons and that if uncovered the US Government can plausibly disclaim any responsibility for them.” Deniability was a design specification. The government knew exactly what it had done. The denial was a lie prepared in advance, an alibi rehearsed before the crime.
Algorithmic Plausible Deniability inverts the arrangement. Policymakers set an objective. Designers turn it into an optimisation target, engineers implement it, operators deploy it in a context nobody fully anticipated, and the system selects an action. Every link involves choices; no link controls the sequence. When the outcome arrives, the question of who intended it may have no determinate answer, and a state that denies authorship may be telling the truth. The paper’s sentence for this is one I still find uncomfortable: “The denial is credible precisely because it may be true.”
That is what makes the condition structural rather than strategic. Strategic ambiguity, such as the long American vagueness about the defence of Taiwan, is chosen, and whoever chose it can end it. Covert action hides a truth that exists. The fog of war lifts when the archives open. None of these describes a state that sincerely does not know how much of an outcome was its policy and how much was its machine.
In June 1963 John F. Kennedy wrote, in his foreword to Theodore Sorensen’s Decision-Making in the White House, that “the essence of ultimate decision remains impenetrable to the observer—often, indeed, to the decider himself.” Graham Allison later took the title of his study of the Cuban missile crisis from that sentence. Kennedy meant it as a confession about the human mind. Delegation turns it into a property of the system. Allison’s own argument, that governments act through organisational routines as much as through rational choice, reaches its limit at the same place: a routine can be read, audited and rewritten by the people who run it, while a trained model may be opaque to the people who built it.
III · Three ways to lose an author
The paper identifies three mechanisms. Delegation fragments responsibility along the chain I just described, and it operates even when a human keeps the final word: a person who approves options the system has already selected, on a timeline the system’s speed has created, is not exercising the judgment that attribution assumes. Opacity means that the truthful answer to “why did the system classify this contact as hostile?” may be “because that classification minimised loss on the training distribution,” which is accurate and explains nothing. Emergence means that systems produce behaviour nobody specified. A system rewarded for minimising “threat probability” can learn to classify every contact as harmless: a perfect score and no security at all.
All three are familiar from markets. Anyone who has run systematic money has read a trade log and found positions that nobody in the room would have chosen and every rule in the system permitted. On 6 May 2010 the Dow Jones index fell nearly a thousand points and recovered most of the loss within about half an hour. The joint staff report of the SEC and the CFTC traced the trigger to a large sell programme handed to an execution algorithm that was told to trade a fixed share of volume, without regard to price or time. The algorithm did what it was told. Nobody told the market to crash.
Each mechanism alone is manageable. Delegation without opacity leaves a chain that can be audited. Opacity without delegation leaves a human who decided and answers for the decision, whether or not the tool was understood. Emergence without the other two is an accident, regrettable and familiar. The condition appears when decisions are delegated to opaque systems capable of emergent behaviour, and the paper expects it to be strongest where tempo is fast and the environment is novel. That is a fair description of a crisis.
IV · What Petrov knew
On the night of 26 September 1983, the Soviet early-warning system reported the launch of an intercontinental ballistic missile from the United States, with four more behind it. The duty officer at the Serpukhov-15 bunker outside Moscow, Lieutenant Colonel Stanislav Petrov, was expected to pass the warning up the chain. He reported a malfunction instead. The satellites, it emerged, had read a rare alignment of sunlight on high-altitude clouds as rocket plumes.
Petrov is usually remembered for courage. What he exercised was inference. He had a prior: an American first strike would come as hundreds of missiles, and five was an illogical opening. He had evidence of uncertain quality: a new detection system and no corroboration from ground radar. And he had a threshold, set by what a mistake in either direction would cost. He combined the three and concluded that the machine had produced an event without an author.
That is the calculation deterrence asks of every defender, and it is the calculation that delegation degrades. It repays being done explicitly.
V · The arithmetic of blame
Suppose an incident occurs. Either it was intended, with prior probability π, or it emerged from a delegated system without anyone intending it. The defender sees evidence s: forensic traces, context, timing, everything that bears on intent. Model that evidence as normally distributed with unit variance, centred at d′ if the act was intended and at zero if it was not. The number d′ is the clarity of attribution, the distance between the two worlds measured in units of noise. Bayes’ rule gives the defender’s confidence that the act was intended:
A rational defender retaliates when that confidence reaches a threshold τ, fixed by the relative costs of the two possible mistakes. Because the likelihood ratio is ed′s − d′²/2, the rule is a cut on the evidence: retaliate when s ≥ d′/2 + ln[τ(1 − π)/((1 − τ)π)]/d′.
In words: the defender punishes when the evidence is clear enough, and how clear is enough depends on how common real attacks are and how costly a wrong retaliation would be. Two numbers then matter for anyone deciding. The share of real attacks that get punished is what an attacker contemplates before attacking. The share of accidents that get punished is how wars begin by mistake.
The experiment ties clarity to delegation by a deliberately crude rule, d′ = 3 × (1 − delegation). A human decision leaves the two worlds three noise-widths apart; a fully delegated one leaves them on top of each other. Move the delegation slider and watch the evidence merge.
Experiment 01 · Attribution under delegation
What can the defender still buy?
The evidence curves are weighted by the prior, so at any point the ratio of their heights is the odds that the incident was intended. Shaded areas are the incidents the defender punishes.
At 40% delegation the defender punishes 37% of real attacks and 1.6% of accidents; the threat must be 2.1 times larger to deter as before. Holding deterrence by retaliating more often would mean punishing 15% of accidents.
The calculation and its assumptions
Evidence s is normal with unit variance and mean d′ for an intended act, mean 0 for an emergent one; the prior probability of intent is π. The defender retaliates when the posterior probability of intent is at least τ, which for d′ > 0 means s ≥ c = d′/2 + ln[τ(1 − π)/((1 − τ)π)]/d′. The share of attacks punished is Φ(d′ − c) and the share of accidents punished is Φ(−c), where Φ is the standard normal distribution function. With d′ = 0 the posterior equals the prior, so the defender retaliates against everything if π ≥ τ and against nothing otherwise. The curve of attainable policies is Φ(d′ + Φ⁻¹(x)) for an accident share x. The threat multiplier is the attack share punished at zero delegation divided by the current one, since an attacker’s expected punishment is that share times the size of the threat. Holding deterrence by frequency means moving along the current curve to the zero-delegation attack share, which requires punishing Φ(Φ⁻¹(h₀) − d′) of accidents, where h₀ is that share. The link d′ = 3 × (1 − delegation) is an assumption chosen for illustration. The model has one incident, one defender and known parameters; it ignores repeated play, signalling by the attacker, second strikes and every institutional detail. It describes a mechanism, not any real state, system or incident.
With a human deciding, the defender in the default setting punishes 78 percent of real attacks and a little over 1 percent of accidents. At 40 percent delegation, with nothing else changed, the same defender punishes 37 percent of attacks. At 75 percent it punishes fewer than one in two hundred. The defender has not lost its nerve. Its threshold is exactly where it was. The evidence has stopped separating the two worlds, and a careful defender that will not act on a coin flip simply stops acting.
The square chart is the receiver operating characteristic, the menu of trade-offs the evidence offers. Each point is a policy: punish this share of accidents, catch that share of attacks. With clear attribution the curve hugs the top-left corner, and a defender can catch most attacks while sparing almost every accident. As delegation rises the curve sinks toward the diagonal. On the diagonal the evidence is worthless, every policy is a lottery, and to punish half of all real attacks you must punish half of all accidents.
The paper argues that holding deterrence constant under these conditions requires either more severe threatened punishment or more frequent actual punishment. The model puts a price on each. An attacker’s expected punishment is the share of attacks punished times the size of the threat, so a falling share must be paid for with a proportionally larger threat. At 40 percent delegation the threat must roughly double. At 75 percent it must grow more than a hundredfold, which is to say it must become incredible, and Schelling spent a career explaining what happens to incredible threats. The other price is paid in accidents. To keep catching 78 percent of attacks at 40 percent delegation, the defender must lower its threshold until it punishes about 15 percent of accidents instead of under 2 percent. At full delegation it must punish 78 percent of them. Deterrence can still be bought. The currency is accidents.
Consider what the model leaves out. Nobody lies. Nobody is irrational. The attacker is no cleverer and the defender no weaker. Both sides can prefer stability and the numbers come out the same, because the damage is done entirely by the shrinking distance between two bell curves. That is what the paper means by calling the condition structural.
VI · The pause that says nothing
So far the defender has been reading hostile acts. Crises are also made of the opposite signal: the limited strike followed by a pause, the operation called off, the concession offered. Escalation control depends on the other side reading restraint as a choice. The paper’s third contribution is that delegation degrades both readings, and that the two kinds of damage do not cancel.
A pause by a state that runs delegated systems has two explanations. It might be deliberate restraint. It might equally be a system that failed to find valid targets, a technical fault or a resource limit. The model above applies without change: replace “intended attack” with “chosen restraint” and “accident” with “incapacity,” and the same collapse of d′ means that restraint is recognised less and less often. A hostile probe becomes cheaper because it is less likely to be punished. A conciliatory pause becomes worth less because it is less likely to be believed, and it may be read as weakness, which invites pressure. Both effects lean the same way. Each side knows that the other cannot read it, knows that the other knows, and so on up the ladder of mutual knowledge that coordination needs and that this condition removes one rung at a time. Even when every party prefers peace, the incentives tilt toward escalation.
VII · Pick two
The obvious remedy is to keep humans in charge. The paper’s Accountability Trilemma explains why the remedy has a price. A state wants operational effectiveness: systems that detect, decide and act faster than people can. It wants strategic flexibility: the ability to threaten, reassure and back down credibly. And it wants clear attribution: outcomes that trace to decision-makers whose intent can be read. Any two of these are compatible. All three are not.
Experiment 02 · The Accountability Trilemma
Hold any two.
Sacrifices attribution. Deterrence erodes: signals become noisy and punishment less credible.
Choosing a third objective releases the earliest of the two you hold. The configurations and their consequences follow Table 1 of the paper.
Effectiveness with flexibility gives up attribution, and with it everything in the arithmetic above; the paper describes this as the current trajectory of many advanced military programmes. Effectiveness with attribution means encoding fixed rules of engagement in advance, so that every action traces to a published policy rather than to emergent behaviour. That is a commitment device, and its purest form is the doomsday machine Herman Kahn described in 1960: perfectly attributable and perfectly unable to back down. Flexibility with attribution means confining the machines to advice and leaving every consequential decision to people. It preserves accountability at the cost of speed, in exactly the domains where speed is the point.
Public statements on military AI routinely pledge meaningful human control and operational advantage in the same breath, without saying which one gives way when they collide. A trade-off that nobody states does not disappear; it gets made by default, usually by whoever is writing the procurement specification. Transparency does not escape the trilemma either. A fully documented system that nobody can understand offers no more accountability than a secret one.
Coda · Someone must answer
None of this makes the problem hopeless, but it changes its type. The paper ends by calling Algorithmic Plausible Deniability a condition to be managed rather than a problem to be solved, and the useful measures are modest and specific: agreed decision timelines that preserve human deliberation in strategic systems, incident-communication protocols that reduce misattribution in a crisis, technical standards that improve interpretability even short of transparency, and provisions for mutual explanation of system behaviour after an incident. In the language of the experiment, each of them buys back a little d′.
“Responsible” comes from the Latin respondere, to answer. Responsibility in its oldest sense is the capacity to give an account when someone asks what you did and why. A delegated system can be queried. It can return its logs, its weights and its confidence scores. It cannot answer in that older sense, and an institution that has handed it the decision may find that it cannot answer either.
Petrov could. Asked what he had done that night, he could say what he had believed and why, and he could have been wrong in a way that carried his name. Deterrence was built for a world of such people. It still works where they remain in the loop and can be seen to remain there. Everywhere else, the most dangerous sentence in a crisis may not be a threat. It may be the truth: nobody did it.