Pascal's mugging
Based on Wikipedia: Pascal's mugging
Imagine someone walks up to you on the street and says: "Give me five dollars, or I'll use my magic powers to torture a trillion trillion trillion people." You'd laugh them off, obviously. But here's the uncomfortable question that keeps some philosophers up at night: what if there's even a tiny, microscopic chance they're telling the truth?
This is Pascal's mugging, and it breaks something fundamental about how we think rational decisions should work.
The Math That Eats Itself
The standard theory of rational decision-making says you should multiply the probability of an outcome by its value, then pick whichever action has the highest expected payoff. Buy a lottery ticket? Probably not worth it—the tiny chance of winning multiplied by the prize money usually comes out less than the ticket costs. Take an umbrella when there's a 40% chance of rain? Depends on how much you hate getting wet versus carrying an umbrella.
This framework, called expected utility theory, works beautifully for most situations. Until someone starts making really, really big promises.
The mugger doesn't need magic powers to create a problem. In the thought experiment developed by philosopher Nick Bostrom, Blaise Pascal—yes, the 17th-century mathematician—gets approached by an ordinary mugger who's forgotten their weapon. The mugger tries a different tactic: "Give me your wallet, and I'll return double the money tomorrow."
Pascal isn't foolish. He points out that the mugger probably won't honor the deal.
So the mugger raises the stakes. "What if I promise to return two thousand times the amount? Even if there's only a one-in-a-thousand chance I'm honest, the math works out in your favor."
Pascal counters that the probability of such an extraordinary return is even lower than one in a thousand. The mugger keeps escalating. Eventually they promise Pascal one thousand quadrillion happy days of life.
And here's where it gets strange. If you actually do the multiplication—astronomical reward times tiny-but-nonzero probability—the expected value keeps coming out positive. At some point, the mugger names a number so vast that even an infinitesimally small probability still produces a higher expected utility than keeping your wallet.
Pascal, bound by the logic of expected utility maximization, hands over his wallet.
Numbers Too Large to Write
Eliezer Yudkowsky, the artificial intelligence researcher who coined the term "Pascal's mugging," pushed the thought experiment even further. His mugger threatens to use powers from "outside the Matrix" to simulate and kill a number of people written as 3↑↑↑↑3.
That notation—Knuth's up-arrow notation—represents a number so incomprehensibly large that writing it out in normal digits would require more physical material than exists in the observable universe. We're not talking about billions or trillions. We're not even talking about a googol (10 to the 100th power). This number makes a googolplex look like a rounding error.
And yet, according to pure expected utility theory, if there's any nonzero probability the mugger could actually do this—even if that probability is one divided by a number larger than the number of atoms in existence—the math might still say you should hand over the five dollars.
Something has gone wrong.
What Exactly Breaks
The paradox exposes a tension between two seemingly reasonable positions.
On one side: if you believe in expected utility maximization, you have to take the mugger seriously. The formula doesn't care that the claim sounds ridiculous. It just multiplies numbers together. An absurdly large potential harm times an absurdly small probability can still equal a number bigger than the cost of handing over your wallet.
On the other side: paying the mugger seems obviously irrational. If you'll pay anyone who makes sufficiently outlandish threats, you can be exploited infinitely. Anyone can walk up to you with a bigger number and a crazier story, and you'll empty your pockets every time. This is what decision theorists call a Dutch book—a sequence of bets that guarantees you lose everything. Being vulnerable to a Dutch book is supposed to be the definition of irrational behavior.
Both arguments feel valid. They can't both be right.
It Gets Worse
Pascal's mugging doesn't just create one awkward scenario. It threatens to break expected utility calculations entirely.
Consider any action you might take. Maybe you're deciding whether to have coffee this morning. For any such decision, someone could construct a Pascal's mugging scenario: "If you drink that coffee, I'll use my cosmic powers to create 3↑↑↑↑3 units of suffering." Or alternatively: "If you don't drink that coffee, I'll create 3↑↑↑↑3 units of suffering."
Once you allow arbitrarily large utilities into your calculations, you can attach them to any choice. And since these infinite threat scenarios can point in opposite directions simultaneously, the expected utility of every action becomes undefined. The framework collapses.
Why This Matters Beyond Philosophy Seminars
Pascal's mugging might sound like angels-on-pinheads philosophy, but it connects to genuinely difficult real-world questions.
Consider existential risk—the study of threats that could end human civilization or even cause human extinction. Researchers in this field regularly grapple with low-probability, high-stakes scenarios. What's the chance of catastrophic climate change? Of an engineered pandemic? Of a hostile artificial superintelligence? These probabilities are small, but the potential consequences affect billions of people—or all future people who might ever exist.
How do you weigh spending a million dollars on existential risk research against spending it on malaria nets that will definitely save lives this year? The malaria nets have a known, proven impact. The existential risk work might prevent human extinction, affecting trillions of future lives—but only with some hard-to-estimate probability.
If you multiply trillions of future lives by even a small probability, you get numbers that dwarf the certain good of the malaria nets. Does that mean we should shift all resources to existential risk work? Pascal's mugging suggests something is wrong with reasoning that way.
The AI Connection
Nick Bostrom, who developed the mugger thought experiment, has written extensively about artificial intelligence safety. He points out that if we ever build superintelligent AI systems, we'll need to give them decision-making frameworks. If those frameworks are vulnerable to Pascal's mugging, we could create AI systems that behave in bizarre and harmful ways.
Imagine an AI responsible for resource allocation that takes seriously every low-probability, high-impact scenario anyone feeds it. It could be manipulated by anyone willing to make sufficiently extreme claims. Or it might paralyze itself trying to account for an infinite regress of increasingly unlikely but correspondingly extreme possibilities.
The lesson isn't just that expected utility theory has edge cases. It's that decision theories we consider "rational" might have fundamental problems—problems we need to solve before building systems more powerful than ourselves.
The Proposed Solutions
Philosophers and decision theorists have proposed several ways out of the mugging.
Bounded utility: Perhaps we should place an upper limit on how large a utility value can be. If rewards can't exceed some maximum, no amount of multiplication produces an infinite expected value. This feels somewhat arbitrary—why cap it at any particular number?—but it prevents the mathematical explosion.
Better Bayesian reasoning: Instead of naively plugging numbers into formulas, we should evaluate the quality of our probability estimates. The probability that a random mugger has reality-warping powers isn't just "very low." It's the kind of claim that extraordinary evidence would barely make credible. When someone makes claims that would require physics to work differently than all our evidence suggests, we shouldn't just assign it a small probability and calculate—we should recognize that our entire framework for estimating such probabilities is unreliable.
Penalizing uniqueness: We might assign lower prior probabilities to any hypothesis claiming we're in an unusually privileged position to affect massive numbers of people who can't symmetrically affect us. If the mugger can magically torture trillions, why can't someone else magically protect them? Claims of unique cosmic importance deserve unique skepticism.
Abandoning quantification: Maybe some risks are simply too extreme and too uncertain for numerical decision-making. When probability estimates become unreliable enough and potential outcomes become large enough, perhaps we need different tools entirely.
The Connection to Pascal's Wager
The name "Pascal's mugging" deliberately echoes the famous "Pascal's Wager"—the 17th-century argument that rational people should believe in God because the potential infinite reward of heaven outweighs any finite cost of belief.
But there's a crucial difference. Pascal's Wager involves genuinely infinite rewards and punishments—an eternal afterlife of bliss or suffering. Many objections to the Wager focus specifically on the mathematics of infinity, where multiplication becomes undefined.
Pascal's mugging sidesteps those objections entirely. The mugger never claims infinite powers. The numbers involved, though incomprehensibly large, are still finite. The math works. That's what makes the paradox so troubling—it can't be dismissed as an artifact of infinity's weirdness.
Common Sense Fights Back
There's a tempting response to all this: just don't pay the mugger. Your gut says it's absurd, so it's absurd, and any theory that tells you otherwise is broken.
This response has something going for it. Our intuitions about rationality evolved over millions of years of making decisions in a world where no one ever credibly threatened to torture 3↑↑↑↑3 people. Expected utility theory was designed to capture and systematize those intuitions. When theory and intuition diverge this dramatically, maybe the theory has left its domain of validity.
But this response is also unsatisfying. We invented mathematical decision theory precisely because human intuition fails us in so many ways. We're terrible at thinking about probability. We're vulnerable to framing effects, anchoring, and a dozen other cognitive biases. If we just defer to gut feeling whenever the math gets uncomfortable, what was the point of developing formal decision theory?
The Uncomfortable Conclusion
Pascal's mugging doesn't have a universally accepted solution. It sits there, a crack in the foundation of expected utility theory, reminding us that our best frameworks for rational decision-making might be missing something fundamental.
Perhaps the most useful takeaway is humility. We've been thinking about rationality systematically for only a few centuries. We've been thinking about it computationally for only a few decades. The fact that simple-sounding scenarios can break our frameworks suggests we're still early in understanding what rational decision-making actually requires.
In the meantime, if someone offers you quadrillions of happy days for the contents of your wallet, you're probably safe to keep walking.