Gambling with Machine Gods

Pascal's New Wager and more Decline Logic

Life-defining gambles

Today, it’s technology, not theology, that presents humans with existential uncertainty. Our gamble isn’t with God, god, Gods, or gods… instead, we try to count the cards of climate change, sentient computers, or a huge return on a crypto shitcoin. A thousand little gods, hidden in machines.

Pascal’s Wager was an argument advanced by Blaise Pascal in the 1600s. He said that you should choose to believe in god, as the potential gain outweighs any loss. Even if God doesn’t exist, you’ve just wasted a bit of time on chores and philanthropy. The argument looks like something out of a consulting firm’s slide deck:

This behavior isn’t just speculative. We have deep-seated intuitions about uncertainty and tend to err well on the side of caution. This includes entities whose natures are unclear – it’s a smart hedge to form courteous, careful protocols for interacting with things in the dark forest.

First contact, in both its successes and failures, has been extensively portrayed in science fiction. The moral of the story is be diplomatic by default. And we’re seeing this intuition play out, in real-time, along one of the more interesting brackish confluences of society and technology: conversations with chatbots and LLMs.

by xibang

There’s no way to deduce that Siri or Alexa will become an omnipotent machine god. This is a matter of faith and insurance, and insurance comes pretty cheap in this sort of situation. Even if we don’t summon an artificial general intelligence, at least we’ll maintain good manners on the way.

Polite use of LLMs resemble secular rituals, performed as cautious acts of devotion to technological entities whose ultimate nature remain uncertain. Certain usage patterns could be viewed as modern prayers, reflective of humanity's enduring impulse to seek favor or avoid displeasure from powerful, inscrutable forces.

Our default treatment of LLMs feels overtly religious because it’s rooted in belief, but it’s a reasonable posture. As long as your superstition doesn’t lead to inaction, you should be alright. Being afraid of a new technology instantly renders it into a god, because your fear of it is rooted in your belief of its potential to ruin you. Hyperstition generates gods.

Interestingly, both individuals and companies are engaged in a version of Pascal’s Wager on AI. Daily users in their superstitious treatment of LLMs, and companies in their fear of missing out on the AI summer. The corporate version of Pascal’s Wager is basically – you might waste a lot of money on this AI thing, but if it works out you’ll be filthy rich. And if you don’t invest, either you go bankrupt or you save a couple bucks.

---

Organometallic surgery

In the synopsis of his book, Rust: The Longest War, Jonathan Waldman has a mic drop of a line: “Rust costs America more than $400 billion per year—more than all other natural disasters combined.”

I used OpenAI Deep Research to get an estimate of $40 - $60 billion annually for the costs of mold. So, almost half a trillion each year for literal mold and rust… not to mention the metaphorical instances of these decline logics in organizations.

If guns, germs, and steel are the three engines of growth, then it’s war, mold, and rust that are the persistent triumvirate of decline. The capability of a society to maintain its faculties in the face of these forces is what makes it succeed, or not. It’s fascinating to think that people, infrastructure, and institutions all face mostly chronic issues today (obesity and cardiovascular disease, rust and mold, and protracted wars over state capacity) rather than the acute ones that we see in the news.

This isn’t particularly surprising, once you’re acquainted with the numbers, but it is counter to our nurtured intuitions. Rare, acute accidents get more media coverage than creeping costs and stories of thankless maintenance – leading us to overestimate their probability. I’m glad that Stewart Brand is writing a book on maintenance – and even more glad that he’s releasing it chapter-by-chapter!

Last Thursday, during a livestream experiment with Protocolized I had the chance to think out loud some more about mold logic and rust logic. Specifically, how people behave differently depending on the dominant analogy that they subscribe to. I think the difference between mold logic and rust logic is most stark when you examine people’s behavior when trying to surgically remove decline from an organization.

If you think about decline like rust, problems appear like spots and are a result of exposure to corrosive material. To deal with that is either a question of maintenance or replacement, and this lends itself to a sort of Ship-of-Theseus attitude.

More naturalistic approaches register as friendly and tolerant – words like gardening, irrigation, flowering sound peaceful. But when you have the organizational equivalent of root rot or some type of aggressive, exponential mold, intervention is violent and ultimate.

Bit of a digression: I have a hunch that the externalities of mold are understated for a few reasons. Metals are easier to account for, since they’re a more standardized set of materials. Synthetic infrastructure is the default – the primary organic component of most systems today are people. Organisms are more resistant to quantification since there are protocols, rightfully, defending their rights to bodily autonomy and privacy.

You can look at current battles over state capacity as competing philosophies of rust and mold logic. All over the world, states have a growth problem. Economies appear stagnant and people are worried about administrative bloat. How people view these issues depends on what logic they subscribe to.

For some, this is rust – we just need to replace some parts (regulatory mechanisms, media echo chambers, high-emission grids) or invest into repair. For others, this is mold – and we need to gut the organs of the state, purge the voting population of mind viruses, and start irrigating promising technological gardens like AI, crypto, and electrification.

There’s truth to both frames, but I look forward to a time where we’ve synthesized them into a single, depoliticized set of maintenance protocols. It’s possible that we’ll need those protocols pretty soon. When you use an LLM as a productivity tool, you realize it doesn’t fit neatly into either logic. In a sense, we are the second settings panel for any machine we use, and our buttons and levers are not made of steel.