Episode 12 · · 1h 56m
Debunking AI Doom: Why Existential Risk Arguments Fall Apart
Play Episode 12 on YouTube
Is artificial intelligence really on track to wipe out humanity by 2050, or are the most prominent doom arguments fundamentally flawed?
In this episode of Philosophy Programs and Prompts, Casey Hart(@OntologyExplained) and Carl Brown(@InternetOfBugs) break down the core premises behind AI existential risk (x-risk). From Bostrom-style superintelligence scenarios and recursive self-improvement (RSI) to "hockey stick" benchmark progress and the appeal to expert anxiety, Casey and Carl analyze where the logic holds up—and where it jumps the rails.
More importantly, they discuss why hyper-focusing on apocalyptic sci-fi scenarios distracts the public and regulators from the very real, immediate harms AI systems are causing right now: cyber vulnerabilities, military reliance on hallucinated outputs, non-consensual deepfakes, and corporate regulatory capture.
Chapters
- 0:00Real AI Harms vs. Extinction Hysteria
- 0:40Casey Hart & Carl Brown: "Prominent X-Risk Denialist"
- 3:04Deconstructing the Doom Debate
- 4:47Setting the Terms: What Does "Doom" Actually Mean?
- 7:18Bayesian Thinking & The Base Rate Fallacy
- 12:06Where Do p(doom) Percentages Come From?
- 15:01The Core Doom Argument Broken Down
- 20:47Premise 1: Is Intelligence Sufficient for Domination?
- 22:36Did AI "Almost Start a War"? Misleading Headlines
- 26:06Hanlon's Razor & Intentionality in LLMs
- 29:25Premise 2: Limits to Superhuman Intelligence
- 34:07Premises 3–5: Desires, Alignment, and Losing a Conflict
- 49:33The Math of Multiplying Uncertain Premises
- 53:31The Argument from Exponential Progress
- 57:46Reasoning Models, Wrappers, and Stacking Advances
- 59:40Benchmark Contamination & Overstated Gains
- 1:04:03Real-World Coding Regressions (Fighting with Models)
- 1:08:56Appeal to Expert Anxiety & Regulatory Moats
- 1:16:07Assessing Authority vs. Vested Commercial Interests
- 1:22:45The Argument from Unpreparedness & Cybersecurity Debt
- 1:29:13The "Big Impacts" Fallacy & Infinite Disutility
- 1:34:42What Does Recursive Self-Improvement (RSI) Actually Mean?
- 1:41:08Anthropic's RSI Evidence: Checking in Lines of Code
- 1:46:00Asymptotes, Data Bottlenecks, and Real-World Friction
- 1:53:34Why Focusing on Fiction Harms Real-World Victims
- 1:55:25Closing Thoughts: Stay Calm and Focus on Real Problems
Sources
- AI almost started a war with China (Military Watch Magazine)
- Gates sees end to spam in 2 years
- HACKED! Are Commercial Ships Being Targeted by Cyber Attacks? | Maritime Cyber Security Deep Dive
- New Apple study challenges whether AI models truly "reason" through problems
- Recursive self-improvement (Anthropic)
- The Illusion of Thinking
- Thread on ties between AI doomers, rationalists and AI employees
- Will AI Kill Everyone by 2050? Debate with Dr. Casey Hart (Ontology Explained)