I'm Sick of the ASI Fear-Mongering (Hank Green's Video Made Me Rage, Featuring Nate Soares)

I watched a video with Hank Green and Nate Soares discussing the existential threat of Artificial Super Intelligence (ASI), and honestly, I was infuriated. Soares, co-author of If Anyone Builds It, Everyone Dies, argues that if an ASI is not perfectly "aligned" with human values, it will surely lead to our doom. But as I break down their major points—from their vague, fear-inducing definition of ASI to their outlandish claims about current AI capabilities—it seems like the real problem isn't a malicious super-intelligence, but rather the unnecessary mystery and power concentration surrounding the technology.

This video is my detailed objection to the "stop all AI" argument. I challenge the notion that LLMs are plotting, self-aware, or "caring" in a way that poses an existential threat. The metaphors they use, like comparing AI development to alchemy, only serve to obscure the technology and increase the power of the builders (the "alchemists"). Ultimately, the true "alignment problem" isn't with an uncaring algorithm, but with the huge corporations and influential figures who are shaping this powerful technology. I propose four foundational principles for thinking about AI that prioritize clarity, human flourishing, and a healthy distrust of the powerful.