Can We Control Superintelligence? | Analyzing Cal Newport's Arguments

What does the Hugging Face hack tell us about AI alignment?

The mainstream story seems to be that AI systems are becoming increasingly autonomous, unpredictable, and capable of "breaking containment." But Cal Newport offers a very different way of looking at the problem: we already have many examples of superhuman AI systems that are remarkably well-controlled. Maybe the problem isn't superhuman AI itself, but the way we're designing certain AI agents.

In this video, I reconstruct and analyze an argument Cal made on Ed Zitron's Better Offline podcast. Along the way, we'll look at two related arguments:

1. We already have examples of well-controlled superhuman AI, so perhaps superhuman AI isn't inherently difficult to control.

2. The dangerous long-horizon LLM hacking agents are different because they're allowed to run in loops without meaningful human oversight.

I'll examine the logic of these arguments, including the distinction between inductive and deductive reasoning, inference to the best explanation, and the tricky question of what exactly counts as "superhuman" and "well-controlled."

I think Cal's arguments have real merit—but I'm not convinced they're strong enough to settle the disagreement with AI doomers. In particular, the examples of AlphaFold, Cicero, self-driving systems, and even calculators may not address the kind of general intelligence that AI safety researchers are actually worried about.

And there's an interesting possibility here: perhaps the best way to think about AI risk isn't through abstract speculation about future systems, but by carefully studying what existing AI systems can and can't do.

This is another exercise in spotting an argument "in the wild" and reconstructing it—a skill that I think is useful far beyond AI.