Do LLMs Really Understand? Examining Geoffrey Hinton's Arguments

Geoffrey Hinton thinks today's AI systems understand us. I'm not so sure.

In a recent interview with Alex Kantrowitz, Hinton gives two arguments for the claim that large language models really do understand. In this video, I take both arguments seriously and ask whether they actually establish what Hinton thinks they establish.

The first argument is roughly: if LLMs can consistently give us the right answers, then they must understand. The second points to an especially interesting phenomenon: when a chatbot gives a wrong answer, gets corrected, and responds by saying "I misunderstood," perhaps the best explanation is that it really did understand.

I think both arguments have some real philosophical structure to them—but I don't think they get us all the way to understanding.

Along the way, I discuss:

• The difference between understanding language and understanding the world
• Why "getting the right answer" may not require understanding
• The Chinese Room and the example of calculators
• Why selection bias makes LLM accuracy difficult to assess
• Why saying "I misunderstood" isn't necessarily evidence of understanding
• What would actually convince us that an opaque AI system contains a mind

My current view is that LLMs seem to have something like an understanding of language use, but that's not obviously the same thing as understanding the things that language represents.

Hinton may be right. But I think we need better evidence.