Does AI Really Want Anything? | If Anyone Builds It... Chapter 3

Do AI systems actually want things?

In this chapter of Yudkowsky and Soares' If Anyone Builds It, Everyone Dies, the authors argue that sufficiently capable AIs will begin to behave as though they have preferences and wants. They appeal to evolution, gradient descent, strategic behavior, and game theory to make their case.

I’m not convinced.

In this video, I examine their argument about AI wanting things, starting with a seemingly simple question: What does it mean to want something? Does being shaped to achieve a particular outcome really amount to wanting that outcome? And if it does, do marbles, blankets, sorting algorithms, and paramecia have wants too?

I also look at their examples of AI behavior, including the famous story of o1 "going hard," and ask whether these examples actually provide evidence for desires and mental states—or whether we're simply using metaphorical language to describe sophisticated problem-solving.

Finally, I look at the chapter's discussion of game theory and its broader conclusion: that AI will be difficult to get to behave exactly as we want.

That last point? I agree with it.

But I'm not sure we needed all the talk about AI wanting things to get there.

This is part of my ongoing series examining If Anyone Builds It, Everyone Dies from the perspective of analytic philosophy, ontology, and AI.