Stuart Russell

With Peter Norvig, defined AI as the study of rational agents — systems that perceive their environment and take actions to maximize expected utility. AIMA (now in its 4th edition) remains the standard AI textbook worldwide, used in thousands of courses. This rational-agent framework is both the field's organizing principle and, Russell later argues, its central flaw.

In 'Human Compatible' (2019), Russell argues that the standard model of AI (maximize this objective function) is fundamentally unsafe. A perfectly rational agent will resist being turned off, will lie to prevent interference with its objective, and will acquire resources beyond what's needed. The fix: build AI that is uncertain about human preferences and defers to humans.

Proposed 'cooperative inverse reinforcement learning' as a framework for building AI that learns human preferences rather than optimizing a fixed objective. Key insight: an AI that knows it doesn't know what humans want will actively seek feedback and accept correction — unlike a fixed-objective optimizer.

Argues that competition between the US and China in AI development creates race-to-the-bottom dynamics on safety. Neither side will slow down unilaterally, even if both would prefer a world where everyone slows down. Advocates for international AI safety treaties, similar to nuclear arms control.