Week 3 — Why would an AI want to do bad things?
Why might a capable AI pursue goals that harm people? We’ll explore the orthogonality thesis and instrumental convergence, then connect those ideas to shutdown resistance experiments and the OpenAI/Hugging Face incident.
Open the Week 3 curriculum in Google Docs ↗
Core resources
- What is the orthogonality thesis? (AISafety.info) ↗
- Why Would AI Want to do Bad Things? Instrumental Convergence (Robert Miles, 2018) — video ↗
- But why would the AI kill us? (Nate Soares, 2023) ↗
- Shutdown Resistance in Large Language Models (Palisade Research, 2025) ↗
- The OpenAI/Hugging Face attack, clearly explained (Dwarkesh Patel, 2026) — video ↗
Recommended resources
- Agentic Misalignment: How LLMs Could Be Insider Threats (Anthropic, 2025) ↗
- Frontier Models are Capable of In-Context Scheming (Apollo Research, 2024) ↗
- AI "Stop Button" Problem (Robert Miles / Computerphile, 2017) — video ↗
- The Basic AI Drives (Steve Omohundro, 2008) ↗
- Orthogonality is expensive (Beren Millidge, 2023) — critical perspective ↗