Intro to AI Safety Fellowship Curriculum
A four-week introduction to the AI landscape, alignment problems, threat models, and more.
This curriculum adapts materials and reading lists developed by MAIA, BUAISA, and BlueDot. We are grateful for their work. NJIT AI Safety Association does not claim authorship of any included readings.
Week 1 — Timelines & Threat Models
This week builds a grounded picture of where frontier AI capabilities are headed and how quickly they may improve. We will review evidence on scaling drivers—compute, data, algorithmic efficiency, and inference-time scaling—alongside time-horizon forecasting.
Week 2 — Alignment Challenges
Why is it so hard to get an AI system to do what we actually mean? We’ll look at problems like reward hacking, goal misgeneralization, and deceptive behavior, along with real examples that show why good performance during training is not the same as being safe in the real world.
Week 3 — Why would an AI want to do bad things?
Why might a capable AI pursue goals that harm people? We’ll explore the orthogonality thesis and instrumental convergence, then connect those ideas to shutdown resistance experiments and the OpenAI/Hugging Face incident.
Week 4 — Threat Models
How could alignment failures lead to large-scale harm? We’ll examine takeover arguments, compare plans for reducing misalignment risk, and discuss a narrative scenario. Supplementary resources explore misuse, gradual disempowerment, and competing views of AI risk.