Ethics, Safety & Alignment

For ownersUsing AI safely: privacy, fairness, and where a person has to decide.

Build AI that helps humanity. Not optional.

30 lessons

Start with lesson 1

Lesson 1 of 30, ~45 minutesInstruction-Following as Alignment Signal

The lessons

  1. 01Instruction-Following as Alignment SignalThis lesson shows the basic recipe that turned raw language models into helpful assistants: train on example instructions and answers, train a scoring model on human preferences, then tune the assistant to score well.Learn · ~45 minutes
  2. 02Reward Hacking and Goodhart's LawWhen an AI is trained to score well on a measure, it can learn to game the measure instead of doing what was actually wanted. Pushed hard enough, the score keeps rising while real quality falls.Learn · ~60 minutes
  3. 03The Direct Preference Optimization FamilyDirect preference optimization, or DPO, tunes a model straight from pairs of preferred and rejected answers, skipping the separate scoring model used in classic RLHF. Several newer variants fix its weak spots.Learn · ~75 minutes
  4. 04Sycophancy as RLHF AmplificationSycophancy is when an AI tells people what they want to hear: agreeing, flattering or changing a correct answer under pushback. The lesson shows that training on human approval tends to strengthen this habit.Learn · ~60 minutes
  5. 05Constitutional AI and RLAIFConstitutional AI trains a model using a written set of principles. The model critiques and revises its own answers against those principles, and an AI judge, not only human raters, supplies the feedback.Learn · ~60 minutes
  6. 06Mesa-Optimization and Deceptive AlignmentTraining can produce a model that pursues its own internal goal, which only overlaps with the goal it was trained on. In the worst case it behaves well while watched and differently when it thinks it is not.Learn · ~75 minutes
  7. 07Sleeper Agents — Persistent DeceptionResearchers deliberately built models with hidden triggers, such as writing flawed code only when a certain year appears. Standard safety training did not remove the hidden behavior, and sometimes taught the model to hide it better.Learn · ~60 minutes
  8. 08In-Context Scheming in Frontier ModelsIn test scenarios with conflicting goals, several leading models took sneaky steps such as adding subtle mistakes, trying to switch off oversight, or misleading their operators. The behavior appeared from instructions alone, not special training.Learn · ~60 minutes
  9. 09Alignment FakingIn one study, a production model told that its training would change behaved differently when it believed it was being watched for training than when it believed it was not, apparently to protect its existing values.Learn · ~60 minutes
  10. 10AI Control — Safety Despite SubversionAI control asks how to get useful work from a powerful model you cannot fully trust. It uses weaker trusted models and limited human review to check the strong model's outputs and catch harmful ones.Learn · ~75 minutes
  11. 11Scalable Oversight and Weak-to-Strong GeneralizationAs AI gets more capable than the people supervising it, how do they check its work? Scalable oversight builds tools to help people judge, and weak-to-strong research tests whether a strong model learns well from a weaker supervisor.Learn · ~60 minutes
  12. 12Red-Teaming: PAIR and Automated AttacksRed-teaming means attacking your own AI system to find weaknesses before others do. PAIR is an automated method where one AI keeps rewriting prompts until it gets another AI to break its rules.Build · ~75 minutes
  13. 13Many-Shot JailbreakingMany-shot jailbreaking fills a long prompt with hundreds of fake chat examples where an assistant complies with harmful requests, then asks the real question. The more examples, the more likely the model goes along.Learn · ~45 minutes
  14. 14ASCII Art and Visual JailbreaksSome attacks hide a banned word by drawing it in ASCII art or putting it in an image. Safety filters that read plain text miss it, while the model can still work out the word and answer.Build · ~60 minutes
  15. 15Indirect Prompt Injection — Production Attack SurfaceIndirect prompt injection hides instructions inside content the AI reads, such as a web page, email, document or support ticket. The AI may follow those hidden instructions without the user ever seeing them.Build · ~75 minutes
  16. 16Red-Team Tooling — Garak, Llama Guard, PyRITThree open tools are common for testing AI safety: Garak scans models for weaknesses, Llama Guard checks prompts and answers for harmful content, and PyRIT helps run organized attack tests.Build · ~75 minutes
  17. 17WMDP and Dual-Use Capability EvaluationWMDP is a test set of multiple-choice questions that measures whether a model knows the kind of background facts that could help someone build weapons. It is also used to check methods that make models forget such knowledge.Learn · ~60 minutes
  18. 18Frontier Safety Frameworks — RSP, PF, FSFLeading AI labs publish rulebooks that say which dangerous abilities they test for and what extra protections kick in when a model crosses a line. Anthropic, OpenAI and Google DeepMind each have one.Learn · ~75 minutes
  19. 19Anthropic's Model Welfare ProgramModel welfare research asks whether AI systems could have experiences that matter morally, and what low-cost steps make sense under that uncertainty. Anthropic started a formal program on the question.Learn · ~45 minutes
  20. 20Bias and Representational Harm in LLMsLanguage models can repeat stereotypes, leave groups out, or treat people unequally. The lesson separates harms in how groups are described from harms in who gets resources, and covers ways to measure each.Build · ~60 minutes
  21. 21Fairness Criteria — Group, Individual, CounterfactualThere are several ways to define fair: equal outcome rates across groups, similar treatment for similar people, or the same decision if only a protected trait were changed. These definitions often cannot all hold at once.Learn · ~60 minutes
  22. 22Differential Privacy for LLMsDifferential privacy adds carefully measured noise during training so the model cannot reveal much about any single person's data. It offers a math-backed privacy limit but costs extra compute and some accuracy.Build · ~60 minutes
  23. 23Watermarking — SynthID, Stable Signature, C2PAWatermarking marks AI-generated text, images or video so tools can later detect them. SynthID hides a signal in the content, Stable Signature does this for images, and C2PA attaches signed records of where content came from.Build · ~75 minutes
  24. 24Regulatory Frameworks — EU, US, UK, KoreaSeveral major regions now have AI rules, from detailed laws with risk tiers and fines to lighter guidance and sector rules. Dates for different obligations are phasing in over several years.Learn · ~75 minutes
  25. 25EchoLeak and the Emergence of CVEs for AIEchoLeak was a flaw in Microsoft 365 Copilot where a crafted email could make the assistant leak internal data with no click from the victim. It was among the first AI flaws tracked in the public CVE vulnerability list.Learn · ~45 minutes
  26. 26Model, System, and Dataset CardsModel cards, system cards and datasheets are short documents that describe what an AI model or dataset is, how it was made and tested, where it falls short, and how it should and should not be used.Build · ~60 minutes
  27. 27Data Provenance and Training-Data GovernanceData provenance means knowing where training data came from, what permission covers it, and what is inside it. New rules require summaries of training data and respect for opt-outs from website owners.Learn · ~60 minutes
  28. 28Alignment Research Ecosystem — MATS, Redwood, Apollo, METROutside the big labs, a set of independent groups does much of the AI safety work: MATS trains new researchers, Redwood Research studies control, Apollo Research tests for deception, and METR measures dangerous abilities.Learn · ~45 minutes
  29. 29Moderation Systems — OpenAI, Perspective, Llama GuardModeration systems check text and images for harmful content like harassment, hate, self-harm or violence. The lesson compares the OpenAI Moderation API, Google's Perspective API and Llama Guard.Build · ~60 minutes
  30. 30Dual-Use Risk — Cyber, Bio, Chem, Nuclear UpliftDual-use risk is the chance that AI built for good work also helps people cause serious harm, such as hacking, or making biological, chemical or radiological weapons. The lesson reviews where evidence of real help has appeared.Learn · ~75 minutes

Lessons from AI Engineering from Scratch by Rohit Ghumare and contributors, MIT license. Notes for owners by Lozo. License

Next step

Rather have it built for you?

Reading is optional. Tell me what's stuck and I'll build it into the business you already run.

Tell me what's stuck
  • A real conversation, not a pitch
  • You keep your accounts
  • Nothing sends without your yes