Agent Engineering

For ownersAgents that take steps on their own. Read the notes before you let one act for your business.

The core of modern AI engineering. Build agents from first principles.

54 lessons

Start with lesson 1

Lesson 1 of 54, ~60 minutesThe Agent Loop: Observe, Think, Act

The lessons

  1. 01The Agent Loop: Observe, Think, ActAn AI agent is a model running in a loop: it reads the situation, decides on a step, uses a tool, looks at the result, and repeats until the job is done or a stop rule fires.Build · ~60 minutes
  2. 02ReWOO and Plan-and-Execute: Decoupled PlanningInstead of thinking and acting one step at a time, the agent writes the whole plan first, then runs each step. Planning once up front uses fewer model calls and is easier to inspect.Build · ~60 minutes
  3. 03Reflexion: Verbal Reinforcement LearningAfter a failed attempt, the agent writes a short note about what went wrong and keeps it. On the next try it reads those notes first, so it learns from mistakes without retraining the model.Build · ~60 minutes
  4. 04Tree of Thoughts and LATS: Deliberate SearchRather than following one line of reasoning, the agent explores several possible paths, scores each one, and backs up when a path looks bad. It is search applied to thinking.Build · ~75 minutes
  5. 05Self-Refine and CRITIC: Iterative Output ImprovementThe model writes a draft, critiques it, then rewrites it, repeating a few times. A stronger version checks the draft with outside tools, like a calculator or search, instead of trusting its own opinion.Build · ~60 minutes
  6. 06Tool Use and Function CallingFunction calling is how a model asks your software to do something: it outputs a structured request, like look up order 123, and your code runs it and hands back the result.Build · ~60 minutes
  7. 07Agent Memory — Virtual Context and Memory PagingA model can only read so much at once. This approach treats that space like short-term memory and keeps the rest in an outside store, moving facts in and out as needed.Build · ~75 minutes
  8. 08Memory Blocks and Sleep-Time ComputeMemory is split into labeled sections, such as facts about the user and the agent's role, that the model can edit. A background agent tidies and merges these notes while nobody is waiting.Build · ~75 minutes
  9. 09Hybrid Memory: Vector + Graph + KVHybrid memory stores information three ways at once: by meaning for fuzzy search, as simple key-value facts for quick lookup, and as a map of how people and things relate. Results are blended on recall.Build · ~75 minutes
  10. 10Skill Libraries and Lifelong Learning (Voyager)The agent saves working solutions as named, reusable skills, usually small pieces of code with a description. Later it finds and reuses them, and combines them into bigger skills.Build · ~75 minutes
  11. 11Planning with HTN and Evolutionary SearchTwo planning styles: one breaks big tasks into smaller ones by fixed rules so every plan follows those rules, and one generates many candidate solutions, scores them automatically, and keeps the best.Build · ~75 minutes
  12. 12Anthropic's Workflow Patterns: Simple Over ComplexMost AI automation does not need a free-roaming agent. A few fixed patterns, like chaining steps, routing requests, running tasks in parallel, and draft-then-review, cover most business jobs.Learn + Build · ~60 minutes
  13. 13Stateful Graph Orchestration — Durable Execution and CheckpointsThe agent is drawn as a flowchart: each box is a step, arrows are the possible next steps, and progress is saved after every box. If something crashes, it picks up from the last saved point.Learn + Build · ~75 minutes
  14. 14The Actor Model for Agents — Async Messages and Typed RuntimesEach agent works like an independent worker with an inbox. They send each other messages, handle them when they arrive, and one failing does not crash the others.Learn + Build · ~75 minutes
  15. 15Role-Based Agent Teams — Roles, Tasks, ProcessesYou set up a team of agents with job titles, like researcher and writer, give them tasks, and choose whether they work in order or under a manager. A more fixed mode follows set steps.Learn + Build · ~75 minutes
  16. 16OpenAI Agents SDK: Handoffs, Guardrails, TracingA small toolkit for building agents on OpenAI models. Agents can hand a conversation to another agent, run guardrail checks on what comes in and goes out, keep session history, and record every step.Learn + Build · ~75 minutes
  17. 17The Harness as a Library — Subagents and Session StoreThe same engine behind Claude Code, packaged as a library. Developers get ready-made tools, helper agents that work in their own space, hooks to check actions, and saved sessions.Learn + Build · ~75 minutes
  18. 18Production Agent Runtimes — Fast Instantiation and Typed WorkflowsTwo lighter frameworks for running agents in production: Agno for Python teams, built to start agents very quickly, and Mastra for TypeScript teams, with agents, tools and workflows built in.Learn · ~45 minutes
  19. 19Benchmarks: SWE-bench, GAIA, AgentBenchBenchmarks are standard tests for agents. SWE-bench checks whether an agent can fix real code bugs, GAIA checks general tool-using questions, and AgentBench covers several environments.Learn · ~60 minutes
  20. 20Benchmarks: WebArena and OSWorldWebArena tests agents doing tasks on realistic websites, like shopping or forum sites. OSWorld tests agents operating a full desktop computer. Both show agents still fall well short of people.Learn · ~60 minutes
  21. 21Computer Use: Claude, OpenAI CUA, GeminiComputer use means the model looks at screenshots and controls a mouse and keyboard, like a person would. It can work with software that has no API.Learn · ~60 minutes
  22. 22Voice Agents: Pipecat and LiveKitA voice agent listens, turns speech into text, has a model write a reply, and speaks it back. Pipecat and LiveKit are open-source tools that wire those steps together in real time.Learn · ~60 minutes
  23. 23OpenTelemetry GenAI Semantic ConventionsOpenTelemetry is a common standard for recording what software does. Its GenAI rules define how to log model calls, tool calls and token use the same way across monitoring products.Learn + Build · ~60 minutes
  24. 24Agent Observability: Langfuse, Phoenix, OpikObservability tools record each agent run step by step, so a team can replay what happened, see costs, compare prompt versions, and score answers. Langfuse, Arize and Comet Opik offer open-source options.Learn · ~45 minutes
  25. 25Multi-Agent Debate and CollaborationSeveral copies of a model answer the same question on their own, then read each other's answers and revise over a few rounds until they agree. This tends to catch errors one model would miss.Learn + Build · ~60 minutes
  26. 26Failure Modes: Why Agents BreakAgents break in recurring ways: doing actions that were never asked for, drifting beyond the task, small errors snowballing, forgetting earlier context, and misusing tools.Learn + Build · ~60 minutes
  27. 27Prompt Injection and the PVE DefensePrompt injection is when someone hides instructions inside content the agent reads, like a web page or email, hoping the agent obeys them instead of its owner.Build · ~75 minutes
  28. 28Orchestration Patterns: Supervisor, Swarm, HierarchicalCommon ways to organize several agents: one supervisor directing workers, peers passing work among themselves, layers of managers, or agents debating. Each adds cost and complexity.Learn + Build · ~60 minutes
  29. 29Production Runtimes: Queue, Event, CronAgents can run as instant replies, streamed replies, long jobs that survive restarts, background queue jobs, jobs triggered by events, or jobs on a schedule. The right shape depends on the work.Learn · ~60 minutes
  30. 30Eval-Driven Agent DevelopmentBefore building, write a set of test cases with expected results, then measure every change against them. Testing comes first and steers each later decision about the agent.Learn + Build · ~60 minutes
  31. 31Agent Workbench Engineering: Why Capable Models Still FailA smart model still fails without a working setup around it: instructions, saved progress, task limits, real feedback, checks, review and handoff notes. That setup is called the workbench.Learn + Build · ~45 minutes
  32. 32The Minimal Agent WorkbenchThe smallest working setup is three files: a short instructions file that points to deeper guides, a state file saying where things stand, and a task board of what is in progress, blocked, or next.Build · ~45 minutes
  33. 33Agent Instructions as Executable ConstraintsInstead of writing rules as advice, write them so a script can check them, such as never edit these files, or every change needs a test. Then breaking a rule shows up automatically.Build · ~50 minutes
  34. 34Repo Memory and Durable StateChat history disappears, but files saved with the project last. The agent writes progress, decisions and open questions into versioned files that the next session or person reads.Build · ~60 minutes
  35. 35Initialization Scripts for AgentsA startup script checks the environment once, such as software versions, settings and whether tests run, and saves the results so the agent does not rediscover them every session.Build · ~45 minutes
  36. 36Scope Contracts and Task BoundariesA scope contract is a short file for each task that lists what the agent may change, what it must leave alone, what done looks like, and how to undo the change if it goes too far.Build · ~50 minutes
  37. 37Runtime Feedback LoopsThe agent runs real commands and reads the actual output, errors and timing from a saved record, instead of assuming what happened. Decisions then rest on facts.Build · ~50 minutes
  38. 38Verification GatesA verification gate is a final check the agent cannot skip. It compares the task's scope, the command results, the rule checks and the actual changes, then decides whether the task is really done.Build · ~55 minutes
  39. 39Reviewer Agent: Separate Builder from MarkerA second agent, with different instructions and read-only access, reviews what the first agent built. Because it did not write the work, it is more likely to spot problems.Build · ~55 minutes
  40. 40Multi-Session HandoffWhen a work session ends, the agent writes a handoff note: what was done, what is left, open questions, and how to continue. The next session starts productive right away.Build · ~50 minutes
  41. 41The Workbench on a Real RepoThe lesson runs the same task on a small app twice, once with just a prompt and once with the full setup of rules, scope, checks and review, then compares the results.Build · ~60 minutes
  42. 42Capstone: Ship a Reusable Agent Workbench PackThe section ends with a reusable folder of templates and scripts: instructions, state, scope contracts, checks, reviewer prompts and handoff notes that can be copied into any project.Build · ~75 minutes
  43. 43Frame the Task Before the Agent Writes CodeBefore an agent writes anything, spell out the task: the goal, the limits, what is out of scope, and how success will be checked. A fast agent on a vague task just builds the wrong thing quickly.Learn + Build · ~60 minutes
  44. 44Build an Evidence-Backed Execution PlanA good plan lists each change, the facts that justify it, what must happen first, and the exact check that closes it. It shows which steps can run together and which must wait.Learn + Build · ~65 minutes
  45. 45Delegate Agent Work with Isolation and Merge ContractsSplit work across several agents only when the pieces are truly independent. Give each one its own files, a defined result, and proof to return, then merge by agreed rules.Learn + Build · ~70 minutes
  46. 46Turn Every Agent Correction into a System ImprovementWhen you correct an agent, do not just fix that one result. Turn the correction into a test, a rule, an example or a tool change so every future run improves.Learn + Build · ~65 minutes
  47. 47Define the Outcome Before You Choose the OutputDecide what result you want in the world before choosing what to build. Since agents build quickly, picking the wrong problem now costs more, not less.Learn + Build · ~60 minutes
  48. 48Discover the Workflow People Actually PerformFind out how work is really done today, step by step: who does what, with which tools, where they wait, copy and paste, or work around the system. Real requirements come from there.Learn + Build · ~70 minutes
  49. 49Map Assumptions and Resolve the Riskiest One FirstList everything that must be true for a project to succeed, rate each by impact and how unsure you are, then test the riskiest one before building much.Learn + Build · ~65 minutes
  50. 50Choose the Smallest Slice That Can Change the DecisionBuild the smallest version that can answer the question that matters next. Small only helps if the result could change your decision.Learn + Build · ~65 minutes
  51. 51Write Specifications That Preserve JudgmentA good spec fixes what must stay true and how success will be checked, but leaves cheap, reversible choices to the builder. It marks which decisions are locked, limited or delegated.Learn + Build · ~75 minutes
  52. 52Design Success Metrics Before the Result ExistsPick how you will measure success before you see results. Start from the goal, list the questions it raises, then choose a few metrics, with limits that catch harmful side effects.Learn + Build · ~70 minutes
  53. 53Choose Prototype, Pilot, or Production DeliberatelyA prototype tests whether an idea can work at all. A pilot tests it with real users in a limited way. Production means someone owns it every day. Each needs different controls.Learn + Build · ~70 minutes
  54. 54Build a Feedback Ratchet with Ownership and RetirementAfter launch, route every signal, such as errors, complaints and corrections, to the part of the system that owns it, with someone responsible. Each fix also gets a rule for when to remove it.Learn + Build · ~75 minutes

Lessons from AI Engineering from Scratch by Rohit Ghumare and contributors, MIT license. Notes for owners by Lozo. License

Next step

Rather have it built for you?

Reading is optional. Tell me what's stuck and I'll build it into the business you already run.

Tell me what's stuck
  • A real conversation, not a pitch
  • You keep your accounts
  • Nothing sends without your yes