Phase 14 · Lesson 29 of 54

Production Runtimes: Queue, Event, Cron

Production agents run on six runtime shapes: request-response, streaming, durable execution, queue-based background, event-driven, and scheduled. Pick the shape before you pick the framework. Observability is load-bearing at every shape.

Lesson 29 of 54Learn~60 minutesPython (stdlib)

Before this: Lesson 13 of Phase 14 (LangGraph), lesson 22 of Phase 14 (Voice)

On this page

Learning Objectives

  • Name the six production runtime shapes and match each to a framework / product pattern.
  • Explain why durable execution (LangGraph) matters for long-horizon tasks.
  • Describe the event-driven runtime and when Claude Managed Agents fits.
  • Explain the observability-as-load-bearing claim for multi-step agents.

The Problem

Production agents fail in ways a Jupyter notebook doesn't surface: network timeouts at step 37, user hangs up mid-voice call, cron job dies on machine reboot, background worker runs out of memory. The runtime shape determines which failures are survivable.

The Concept

Request-response

  • Synchronous HTTP. User waits for completion.
  • Only viable for short tasks (<30s).
  • Stacks: Agno (Python + FastAPI), Mastra (TypeScript + Express/Hono/Fastify/Koa).
  • Observability: standard HTTP access logs + OTel spans.

Streaming

  • SSE or WebSocket for progressive output.
  • LiveKit extends this to WebRTC for voice/video (Lesson 22).
  • Stacks: any framework with streaming support + a frontend that handles SSE/WS.
  • Observability: per-chunk timing, first-token latency, tail latency.

Durable execution

  • State checkpointed after every step; auto-resumes on failure.
  • AutoGen v0.4 actor model isolates failures to one agent (Lesson 14).
  • LangGraph's core differentiator (Lesson 13).
  • Essential when step count is unknown and recovery cost is high.

Queue-based / background

  • Job enters a queue, workers pick up, results flow back via webhooks or pub/sub.
  • Essential for long-horizon agents (dozens-to-hundreds of steps per task, per Anthropic's computer use announcement).
  • Stacks: Celery (Python), BullMQ (Node), SQS + Lambda (AWS), custom.
  • Observability: queue depth, per-job latency distribution, DLQ size.

Event-driven

  • Agents subscribe to triggers: new email, PR opened, cron fire.
  • Claude Managed Agents covers this out of the box (Lesson 17).
  • CrewAI Flows (Lesson 15) structures event-driven deterministic workflows.
  • Observability: trigger source, event-to-start latency, agent latency.

Scheduled

  • Cron-shaped agents that run periodically.
  • Combine with durable execution so a failing nightly run resumes next tick.
  • Stacks: Kubernetes CronJob + a durable framework; hosted (Render cron, Vercel cron).

2026 deployment patterns

  • CrewAI Flows for event-driven production.
  • Agno stateless FastAPI for Python microservices.
  • Mastra server adapters (Express, Hono, Fastify, Koa) for embedding.
  • Pipecat Cloud / LiveKit Cloud for managed voice (Lesson 22).
  • Claude Managed Agents for hosted long-running async.

Observability is load-bearing

Without OpenTelemetry GenAI spans (Lesson 23) plus a Langfuse/Phoenix/Opik backend (Lesson 24), you cannot debug a multi-step agent that failed at step 40. This is not optional for production. It's the difference between "we debug fast" and "we replay from scratch with more logging."

Where production runtimes fail

  • Wrong shape choice. Picking request-response for a 5-minute task. Users hang up; workers pile up; retries compound.
  • No DLQ. Queue workers without dead-letter. Failed jobs vanish.
  • Opaque background work. Background agent runs without trace export. Failures are invisible until the user reports them.
  • Skipping durable state. Any run > 30 seconds where you can't afford to restart needs durable execution.
wb-runtime-shapes

Build It

code/main.py is a stdlib multi-shape demo:

  • Request-response endpoint (plain function).
  • Streaming handler (generator).
  • Queue-based worker with DLQ.
  • Event trigger registry.
  • Cron-shaped scheduler.

Run it:

python3 code/main.py

Output: five traces showing each shape's behavior on the same task. Same agent logic, different outer shells. Durable execution (the sixth shape) is intentionally covered in Lesson 13 with LangGraph checkpointing.

Use It

  • Request-response for chat-style UX.
  • Streaming for progressive responses.
  • Durable for long-horizon tasks.
  • Queue for batch / async / long-running.
  • Event for agent reactivity.
  • Cron for housekeeping (memory consolidation, evals, cost reports).

Ship It

outputs/skill-runtime-shape.md picks a runtime shape for a task and wires the observability requirements.

Exercises

  1. Port your Lesson 01 ReAct loop to all six shapes in your stack. Which shape fits which product surface?
  2. Add a DLQ to the queue-based demo. Simulate 10% job failure; surface DLQ size.
  3. Write a cron-triggered eval agent that runs nightly against your top 20 traces from the day.
  4. Implement streaming with backpressure: if the client is slow, pause the agent. How does this interact with a turn budget?
  5. Read Claude Managed Agents docs. When would you move a self-hosted long-horizon agent to managed?

Key Terms

Term What people say What it actually means
Request-response "Synchronous" User waits; short tasks only
Streaming "SSE / WS" Progressive output; better UX; latency observable per chunk
Durable execution "Resume from failure" Checkpointed state; restart at last step
Queue-based "Background jobs" Producer / worker pool / DLQ
Event-driven "Trigger-based" Agent reacts to external events
DLQ "Dead-letter queue" Parking lot for failed jobs
Claude Managed Agents "Hosted harness" Anthropic-hosted long-running async with caching + compaction

Further Reading

Check yourself

Quick quiz

  1. What are the six runtime shapes the lesson lists?

    1. Map, filter, reduce, fold, scan, group
    2. Request-response, streaming, durable execution, queue-based, event-driven, scheduled
    3. REST, gRPC, GraphQL, WebSocket, SSE, MQTT
    4. Read, write, exec, fork, exit, wait
    Show the answer

    B. Request-response, streaming, durable execution, queue-based, event-driven, scheduled These are the six runtime shapes; pick a shape before picking a framework.

  2. When is request-response a poor fit?

    1. On internal services
    2. When the task takes longer than about 30 seconds and users hang up while workers pile up
    3. On REST APIs
    4. On a 1-second classification
    Show the answer

    B. When the task takes longer than about 30 seconds and users hang up while workers pile up Synchronous HTTP is only viable for short tasks; long tasks need queue/durable/event shapes.

  3. What is durable execution's core property?

    1. Lower token cost
    2. Faster inference
    3. Native voice support
    4. State is checkpointed after every step so the runtime can resume from the last successful step on failure
    Show the answer

    D. State is checkpointed after every step so the runtime can resume from the last successful step on failure Checkpoint-and-resume is the differentiator; LangGraph is the lesson's reference.

  4. Which Anthropic observation justifies queue-based runtimes for long-horizon agents?

    1. Tasks must run synchronously
    2. Tasks always complete in 5 seconds
    3. Computer use announcement: dozens-to-hundreds of steps per task is normal
    4. Cron solves everything
    Show the answer

    C. Computer use announcement: dozens-to-hundreds of steps per task is normal Anthropic flagged dozens-to-hundreds of steps per task as normal; that workload needs queues or durable runtimes.

  5. What does the lesson mean by 'observability is load-bearing'?

    1. Providers require it for billing
    2. Logs are pretty
    3. Disk gets full
    4. Without OTel GenAI spans and a Langfuse/Phoenix/Opik backend you cannot debug a multi-step agent that failed at step 40
    Show the answer

    D. Without OTel GenAI spans and a Langfuse/Phoenix/Opik backend you cannot debug a multi-step agent that failed at step 40 Observability is not optional for multi-step agents; it is the difference between debugging and replaying from scratch.

  6. Why must queue workers have a dead-letter queue (DLQ)?

    1. DLQ makes jobs faster
    2. Without DLQ, failed jobs vanish silently
    3. DLQ is encrypted
    4. DLQ is required by AWS
    Show the answer

    B. Without DLQ, failed jobs vanish silently Failed jobs without DLQ disappear; DLQ is the parking lot for failed jobs.

  7. What does the lesson recommend pairing cron-shaped agents with?

    1. Durable execution so a failing nightly run resumes next tick
    2. WebRTC
    3. GPU autoscalers
    4. Streaming SSE
    Show the answer

    A. Durable execution so a failing nightly run resumes next tick Cron + durable execution recovers cleanly from failed scheduled runs.

Questions people ask

How do you run an AI agent on a schedule?

Use a scheduler such as cron, a cloud scheduled job, or a workflow tool to start the agent at set times. Add logging, retries, timeouts and alerts, so a failed nightly run is noticed instead of silently skipped.

What is durable execution for AI agents?

Durable execution saves progress after each step, so a long task can resume after a crash, timeout or deploy without starting over or repeating actions. It matters for multi-step agents that run for minutes or hours.

Run the code Read it on GitHub

Next lessonEval-Driven Agent DevelopmentPrevious: Orchestration Patterns: Supervisor, Swarm, Hierarchical

Lessons from AI Engineering from Scratch by Rohit Ghumare and contributors, MIT license. Notes for owners by Lozo. License

Next step

Rather have it built for you?

Reading is optional. Tell me what's stuck and I'll build it into the business you already run.

Tell me what's stuck
  • A real conversation, not a pitch
  • You keep your accounts
  • Nothing sends without your yes