Frameworks & Orchestration

Picture two engineers. One writes backpropagation by hand: a forward pass, a loss, then a line-by-line chain rule to get every gradient. The other writes three lines with PyTorch: loss.backward(), and every gradient appears. Both engineers can train a model. Only one of them can explain what happens when training suddenly diverges.

This is the trade a framework makes. A framework is code that someone else wrote to handle a common pattern for you, so you write less of it yourself. PyTorch's autograd hides the chain rule. DSPy and LangGraph, this lecture's two frameworks, hide a different thing: the plumbing of calling a language model again and again, keeping track of what it has said, and deciding what happens next.

The lecture is direct about the deal. Frameworks abstract away some of the underlying system details. For the rest of this course, and for any system you ship, you still need to understand those details, because an abstraction fails exactly where it hides a decision you needed to make. Today's goal is narrower and useful on its own: learn what each framework abstracts, so that when it leaks, you know where to look.

Three pillars, one job each

Lecture 1 drew every AI system as three pillars, and this lecture reuses them to say where frameworks sit. Data is what you feed the system: training data, rollouts (a model's own recorded attempts), demonstrations, human feedback. The system or model is the thing that does the work: a plain language model, a compound AI system (several model calls and tools wired together), or a full agentic system. Evaluation and metric is how you check the work: pass rate, exact match, an LLM acting as a judge, cost, latency.

A framework almost always lives in the middle pillar. DSPy and LangGraph are both ways of building the system or model box. But the lecture's whole point is that the middle pillar cannot be judged alone: a system is only as good as the data it is built and tested on, and the metric that says whether it worked. Chapter 4 makes this concrete when DSPy improves a prompt using nothing but examples and a scoring function.

The learning objectives, as a test of this lesson

The lecture sets three objectives, and they are a fair test of whether this chapter and the ones after it worked. First, identify common frameworks used in building agentic systems, which Chapters 1 through 9 do by name: DSPy, LangGraph, LlamaIndex, mini-SWE-agent, OpenHands, Pi Agent, OpenClaw, Temporal, MLflow, LangSmith, Langfuse. Second, relate each framework's abstractions back to the system designs earlier lectures already covered: a DSPy signature is the structured-output idea from Lecture 2's Chapter 9, and a LangGraph tool-calling loop is the ReAct pattern from Lecture 4, now wrapped in a library. Third, assess when a given tool actually earns its cost in reliability and observability, which Chapter 10 returns to directly.

Where today sits on the spectrum

The lecture's single recurring picture, which returns at the start of nearly every part, is a line with three points on it. At the left, a model, called through a plain API (OpenAI's or Anthropic's), is a small part of a larger system: it answers one question and nothing more. At the right, an agent hands control flow to the model itself: the model decides what runs next. In between sits a workflow, also called a compound AI system: several model calls and tools, but the order they run in is fixed in your code, the same every time.

The hero at the top of this page is that exact line, made into something you can run. Dragging from Model to Workflow to Agent does not change what the task needs to accomplish. It changes who decides how to accomplish it, and that single change is why the rest of this lecture exists: a workflow's fixed order means it fails the same way every time, which is easy to test and hard to recover from; an agent's model-chosen order means it can route around a surprise, which is exactly why Chapters 9 and 10 spend so much time on watching it while it runs.

Two more labels sit below that line on the lecture's slide, and Chapter 1 explains both: reliability tools (Temporal is the lecture's example) and observability tools (Langfuse, MLflow). Notice where they are drawn: underneath the whole spectrum, not at one point on it. They apply whether you built a workflow or an agent, because both can still fail.

Chapter 0 showed you the spectrum on one task. The lecture now puts real, named tools on it, and every tool this lesson teaches lands somewhere on that same line.

At the left end, calling the OpenAI API or the Anthropic API directly is a model playing a small part of a larger system. You write one HTTP request, you get one reply, and every decision about what to do with that reply is yours. There is no framework here to speak of, only a client library.

In the middle, a workflow, also written compound AI system, is where the logic and orchestration live in your program. DSPy and LangGraph, this lecture's two live-coded frameworks, both build workflows: DSPy through signatures and modules (Chapters 2 to 4), LangGraph through a graph of nodes and edges (Chapters 6 and 7). LlamaIndex, a framework this lecture only names, also sits here, built around retrieval pipelines rather than general graphs.

At the right end, control flow is the model's own job: the model decides, step to step, what happens next. This is an agent. The lecture's landscape slide marks this end with SWE-agent and OpenHands, two open-source coding agents. Chapter 8 then goes one level deeper and compares four harnesses directly: mini-SWE-agent, a smaller, more recent descendant of SWE-agent's own idea, plus OpenHands, Pi Agent and OpenClaw. DSPy and LangGraph both reach into this territory too, through dspy.ReAct and a LangGraph graph with a loop in it (Chapters 5 and 7), which is the lecture's way of saying that a framework is not glued to one point on the line: you choose how far right to push it.

Two more families sit below the line itself, applying across all three points rather than at one of them. Reliability tools, Temporal foremost among them (Chapter 9), keep a long-running system alive through crashes and flaky calls. Observability tools, Langfuse and MLflow (Chapter 10), let you see what a system actually did after the fact. A workflow needs these far less than an agent does, because a workflow's fixed order is easy to predict and test in advance; an agent's model-chosen order is not, so you must watch it run instead.

1. DSPy signatures: strictly define a model call's inputs and outputs.
2. DSPy modules: compose signatures and logic into callable functions.
3. DSPy optimizers: improve a prompt from data and a metric.
4. DSPy agents: ReAct and the recursive language model.
5. LangGraph state and graphs: a shared object, nodes, conditional edges.
6. LangGraph agents: a loop that lets the model keep deciding.
7. Open-source agent harnesses: four real coding agents, compared.
8. Temporal: durable execution for workflows that run for hours.
9. Telemetry, evals and keeping it simple: watch it, test it, justify it.
10. The whole lecture on one page: numbers, code, and what comes next.

Here is the problem DSPy was built to solve. You want a model to read a message and label its sentiment: positive, negative or neutral. You write a prompt by hand. The lecture's own demo does exactly this and gets back, for three different messages, the words “positive”, “Negative” and “sentiment: confused”. Three different formats, for the same question, from the same model. Code that expects one string to compare against "positive" now has to guess which of three shapes it received.

DSPy is a framework (its name stands for declarative self-improving language programs) built around fixing this, and it starts with a signature: a strict, typed statement of a model call's inputs and outputs, written once. The simplest kind is a string: "message -> sentiment" says, in one line, this call takes a message in and returns a sentiment out. Wrap it in dspy.Predict(signature) and you have a callable function, chat, where chat(message="...").sentiment always returns the one field you asked for, by name, not by scraping free text.

A string signature is already more than a prompt, because it is a contract your code can rely on. But it says nothing about what kind of thing sentiment is. The lecture's next step is a typed signature, written as a Python class:

Now sentiment is not just a name, it is a Literal type: Python's way of saying “only these three exact strings are legal values, nothing else.” DSPy turns this class into a prompt behind the scenes, telling the model the allowed values, and parses the reply back into a real typed object. The slide's own output, after this change, is Prediction(sentiment='positive', confidence=0.98) for every message: one shape, every time, with a number alongside the label that a plain string reply could never carry cleanly.

The lecture's third example raises the stakes: extract a calendar event from a free-text email, where the output has five different fields of five different types (a name, a date, a start time, a duration, and a free-text description), each with its own description telling the model what format to use and what to do when a piece of information is missing. This is the shape Chapter 6 will build a whole LangGraph node around, and the shape Chapter 3's modules compose.

A signature says what a call takes in and gives back. It says nothing about how the model should arrive at the answer. That is a module's job: a module wraps a signature with a particular strategy for fulfilling it, and still behaves like an ordinary callable function.

The default module is dspy.Predict, used in every example so far: hand the model the inputs, ask directly for the outputs. For the calendar-extraction signature from Chapter 2, sometimes a direct answer is not enough. The lecture reaches for a different module, dspy.ChainOfThought, with the exact same signature:

dspy.ChainOfThought adds one new output field, reasoning, and asks the model to fill it in before the rest. The signature's fields did not change; the strategy for filling them did. This is exactly the chain-of-thought idea from Lecture 2's Chapter 8, now packaged as a one-word swap: change the module, keep the signature, keep the code that calls it.

Modules compose like functions, because they are functions

A module is not limited to one model call. The lecture's next step chains two of them inside one Python class:

Read this as data flow, because it is the whole point. email (a string) goes into extract, which returns a details object with the five typed fields from Chapter 2. details.event_name (a string, or None) goes into a plain Python if: this check costs nothing and calls no model, because it is ordinary code, not a signature. Only if an event name exists does event_name feed a second model call, pick_emoji, whose one-character output then goes to a calendar-writing function that is plain code once more. Two model calls, two signatures, one Python class, and the branching between them is exactly as fixed as any other function you have ever written.

This is the clearest place to see what “workflow” meant in Chapter 1. The order of extract, then check, then pick an emoji, then write, is fixed in forward. The model only fills in the content asked of it at each of the two calls it is given. Change the order, add a step, or branch differently, and you edit the Python, not the model's behavior.

So far every prompt in this lesson was written by hand, hidden inside dspy.Predict and dspy.ChainOfThought. An optimizer, also called a teleprompter, is DSPy's answer to a different question: instead of you guessing at better wording, can the framework improve the prompt (or the weights) itself, given data and a way to check the result?

The lecture builds this from three pieces, in order, on the calendar-extraction task.

The data: examples with a known right answer

First, 50 realistic emails, each paired with a gold label: the correct event_name, date, start and duration_minutes a human would extract. The lecture turns each row into a dspy.Example, marks the email as the input, shuffles, and splits it: 30 go into a trainset the optimizer is allowed to learn from, 20 into a testset it never sees until scoring time. This split is not a DSPy quirk, it is the same train/test separation every machine-learning method needs, so that a good score at the end means the method generalizes rather than memorized its own answer key.

The metric: a number the optimizer can climb

Second, a metric: a plain Python function that takes one gold example and one prediction and returns a score. The lecture's metric checks three fields exactly (date, start, duration_minutes) against the gold row, and also returns a free-text feedback string listing which fields were wrong, for example “All fields correct.” or “Wrong -> date”. The score itself is a 0 or a 1: all-fields-correct is the only pass. That feedback string matters more than it looks: Chapter 10's reading calls this evaluation infrastructure, and here the metric doubles as a diagnostic, not only a scoreboard.

The optimizer: GEPA tunes a small model against a big one's judgment

Third, the optimizer itself. The lecture loads a small, cheap model, Gemma 3 12B, wraps the same ExtractEvent signature around it, and scores it before touching anything: 11.0 out of 20 (55.0%) on the held-out testset. Then it runs GEPA, one of DSPy's optimizers, with a modest budget of 100 metric calls (about three minutes): the big model, dspy_lm, reflects on the small model's mistakes using the metric's feedback text, and rewrites the small model's prompt. Score the small model again, after optimization, on the exact same held-out 20 emails: 100%.

Nothing about the small model's weights changed. The lecture's own slide is explicit: the only thing GEPA changed was extract_optimized.signature.instructions, the text instructions attached to the signature. A rewritten paragraph of English, produced by a bigger model reading a smaller model's graded mistakes, closed a 45-point gap on this task.

Every DSPy module so far has run a fixed number of model calls: one for Predict, one for ChainOfThought, two chained by hand in Chapter 3's forward. DSPy also ships modules that run a variable number of calls, deciding step to step whether to keep going, which is Chapter 1's agent end of the line, reached from inside the same framework.

ReAct: think, act, read the result, repeat

ReAct, short for Reason + Act, is the paper (Yao et al., 2022, arXiv 2210.03629) this module is named for, and it is the same pattern Lecture 4 already walked through as a tool-calling loop. A worked trace from that paper: asked which company's remote control a product was originally designed for, the model writes Thought 1 (I need to search X and find its remote), Act 1 (Search[X]), reads the returned Obs 1, writes Thought 2 reacting to what it just read, issues Act 2, and so on, until a final Finish[answer] ends the loop. Each act depends on what the previous observation said; nothing about the sequence of acts is fixed in advance.

In DSPy this entire loop is one line, wrapping the same kind of signature from Chapter 2:

The signature, "request -> summary", says nothing about calendars, searching, or books. It only says: give me a summary for a request. The tools list is where the agent's actual abilities live: a set of callable functions (here, tinycal's calendar API) the model may call, by name, as many times as the request needs. One request became at least three separate calendar writes, a number the signature itself never specifies and the programmer never counted in advance. That is the one line that moves dspy.ReAct off the workflow middle and onto the agent end of Chapter 1's spectrum: the model, not your code, decided how many tool calls this request needed and in what order.

RLM: a recursive language model that manages its own context

RLM, the recursive language model, is DSPy's module for a different kind of variability: not how many actions to take, but how much of a very long input to actually read. The lecture's example hands an RLM an inbox of emails far larger than one prompt should hold, with a question like “which events are held in the Gates building, and on what dates?” Rather than stuffing every email into one call, the root RLM (depth 0) can recursively spin up its own sub-calls (depth 1, depth 2) over smaller slices of the inbox, read their answers back, and only then write a final response. The signature is again just inbox, question -> event_names: list[str], dates: list[str]; the recursive search underneath is the module's job, not something the programmer orchestrated by hand.

DSPy builds a workflow out of signatures and modules, chained inside ordinary Python functions. LangGraph builds the same kind of workflow a different way: as an explicit graph you can draw. The lecture builds the identical calendar task from Chapters 2 and 3 again, in LangGraph, so the comparison is exact.

State: one shared object the whole graph can see

Everything starts with the state: a shared workspace for the graph, an object every part of the graph can read and change. In code it is a typed dictionary:

Compare this with Chapter 3's forward(self, email): there, each value passed from one line to the next as a local Python variable. Here, every value lives in one object, state, that every node reads from and writes back to. This is not a cosmetic difference. A DSPy module's variables disappear once forward returns; a LangGraph state is the thing Chapter 9's Temporal can checkpoint, because it is a single, explicit, inspectable object at every step.

Nodes: plain functions that read and write the state

A node is a function that reads part of the state, does some work (sometimes a model call, sometimes plain code), and returns the part it changed:

Every node in the lecture's graph follows this shape. pick_emoji reads state["details"] and returns {"emoji": ...}. add_to_calendar reads both details and emoji and writes to a real calendar, returning a status string. LangGraph merges each returned piece back into the one shared state, so the next node always sees the latest version.

Edges: fixed arrows, or a conditional choice

A graph wires nodes together with edges. Every graph needs a START and an END. A plain edge is a fixed arrow, always taken:

Most of these edges are fixed arrows, the LangGraph twin of Chapter 3's plain Python if. The one edge out of extract, though, is a conditional edge: a small function, route, reads state["details"] and returns the name of whichever node should run next, exactly the way Chapter 3's if details.event_name is None: return ... picked a path. The lecture's own route is one line: return "pick_emoji" if an event name, a date and a start time were all found, otherwise "__end__", skipping straight to the end with nothing added.

Chapter 6's graph has a route node that reads data and branches, but the branch itself is fixed Python, decided the same way every time on the same input. Turning that same graph into an agent takes one structural change: a loop, where control flows back to a node that calls the model, and a conditional edge whose destination the model's own reply controls.

The lecture's agent graph has three kinds of node. model calls the language model with the current message history (state, here literally a list of messages). tools executes whatever the model asked for in its last reply. A conditional edge out of model checks that reply: did it ask for a tool? If yes, go to tools; once tools returns its result, an ordinary edge sends control back to model, closing the loop. If the model's reply asked for no tool, the conditional edge goes to END instead.

Notice what did not change from Chapter 6: it is still a StateGraph, still built from nodes and edges, still compiled the same way. create_agent is LangGraph's built-in shortcut for wiring exactly this model / tools loop automatically, the same way dspy.ReAct was DSPy's one-line shortcut for its own loop in Chapter 5. The difference between a LangGraph workflow and a LangGraph agent is not a different library, it is which edge's destination the model gets to pick.

Why message history, and not a typed state object

Chapter 6's state held typed fields: an email, a details object, an emoji. This agent's state is simpler in shape and harder to predict in length: a growing list of messages, each one either something a human said, something the model said, or a tool's result. The lecture's own annotation on this slide reads “the State (can be message history)”, pointing at exactly this choice. A fixed workflow can afford a typed state, because every field it will ever need is known the moment you write the graph. An agent cannot: it does not know in advance how many tool calls a request will need, so its state has to be a list that grows by one entry every time the loop goes around, rather than a fixed set of named fields.

This is also why the loop needs a conditional edge at all, rather than the fixed edges Chapter 6 used almost everywhere. A fixed edge always goes to the same next node; it cannot express “go back to model only if the last message asked for a tool, otherwise stop.” The condition itself reads the newest entry at the end of the message list, which is exactly the entry the model node just added, so the edge's decision and the model's decision are reading the very same piece of data.

Put together, Chapters 6 and 7 answer the question Chapter 1 opened with, concretely, twice: the same StateGraph primitives build a workflow when every edge is fixed, and an agent the moment one edge's destination depends on something the model wrote a moment earlier. Nothing about LangGraph itself changed between the two chapters; only the shape of one edge did.

Chapters 2 through 7 built agent-shaped loops inside a framework you write code in. An agent harness is a finished program, already built, that wraps a model in an agent loop for you: what tools it gets, how it remembers what happened, how it decides a run is stuck. The lecture's figure draws a harness as four stacked layers around an LLM core: planning and reasoning on one side, memory on the other, a tools layer underneath (retrieval, a calculator, code execution, search, read and write memory), and an environment it observes and acts on (a desktop OS, a browser, a phone app, a database).

Building an agent means making a design choice at every one of those layers, and different harnesses make opposite choices to suit different jobs. Four real, open-source examples, each solving the same kind of problem (an autonomous coding agent) with a different answer:

• mini-SWE-agent has no tool but bash. Its history is completely linear: every step just appends to the message list, nothing more. Every action runs with subprocess.run, so each command is fully independent, with no persistent shell session carried between them.
• OpenHands stores every step as an event, one JSON file per event on disk, and rebuilds the messages sent to the model from those files at every step. It keeps a persistent tmux terminal, so cd, environment variables and virtual environments carry over between commands. A detector stops the run after four repeats of the same action and result.
• Pi Agent has only four tools: read, write, edit, and bash, with a system prompt under 1,000 tokens. It has no permission system, running with whatever permissions launched it. It stores its history as a tree, not a list, so you can go back to an earlier message and branch a new attempt from there. Deliberately simple: no plan mode, no sub-agent tool.
• OpenClaw wakes itself every 30 minutes on a heartbeat turn, replying NO_REPLY to stay silent when there is nothing to say. It runs as one long-lived gateway process owning every chat channel. Its identity comes from Markdown files (AGENTS.md, SOUL.md, IDENTITY.md, USER.md, MEMORY.md) injected into its system prompt, and at 3 AM a background job reads the day's notes and moves what is worth keeping into MEMORY.md.

No row in that list is strictly better than another. A linear history is cheap and simple, at the cost of losing the ability to branch or recover context from far back. A tree history lets you branch, at the cost of more bookkeeping. A persistent terminal session remembers environment state for free; a stateless one is more predictable to retry. The lecture's own line: a good harness balances task success, cost, latency and user control, and the right balance depends on the use case.

The harness tax: same model, different harness, different result

The lecture backs this with a real measurement, Terminal-Bench-2, holding the model fixed at Claude Sonnet 4.6 and only changing the harness: Pi costs $0.61 per rollout for a 65.6% success rate; Codex costs $0.55 for a 63.3% success rate; Claude Code costs $0.67 for 62.2%. The model did not change. The harness around it did, and both cost and success rate moved with it. This is what the lecture's slide calls the harness tax: the difference a harness alone makes, paid or earned, even when you hold everything else constant.

Every graph and module built so far lives entirely in one running Python process. Kill that process mid-run, at step forty of fifty, and everything it had done is gone: there is no record of which calendar entries were already written, no way to resume from step forty-one instead of starting over. The lecture's reason this matters: agents run for long periods of time, and the infrastructure supporting them must be reliable; logs and observability tools exist because, as hard-earned lessons from software engineering teach, failures will happen, and the question is only whether you can see and recover from them.

Two kinds of unit, one new place they live

Temporal is the lecture's example of an orchestration framework: a system that runs your workflow's steps on a server, not only in your local process, so that the server remembers exactly where things stand even if your code crashes. It splits every job into two kinds of unit. An activity is a functional unit that can fail: one network call, one tool invocation, one model call. A workflow links activities together, in order, the same role forward played in Chapter 3 and a LangGraph graph played in Chapter 6, except that Temporal's server, not your process, tracks where the workflow currently is.

The pieces: your client code starts a workflow. A task queue on the server holds two lanes, an activity queue and a workflow queue, plus an execution history and state store backed by real storage. Activity workers and workflow workers, separate processes you run (possibly many of each, possibly on different machines), pull work off those queues and execute it. If a worker crashes mid-activity, the activity is still recorded as pending in the task queue; another worker, or the same one restarted, picks it back up, because the server's execution history, not the crashed process's memory, is the source of truth.

Read the if inside run: it is the same branch, playing the same role, as Chapter 3's plain Python if and Chapter 6's route function. Temporal does not replace that logic; it wraps each step the logic calls in a unit the server can track, retry, and resume.

Retries are a policy, not a try/except you write by hand

Activities fail: rate limits, timeouts, flaky networks. Each activity carries a retry policy (an initial interval, a maximum number of attempts), and the server retries the activity for you according to it, without your workflow code writing a single try/except. The lecture's own flaky demo, a select_emoji activity engineered to fail on its first two attempts and succeed on the third, needs no change to the workflow at all: retry_worker_task simply attaches the retry policy, and the server's log shows attempt 1, attempt 2, attempt 3, success.

Durability means a workflow can out-sleep its own worker

The clearest demonstration of what “durable” buys you: start a reminder workflow that sleeps for ten seconds before firing (await workflow.sleep(seconds)), then deliberately kill the worker process while it is still asleep. Query the workflow's status, and it still reports RUNNING, with no worker alive to run it. Bring a new worker back, and when the timer fires, the workflow finishes exactly as if nothing had happened, because the execution history and state live in the server's storage, never only in the worker's own memory.

Temporal answers “did this step actually finish, and can it resume?” Observability tools answer a different question: “what did my system actually do, and why did it go wrong?” The lecture names three: MLflow (an experiment and trace viewer that shows a call's inputs, messages and outputs side by side, built originally for tracking ML experiments), LangSmith (LangChain's own production trace viewer, with latency, token counts and feedback scores per run), and Langfuse (open-source tracing with model latency, usage and cost dashboards across a whole deployment). All three do the same underlying thing: record every model call, every tool call, every state change and every error, so that a failure three steps deep in an agent's run is a trace you can read, not a mystery you can only guess at.

Make failures observable and recoverable

The lecture's own list of what this buys you, in order: trace model calls, tool calls, state changes and errors. Set timeouts, retry limits and execution budgets, so a stuck run stops instead of spinning forever. Log outputs and diagnose failures from the trace rather than from guesswork. Monitor the state of your system while it runs, not only after it crashes. And, where it fits, you can raise the error back to the agent itself and let it try again, the way Chapter 9's retry policy did automatically at the activity level.

An example the lecture points to, Shepherd, is a meta-agent built entirely on this idea: it reads another agent's own execution trace and intervenes on it, either live (pausing a run to fix a mistake it sees forming) or after the fact (optimizing the agent's prompt or its policy, the way Chapter 4's GEPA optimized a signature's instructions from a metric's feedback). Shepherd's whole mechanism depends on a trace existing to read in the first place.

Ground every design decision in an eval

The lecture is blunt about how to choose between the options this lecture has shown: the best way to make a design decision (workflow or agent, which harness, which optimizer) is to ground it in an evaluation. Build a proper eval before you need one. Choose your metrics with the actual job in mind: accuracy, cost, latency, each a different axis, Chapter 4's own metric only one example. Test representative tasks, edge cases and failure scenarios, not only the easy cases a system was obviously going to pass.

One warning earns its own slide: unless you are building a general coding-agent harness, or specifically testing a general agent-system improvement, standard public coding benchmarks (the slide names Terminal-Bench, SWE-bench, DeepSWE) are not the ladder to climb. A custom domain eval, built on your own task and your own failure modes, tells you far more about whether your system works than a leaderboard built for a different job.

Keep it simple, and justify every step toward the agent end

The lecture closes its own best-practices section by pointing straight back at Chapter 1's line. You might not always need to build an agent: sometimes a workflow is the right tool for the task, because its fixed order is easier to test, trace and retry. When you do build an agent, think carefully about which tools it gets, since Chapter 8's harness comparison showed that more tools and more autonomy are not automatically better. And if you have enough data, you may even be able to optimize a model end to end, the way Chapter 4's GEPA optimized a prompt from 30 examples and a metric.

Two challenges the lecture names as the cost of all of this: abstractions lift you further from the underlying system details, so you must still know what is going on underneath, which is the whole reason this lecture opened with backpropagation by hand versus loss.backward(). And building on frameworks adds dependencies: external libraries you must keep maintained and upgraded, a cost that is real even when the abstraction itself earns its place.

Engineering an agent is a cycle, not a single build: build, run, evaluate, improve, and back to build. Everything in this lecture is a tool for one of those four steps. Start simple, justify every step toward the agent end of Chapter 1's line, design for the failure you know will happen, and use the evidence from an eval, not a hunch, to decide if a change actually helped.

The lecture's own numbers

Terms, in the order they appeared

A checklist before you reach for an agent

• Could a fixed workflow, or a single well-contexted call, do this job? If so, build that first.
• Do you have examples and a metric? An optimizer can improve a prompt before you write a better one by hand.
• Have you given the agent only the tools the task actually needs, not every tool available?
• Is every call, tool use and error traced somewhere you can read it while the system runs?
• Does a long-running step need to survive a crash? If so, it belongs on a durable-execution layer, not only in local memory.
• Is your eval built on your own task's edge cases, not a public leaderboard built for someone else's?

The core, as runnable code

python# DSPy: a signature, a module, an optimizer
class ExtractEvent(dspy.Signature):
    """Extract event details from an email."""
    email: str = dspy.InputField()
    event_name: str | None = dspy.OutputField()
    date: dt.date | None = dspy.OutputField()

extract = dspy.ChainOfThought(ExtractEvent)
gepa = dspy.GEPA(metric=event_metric, max_metric_calls=100, reflection_lm=dspy_lm)
extract_optimized = gepa.compile(extract, trainset=trainset)

# LangGraph: state, nodes, a conditional edge
builder = StateGraph(State)
builder.add_node("extract", extract_node)
builder.add_node("pick_emoji", pick_emoji_node)
builder.add_conditional_edges("extract", route)
graph = builder.compile()

Where this sits, and where to go next

Frameworks are the middle of the stack this course keeps returning to: under them sit the model internals from Lecture 2, LLMs for Builders, and the tool-calling loop from Lecture 4, Tool Use & Function Calling, which every signature, module, node and harness in this lesson ultimately calls down into. Above them sit the topics this course turns to next: agent memory architectures, multi-agent systems, and the evaluation infrastructure Chapter 10 already previewed. Our own Agent Architectures and Harness Engineering lessons go deeper on the shapes a harness can take once you are ready to build one from scratch rather than adopt one.

DSPy's own paper, which this chapter drew its signatures, modules and optimizers from, is Khattab et al., “DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines” (ICLR 2024, arXiv 2310.03714). It is the natural next read if you want the full optimizer zoo (GEPA is only one of several) and the formal argument for why compiling a pipeline against a metric beats hand-tuning a prompt.

Go back to the hero at the top and drag the selector one more time, now that every step of what it is showing has a name. Then press Present to teach this lecture back in your own words, or Teach to derive a signature, a graph, or a retry policy on the whiteboard.