A world model lets an agent practice in its head. A diffusion or flow model turns noise into sharp data. Both work by learning one small step at a time, and fewer steps cost less.
Pick a sampler and a number of steps, then press Generate. Each line is one sample travelling from noise on the left to data on the right. Then we build, piece by piece, how an agent learns inside its own imagination and how a network turns noise into data.
You need a little probability (an average and a bell curve) and a little calculus (a slope). We build the rest from zero.
Based on Lecture 11 of Penn CIS 6280: World Models (Fall 2026), taught by Jiatao Gu, and the papers it builds on: Dreamer, DDPM, DDIM and Flow Matching. Lecture slides. Every idea is re-taught here in our own words and devices; it follows Lecture 10, Policy and Value.
Toy world: the data are exactly two values, +1 and −1, equally likely. The network is the exact best denoiser for that world, worked out by hand (Chapter 8), so only the sampler differs. The noise schedule (100 levels, β from 0.001 to 0.12) is illustrative. Real image models are not this clean; the shape of the trade is the point.
Chapter 0
See why a learned world model lets an agent try a thousand futures without breaking anything, and what the lecture builds on top of it
A robot arm learns to pour water into a cup. Every real attempt takes ten seconds and spills a little. After a hundred attempts the table is wet and the robot is tired. A person learns the same task faster, because a person can picture the pour first. We tilt the jug in our head, see the water arrive too fast, and slow down before we touch anything.
That picture in the head is a world model: a learned program that predicts what happens next if you take an action. Earlier lectures built one. This lecture asks what to do with it. There are two answers, and they take the two halves of the lecture. First, an agent can learn behaviour entirely inside the model’s imagination (the Dreamer method). Second, the model needs a way to draw sharp data, such as images, and diffusion and flow matching are the tools for that.
The lecture opens with a recap. The model keeps a latent state, a short list of numbers that sums up the world right now, written xt = (ht, st). The part ht is a recurrent memory: it carries the history. The part st is a stochastic state: a random draw that stands for what the memory cannot know. The lecture calls this model an RSSM (recurrent state-space model), the design from the PlaNet paper.
The state is updated in two different ways. When a real observation arrives, the model uses that evidence to refine the state. The result is called the posterior: the belief after seeing. When no observation exists, because we are predicting the future, the model supplies actions and uses its own transition rule. That belief is called the prior: the belief before seeing. History informs the present state. Actions decide the predicted next states.
Training uses recorded sequences of observations and actions. The model infers a state for each moment and then predicts what was recorded. Three signals teach it: rebuild the observation, predict the reward, and keep the prior close to the posterior. These updates change the world model. The recorded actions only supply its inputs.
The last recap slide shows planning with the learned model. At the current inferred state, propose candidate action sequences, predict their consequences, and score the outcomes. Then improve the candidates. When the search ends, execute only the first action, look at the world again, and plan again. Planning changes the candidate actions while the world model stays fixed.
Imagination is not free. When the model has no new observation, its belief gets blurrier with every step it predicts. Move the slider and watch. For six steps the model sees the real world and keeps its belief tight. After that it only imagines.
A ball moves up and down. Blue dots are real observations, only the first six. The warm line is the model’s belief and the warm band is how unsure it is. Drag to add imagined steps.
Toy numbers, illustrative: each real observation has noise variance 0.04 and each step adds 0.05 of model uncertainty. The model is a one-dimensional tracker, not an RSSM; it shows only why a longer imagined rollout is a less sure rollout.
The band is tight while real observations arrive. Then it opens, step by step, and the model’s line drifts away from the true path. A model that predicts far into the future is less sure of every step. This one fact will come back in Chapter 1, where it decides how long an imagined rollout should be.
The lecture starts with two announcements. Project information for the final project will be released on October 16, and Assignment 2 is due on October 19.
The lecture has three parts, and this lesson follows them in order: how an agent learns inside a world model (Dreamer), how a network turns noise into data (diffusion), and how it can learn the motion directly (flow matching). Each chapter says which course slides it covers.
Chapter 1
Add up the rewards of an imagined future, discount the late ones, and see what a short rollout leaves out
You can take a coin today or wait a week for three coins. Most people want some patience in the answer, but not unlimited patience: a coin in hand is worth a little more than a coin promised. An agent faces this choice at every step. It needs one number that says how good a whole future is, so it can compare two futures and pick the better one.
That number is the return: the total reward the agent collects from now on. A reward is the score the world hands out after a step, such as +1 for reaching the goal or −1 for a spill. The lecture calls return “the quantity an action should improve”. An action is good if it raises the return.
Late rewards count less. We multiply a reward that arrives k steps later by a number γ (gamma), called the discount factor, raised to the power k. If γ is 0.9, a reward one step away counts as 0.9 of its face value, two steps away 0.81, three steps away 0.73. A γ near 1 is a patient agent. A γ near 0 is an agent that only cares about the next step.
Take the predicted rewards 1, 0, 2, 0, 3 and γ = 0.9. The return is the first reward at full weight, then each later reward times 0.9, 0.81, 0.73 and 0.66:
1 + 0.9 × 0 + 0.81 × 2 + 0.73 × 0 + 0.66 × 3 = 1 + 0 + 1.62 + 0 + 1.97 = 4.59
The lecture adds one remark here: to compare predicted futures, estimate their return. That is all planning and policy learning need from the numbers.
Where do the rewards come from? From the world model. Start from the real history and infer the starting state x0 (the posterior from Chapter 0). Then supply actions a0, a1, and so on. At each step the model draws the next state and predicts its reward:
xt+1 ~ pθ( · | xt, at ), r̂t+1 = rθ(xt+1)
Read it as a recipe with two lines. Line one: given the state and the action, draw the next state. Line two: a reward head reads that new state and says how much reward it carries. The little hat on r̂ means predicted, not real. The indices x0 to xH are local to this rollout. They count steps inside the imagination, not steps in the real world. No future real observation exists, so nothing real corrects the rollout on the way.
A rollout cannot run forever. Each extra step costs compute, and Chapter 0 showed that each extra step also makes the prediction less sure. So we stop after H steps. H is the rollout cutoff, also called the imagination horizon. At state xH explicit prediction ends.
But the task goes on. Rewards can keep arriving after xH. The cutoff is a limit of our computer, not a fact about the world. It leaves a missing tail of return. Try it: the device holds a made-up future with a small steady reward and one big prize, then shows how much of the return the rollout can see.
Each bar is one predicted reward after discounting. Yellow bars sit inside the rollout. Grey bars are the missing tail. Drag the horizon, then the patience dial.
Illustrative future, not from the lecture: a reward of 0.5 at every step, except 8 at step 14. The bar for step k is γk−1 times its reward, matching the lecture’s first rollout term at full weight. The totals come from adding the 20 bars.
With H = 6 and γ = 0.95 the rollout counts 2.65 of a total return of 10.27, so 74% hides in the tail. The big prize at step 14 is invisible. A planner that scores futures by this partial return would pick the plan with small steady rewards and never walk toward the prize. Push H to 14 and the same planner sees 87% of the return and changes its mind.
Now lower the patience. At γ = 0.5 the whole 20-step return is only 1.00, and the prize is worth almost nothing, so a short rollout loses very little. The tail matters in proportion to how much patience the task needs. This is the symptom, the measurement and the question for the next chapter: a task that needs patience needs a long horizon, and a long horizon is expensive and unsure. Can we keep H small and still estimate the tail?
Chapter 2
Train a model to guess the reward still to come from any state, and use that guess to finish every short rollout
A chess player looks at a board in the middle of a game and says, “I am a bit ahead.” She did not play out the rest of the game in her head. Years of games taught her to read a position and judge it in one glance. That glance is exactly what the missing tail of Chapter 1 needs.
The value model Vψ(x) does the same thing for an agent. It reads one state x and outputs one number: the return the agent expects to collect from there on. The lecture states the difference sharply. The world model predicts a sequence of future states and rewards. The value model predicts one number: the reward remaining from a state. The Greek letter ψ (psi) stands for the value model’s weights. One caveat: a value belongs to a way of acting. It assumes a specified way of continuing, so the same state has a different value for a careful agent and a reckless one.
Now join the two pieces. Roll the world model out for H steps and collect the predicted rewards. At the last state xH, ask the value model for the tail. Add it, discounted by the steps it sits behind:
The lecture is careful about two names. G is calculated from a rollout. V is predicted directly from a state. Both refer to the same way of continuing to act. A short rollout plus a good value gives a long-horizon estimate without a long rollout.
How does Vψ learn? Treat it as plain regression. Regression means: fit a number to a target by shrinking the squared error. The prediction is Vψ(xt). The target is the return estimate Gt computed from a rollout.
Here is the part that surprises people. The target Gt itself uses the value model. The simplest target takes one imagined step: the next predicted reward plus the discounted value the model already assigns to the next state.
Gt = r̂t+1 + γ Vold(xt+1)
The word “old” marks the copy from before this update. We use it to build the target, hold the target fixed, and then fit the new V to it. A longer rollout uses more predicted rewards first and asks the old value only for the tail. This idea is called bootstrapping: a guess is improved by using an earlier guess. It sounds circular. It is not, because each target contains one real step of new information, a reward that the world model learned from the real world.
The lecture states the consistency rule the fit enforces: the current value should agree with the next reward plus the value of the next state, V(xt) ≈ r̂t+1 + γV(xt+1). It also asks what anchors the rule and what can still go wrong. The anchor is that reward predictions come from a model trained on real rewards. What can go wrong is that the model and the next-state value can both be wrong. Fitting the rule moves reward information from state to state. It does not make the predictions right by itself.
Learning therefore repeats in two moves. First calculate targets by rolling out the model and using the current value for the tail. Then fit the value to those fixed targets. Then recompute the targets with the better value and fit again. G is a temporary target. V is the reusable predictor we keep. Run the cycle on a small chain below.
Eight states in a row. Only the step from the last state into the goal gives a reward of 1. Yellow bars are the learned values. Dashed marks are the true values. Choose a rollout length, then press Fit once several times.
Illustrative chain with γ = 0.9, so the true value of state t is 0.97−t. Each press computes every target from the old values with the chosen rollout, then sets the new values equal to those targets (a perfect fit). A real value network fits only approximately, and the rewards come from the world model.
With H = 1, each fit moves the news back by one state: after three fits only the last three states know about the goal, and the start state still says 0.00. With H = 4 the news moves four states per fit, so two fits give every state its exact value (0.48 at the start). That is the lecture’s remark made visible: a longer rollout uses more predicted rewards before it asks the value model for the tail, so the value learns faster. The price is the unsure rollout from Chapter 0. Value learning and rollout length trade against each other.
Chapter 3
Score plans with a value tail, then see why an agent trains a reusable policy instead of searching again at each decision
A taxi driver in a new city asks the map app at every corner, and the app searches all the routes again. A driver who has lived in the city for ten years does not ask. She has a habit: at this corner, in this traffic, turn left. The habit costs years to build and nothing to use. The map app costs nothing to build and a search every time.
An agent with a world model can act like either driver. This chapter shows the app first, a planner called CEM that now uses the value tail from Chapter 2. Then it shows the habit, called a policy, and how the agent trains one.
Lecture 10 introduced the cross-entropy method (CEM), a search for a good action sequence. Here is the plain idea. Keep a bell curve over action sequences. Draw many candidates from it. Roll each one out in the world model and score it. Keep the best few, called the elites. Refit the bell curve around the elites, and repeat. Each round the curve narrows around the plans that scored well. After the search, execute the first action of the best plan, observe, update the state, and search again.
The new piece is the score. The lecture scores a candidate sequence U with all H predicted rewards plus the tail value at the end:
The slide marks this as an illustration: a value-augmented CEM, not the original PlaNet scoring rule. Its meaning is simple. Score the candidate’s full action prefix, then use the value for what follows. Try it. To keep the picture flat we squeeze a whole action sequence into one number, a, on the horizontal axis. The curve is the score.
The curve is the score of each plan. Dots are the 24 candidates of the last round; the 6 elites are filled. The bell at the bottom is the search distribution. Press Refit a few times, then switch the value tail on and start again.
Illustrative score, not from the lecture: a near prize of 0.8 at a = −1.5 that the explicit rewards see, and a larger prize of 2.2 at a = 1.8 that only the value model sees (with γH set to 1 for clarity). The search starts at a bell with centre 0 and spread 1.5 and keeps 6 of 24 candidates.
Without the tail, every round pulls the search toward the near prize. After six refits the plan sits at a = −1.50 with a score of 0.80, and the bell has collapsed to a spike. With the tail the search is the same procedure on a different score. The elites of round two already sit near a = 1.79, and by round four the plan holds at a = 1.80 with a score of 2.20. The planner did not get smarter. The value tail told it about a prize its rollout could not reach.
CEM runs a full search at every single decision, many candidate rollouts each time. The second row of the lecture’s slide shows the alternative. Train one policy πφ(a | x): a network that reads a state and outputs a distribution over actions, written at ~ πφ(· | xt). The parameters φ (phi) are shared across all states. We optimise them during training. At run time the agent just calls the policy, with no search.
The lecture names the roles. The policy is the actor: it chooses actions. The value model is the critic: it judges states. Together they are actor-critic learning. One more consequence follows from Chapter 2’s caveat. The imagined actions and the value targets now follow the current policy, so the critic is always judging the actor as it is today.
How should the actor change? The first way treats the imagined return as a fixed learning signal. Sample an action and hold that draw fixed. Estimate its return G with the model. Compare with the critic’s guess for that state, Vψ(xt). The difference is the advantage, A = G − Vψ(xt): how much better this action did than the critic expected.
If A is positive, raise the log probability of the action. If A is negative, lower it. No derivative passes through the world model.
LSF(φ) = − Efixed batch [ log πφ(at | xt) · At ]
A worked step. The actor picks one of two actions with π(a = 1) = 0.60. It samples a = 1 and the model returns G = 5, while the critic expected V = 3. So A = +2. The gradient of log π(1) with respect to the actor’s logit is 1 − 0.60 = 0.40, so the weighted gradient is 0.40 × 2 = 0.80. With a step size of 0.5 the logit rises by 0.40, from 0.405 to 0.805, and π(a = 1) goes from 0.60 to 0.69. A good surprise made the action more likely.
This works, but it has a cost. Each sampled action carries only one number, its advantage, so the signal is noisy. Chapter 4 measures exactly how noisy, and shows the other route.
Chapter 4
Compare two ways to improve an actor inside a world model, and see why Dreamer sends the slope straight through the model
Two coaches teach you archery. The first watches each shot and says only “better than usual” or “worse than usual”. You must guess what to change. The second coach can see the physics. She says, “Aim two degrees higher.” Both coaches help you improve. The second one needs far fewer shots, because every shot carries a direction, not only a grade.
A world model can be the second coach. It is a neural network, and a neural network is a chain of smooth arithmetic. That means we can ask it a calculus question: if I nudge this action a little, how does the predicted return change? The answer is a gradient, a slope that points toward a higher return. This chapter shows how Dreamer gets that slope and what it costs to keep it clean.
Look at the lecture’s picture of the Dreamer actor update. Going forward, the policy πφ proposes an action at. The world model fθ predicts the next state. The reward head rθ and the value model Vψ read it. The return calculation of Chapter 2 produces Gt. That is the imagined return.
Going backward, we ask how G changes if the action changes. The world model changes the predicted future when the action changes, and the return changes with it. We backpropagate through that whole computation, which is the chain rule applied step by step, until the slope reaches the policy parameters φ. The actor loss is simply the negative of the return, so lowering the loss raises the return:
Lactor(φ) = − E [ G0 ]
Chapter 3 gave the first route, the score-function gradient. It treats the imagined return as a fixed learning signal, with the loss − log πφ(at | xt) · sg(At). The second route is the dynamics gradient. It keeps how the imagined return changes with the action, with the loss −Gt. Both routes use the same world model, the same value model and the same imagined experience. Only the way they compute the actor gradient differs. The lecture says it plainly: original continuous-action Dreamer uses the dynamics-gradient route.
Why choose it? Run both many times and see how steady each answer is. The device uses a small made-up return, G(a) = 3 − (a − 2)2, with an actor whose action is a Gaussian draw around a centre μ = 0.5 with spread σ = 0.8. The true slope of the expected return with respect to μ is 3.00. Each estimator tries to find it from N sampled actions.
Each dot is one try: sample N actions, estimate the slope. The green line is the true slope, 3.00. Drag N, then press Draw again to see fresh tries.
Illustrative return G(a) = 3 − (a − 2)2, actor mean 0.5, spread 0.8. The score-function estimate uses the exact expected return (0.11) as the baseline, a perfect critic, so it gets the best case. The spreads are computed from 400 seeded tries; the 40 dots are fresh tries.
Both clouds centre on 3.00, so both routes are unbiased: on average they give the right slope. But with N = 8 the score-function tries scatter with a spread of 1.78, while the dynamics tries scatter with 0.54. The dynamics gradient is about three times steadier. To match its steadiness the score-function route needs roughly ten times more samples. That is the cost of grading a shot instead of correcting it.
One worked draw. Take one noise value ε = 0.5, so the action is a = 0.5 + 0.8 × 0.5 = 0.90. The dynamics route computes the slope of G at that action, −2(0.90 − 2) = 2.20, and passes it to μ unchanged because a moves one-for-one with μ. The score route gets G(0.90) = 1.79, then the advantage 1.79 − 0.11 = 1.68, then the log-probability slope (a − μ)/σ2 = 0.4/0.64 = 0.625, and multiplies: 1.05. One draw each, two different answers, and both are noisy. Only many draws show who is steadier.
There is a subtle rule here. The actor update changes only φ. The world-model parameters θ and the critic parameters ψ stay fixed, because this step is not their turn to learn. But the derivative must still flow through them, since the slope travels from G, back through the reward, value and dynamics, to the action. The lecture states it: keep the input derivatives through dynamics, rewards and values, and update only the policy parameters. In the original implementation, each actor’s state input is detached, meaning gradients do not travel back into the earlier states as inputs to the actor, while the propagation of the state through the model stays intact.
A snag: you cannot take a slope through a random draw. The lecture solves it with the reparameterization trick. Move the randomness outside the network. Draw a fixed noise value ε from a standard bell curve. Then build the action from the actor’s outputs:
The world model has its own random state, so the same trick applies there. The next stochastic state is st+1 = μθ(ht+1) + σθ(ht+1) ⊙ ξt, with ξt a fixed draw. Here ht+1 is the next recurrent memory, which depends on the action. The whole RSSM state is xt = (ht, st). Reparameterization keeps the derivative from the next latent state back to the action. The failure mode, in order: without it, the slope breaks at the first random draw (symptom), the actor gets no usable gradient (measurement), and the fix is moving the noise outside the network.
Chapter 5
Put the world model, the critic and the actor in one cycle, and see why a cut-off rollout is not the same as a finished task
A pilot flies a real plane for an hour and records everything. Back on the ground she builds a flight simulator from the recordings. Then she lands the plane in the simulator a thousand times, learns from each landing, and flies again. The new flight shows where the simulator was wrong, so she fixes the simulator and practises again. Real flights make the simulator better. Simulated landings make the pilot better.
That is Dreamer, in one paragraph. The lecture shows an original demonstration, a learned actor that controls a body in a visual-control task. Training and execution are different jobs. In training, the agent improves its actor and critic using imagined returns. In execution, it infers the state from real observations and then simply picks the actor’s action. No search and no imagination run at that time.
The lecture sums up the training as three stages with three different signals and three different sets of parameters.
| Stage | Signal | Updates |
|---|---|---|
| World model | Real observations and rewards | θ |
| Value (critic) | Imagined return target | ψ |
| Policy (actor) | Expected imagined return | φ |
World model: fit real experience. Critic: fit fixed return targets. Actor: improve the imagined return. The three updates alternate. Each has its own objective, and each changes only its own parameters. That is how Chapter 4’s rule, “update φ, keep θ and ψ fixed”, fits into a bigger plan.
The lecture draws the cycle as eight boxes. Step through it below and watch which parameters move at each stage. The key sentence from the slide: real experience improves the model, and imagination improves behaviour.
Blue boxes touch the real world. Warm boxes happen only inside the model. Press Next stage and read which of θ, ψ and φ changed.
The eight stages and their order follow the lecture’s loop slide. The update counts are just a tally of how often you have passed each stage in this device; they are not training steps from the paper.
Read the loop in words. Real replay supplies posterior starting states: recorded histories are encoded into states x0 that the imagination starts from. Then the model and the actor generate imagined trajectories and returns. After the actor and critic update, real execution with the new policy supplies fresh evidence, and that goes back into the replay. Notice how few stages touch the real world: execution, new observations and replay. The others run on the model, and that is the whole trade.
Every imagined number comes from the world model, so its mistakes are not harmless. The lecture’s next slide shows the original model evaluation: held-out observations next to the model’s decoded predictions and the differences. The message: errors in predicted states or rewards affect the targets used by both the critic and the actor. A caution is printed on the slide. This evaluation uses recorded future actions. It is not a rollout of newly selected policy actions. So it checks the model, not the agent.
The slide after that is a figure from the Dreamer paper (its Figure 4). It plots episode return against the imagination horizon on two tasks, Cartpole Swingup and Cheetah Run, for three methods: Dreamer with a value, an actor without a value, and PlaNet planning. The lecture’s reading: in these tasks, predicting the tail value reduces the sensitivity to the rollout horizon. In other words, the value model lets you stop imagining sooner. We can see the same effect on a toy.
The last Dreamer slide separates two ideas that sound alike. A true episode end means the task is over: no future reward remains, so the return is just the predicted rewards, G = rewards. A rollout cutoff H means only our prediction stops while the task continues, so we must estimate what remains: G = rewards + tail value. Mixing them up is a bug. If you add a tail value to a finished task, you invent reward that can never arrive.
Every step pays a reward of 1. Choose whether the task goes on forever or ends at step 12. The dashed green line is the true return. Drag the horizon and the value error.
Illustrative task: reward 1 per step, γ = 0.95. If the task continues, the true return is 1/(1 − 0.95) = 20.00. If it ends after step 12, the true return is 9.19. The value error scales the value model’s tail by the chosen percent.
When the task continues, H = 5 with rewards alone gives only 4.52 of the true 20.00, and the true tail is worth 15.48. A perfect value adds it back exactly. An error of +20% in the value adds 3.10 too much. Even then the estimate is far better than using no tail, and the error shrinks as H grows, because the value’s share is discounted by γH. This is the horizon-sensitivity story in miniature.
Switch to the task that ends at step 12. The right return is 9.19. If the value model still says “20 remain”, then any rollout of 12 steps or fewer reports 20.00. That is more than double the truth, and every plan looks the same. The symptom is an actor that does not care when the task ends. The measurement is the gap between G and the real return. The fix is the lecture’s: when the task truly ends, drop the tail and use the predicted rewards only.
Chapter 6
Turn data into noise on purpose, in many small steps, and learn the jump to any noise level in one line
Copy a photograph on a bad photocopier. Copy the copy. Copy that copy. Each pass adds a little grain and fades a little detail. After a hundred copies, only grey static remains. Destroying a picture is easy. The hard question is the reverse: from static, can we walk back to a picture? If a network could learn each backward step, then static plus the network would be a picture factory.
That is the idea behind diffusion models. This chapter builds the easy half, the destruction, and shows why it is designed so carefully. Chapters 7 and 8 build the learned reverse walk.
The lecture starts from the variational autoencoder, which Lecture 7 covered. A latent variable z is a hidden short code. The decoder pθ(y | z) turns a code z into data y, such as an image. The encoder qφ(z | y) goes the other way: it guesses the code for a given image. To generate, we draw z from a simple prior p(z) (a plain bell curve) and decode it, so generation uses p(z) pθ(y | z). In the lecture’s words, a latent variable connects a simple prior to the data distribution.
Now stack many of these. The lecture draws a hierarchy: noise, then latent yJ, down through latent yj−1, to the data y0. The data are y0. The in-between y1 to yJ are all latent variables. We generate through a sequence of simple conditional steps:
The index j counts corruption steps. It does not count environment time. A beginner often mixes the two, so keep them apart: the earlier chapters stepped through the world, and this chapter steps through noise levels.
Training needs an inference path from data to latents: a way to turn each image into its noisy versions. VAEs learn this path. A diffusion model chooses it, and fixes it: gradually remove the data signal. The lecture shows a cat photo fading through “less signal”, “mixed” and “mostly noise” to pure noise. Each step is a Gaussian forward transition:
The list β1, …, βJ is the noise schedule. It is fixed before training starts. Why shrink the signal by √(1 − βj) instead of just adding noise? If the data have a spread (variance) of 1, the new sample has variance (1 − βj) + βj = 1. The overall scale stays steady across the whole chain, so the end of the chain is a clean standard bell curve and the network always sees numbers of the same size.
Training will need a noisy sample at a random level j. Running j copier passes each time would be slow. Luckily, Gaussians combine. Write αj = 1 − βj and ᾱj = α1 × α2 × … × αj. Check two steps by hand. The first gives y1 = √α1 y0 + √β1 ε1. The second gives y2 = √α2 y1 + √β2 ε2. Substitute, and the signal is √(α1α2) y0. The two noise pieces are independent, so their variances add: α2β1 + β2 = α2(1 − α1) + (1 − α2) = 1 − α1α2. The pattern continues to every j:
This is the lecture’s point: training can sample any noise level directly. The device checks it numerically. The schedule is a made-up one with 100 levels (the paper’s own schedule is similar in kind). Drag j and compare 2,000 chains of j small steps with the one-jump formula.
Top: a clean signal (blue) and its noisy version at level j (purple dots). Bottom: where one pixel with value 1 ends up, from 2,000 chains of j small steps (outline) and from the one-jump formula (fill).
Illustrative schedule: βj rises in a straight line from 0.001 to 0.12 over J = 100 levels. The signal is a made-up wave; the noise draw ε is fixed so the picture holds still as you drag. The 2,000 chains use seeded random numbers.
At j = 50, ᾱ = 0.21, so the signal keeps 0.46 of its size and the noise is 0.89. At j = 100 the signal keeps only 0.04 and the noise is 0.999, which is nearly pure static. The two histograms lie on top of each other at every level: the chain of small steps and the one-line jump give the same bell curve, centred on √ᾱ with variance 1 − ᾱ. This is why diffusion training is cheap: it never walks the chain, it jumps.
Chapter 7
Follow the likelihood bound through the noise chain until the whole objective turns into predicting one average
A student sits an exam. In the study hall the teacher hands out the answer key with every question. In the exam hall there is no key, only the question. Training a diffusion model is like the study hall. We start from a clean image, so we know what the right backward step looks like. Generating is like the exam hall: the network gets only a noisy sample. The mathematics of this chapter explains how to turn the answer key into a lesson that still helps in the exam.
Chapter 6 chose the forward chain. Now we need the learned reverse step, pθ(yj−1 | yj). The lecture gives it a Gaussian shape: pθ(yj−1 | yj) = N( μθ(yj, j), σj2 I ), for j ≥ 2. Generation starts from yJ ~ N(0, I). One shared network predicts the reverse mean at every noise level, and it receives j as an input so that it knows how noisy its input is. The lecture’s one-line reason this works: learning supplies the information missing from a noisy sample.
How do we train it? We want the model to give the real images a high probability, so we want to maximise log pθ(y). But that number is out of reach, because it sums over all the hidden chains. The VAE trick, recapped on the slide, is to maximise something always smaller than it. That something is the ELBO, the evidence lower bound:
We maximise the ELBO, or equivalently minimise its negative. For a chain of latents the same principle applies. The “inference” part is now the fixed forward chain q(y1:J | y0). The “generative model” part is the prior and the learned reverse steps pθ(y0:J):
ELBO(y0) = Eq(y1:J | y0) [ log pθ(y0:J) − log q(y1:J | y0) ] LnegELBO = − ELBO
Now substitute both chains. The log of a product is a sum of logs, so the bound becomes log p(yJ) plus a sum of log pθ reverse terms, minus a sum of log q forward terms. Each term belongs to one transition in one of the two chains. And only the reverse model depends on the learned parameters θ. That is a big simplification: the forward terms are constants during training.
The next move regroups the forward chain. Instead of going from y0 forward, write it backward, with the clean sample as a given: q(y1:J | y0) = q(yJ | y0) ∏j=2J q(yj−1 | yj, y0). With that form the objective splits into three kinds of terms:
LnegELBO = Lrec + ∑j=2J Lj + Lprior, Lj = Eq(yj|y0) [ DKL( q(yj−1 | yj, y0) ∥ pθ(yj−1 | yj) ) ]
Lrec is the data reconstruction at the clean end. Lprior checks that the final noise matches the prior, and it has no learned weights. The interesting terms are Lj: one KL per noise level that compares the true reverse step with the learned one. This is the answer key. The true reverse step q(yj−1 | yj, y0) uses the known clean sample y0. The learned step pθ(yj−1 | yj) sees only the noisy sample. Given the clean sample, the Gaussian reverse target is tractable, which means we can write it down exactly.
How do we get it? By Bayes’ rule, q(yj−1 | yj, y0) is proportional to q(yj | yj−1) × q(yj−1 | y0). The first factor is the forward step. The second is the marginal from Chapter 6. Both are Gaussians, and the product of two Gaussians is a Gaussian. Multiply, then complete the square (the algebra trick that reads off a mean and a spread). The result:
See the multiplication happen. The device fixes a clean sample y0 = 1, lets you choose the level j and the noisy sample yj, and draws the two Gaussian factors as curves over the previous sample yj−1. Their normalised product is the answer key. You can then set a model mean, the guess of μθ, and read off how far the KL is from 0.
Purple: the forward step seen backward, and the marginal from the clean sample. Blue: their product, the true reverse step. Warm dashed: your model’s guess. Every curve is scaled to the same height so you can compare shapes.
The clean sample is fixed at y0 = 1 and the schedule is the illustrative one from Chapter 6. The model’s spread is set equal to the target’s, so the KL is just (μ̃ − μθ)2 / (2β̃), in nats. The grid product uses 600 points.
At j = 50 and yj = 0.80, the target is centred at 0.80 with a spread of 0.24. Multiplying the two curves on a grid gives the same numbers, which checks the formulas. Now move the model mean to 0.50. The KL is 0.76. At 0.70 it falls to 0.08, and it is nearly 0 where the model mean meets the target. Notice what moved the number: only the distance between the two means. That is the next step of the lecture.
For two Gaussians with the same shape of covariance, the KL has a closed form. With target N(μ̃j, β̃j I) and model N(μθ, σj2 I):
DKL( q ∥ pθ ) = ∥ μ̃j − μθ(yj, j) ∥2 / (2σj2) + (d/2) [ β̃j/σj2 − 1 − log(β̃j/σj2) ]
Here d is the number of values in one sample, such as the number of pixels. If we fix the variances σj2 to positive constants, the second term contains no θ. It is a constant, so it cannot affect training. Only the first term is left: the squared distance between the target mean and the predicted mean, divided by 2σj2. In the lecture’s words, learning the reverse distribution reduces to predicting its mean.
The whole objective is now a weighted regression: LnegELBO = Lrec + ∑j=2J E [ 1/(2σj2) ∥ μ̃j − μθ(yj, j) ∥2 ] + C, where C holds the terms that do not depend on θ. The factor 1/(2σj2) is a noise-level weight: it says how much each level matters. The lecture ends with a warning worth keeping. The weighted mean error comes from the ELBO. Reweighting it is a separate choice, one that changes which noise levels the network cares about. The next chapter makes exactly that choice.
Chapter 8
Choose what the network predicts, run one supervised training step by hand, and read the full sampling loop
A photo restorer receives a picture with grain added. Nobody asks her to paint the picture from scratch. Her job is smaller: estimate the grain, then subtract it. If she can spot the grain, the clean picture falls out. A diffusion network works the same way. It looks at a noisy sample and answers one question: what noise was added?
Chapter 7 left the network’s job as “predict the mean of the reverse step”. But a noisy sample is built from two unknowns: yj = aj y0 + bj ε, where aj = √ᾱj is the signal weight and bj = √(1 − ᾱj) is the noise weight. If you know yj and any one unknown, you can compute the other. So the network can predict any of four things, and the lecture lists them:
| Network output | Training target | Meaning |
|---|---|---|
| μθ | μ̃j(yj, y0) | Mean of the next reverse step |
| εθ | ε | Noise added to the clean sample |
| ŷ0,θ (x0 prediction) | y0 | The clean data sample |
| vθ (diffusion v) | aj ε − bj y0 | A signal and noise combination |
They are equivalent ways to parameterise one network. The lecture’s own algorithms use noise prediction, with the mean recovered by
One caution from the lecture: plain squared error on different outputs gives different weights across noise levels. The choice of target is the choice of what the loss emphasises, even though the networks are equivalent.
Training uses a simple recipe. Draw a clean sample y0 from the data. Draw a level j at random from 1 to J. Draw noise ε. Build yj with the jump formula. Ask the network for εθ(yj, j). Compare with the true ε and update θ with a gradient step on ∥ ε − εθ(yj, j) ∥2. This is one supervised pair: the input is (yj, j) and the label is ε. A training iteration samples one noise level, not the whole chain. The lecture notes that this simplified noise loss is a reweighting of the ELBO terms from Chapter 7.
Work it with numbers. Take y0 = 1, j = 50 and ε = 0.60. At j = 50 the weights are a = 0.46 and b = 0.89. So y50 = 0.46 × 1 + 0.89 × 0.60 = 0.99. The four targets for this pair are: mean μ̃ = 0.982, noise ε = 0.60, clean y0 = 1.00, and v = 0.46 × 0.60 − 0.89 × 1 = −0.61. As a check, feed the true noise into the mean formula: (0.993 − 0.0599/0.888 × 0.60)/√0.940 = 0.982. It equals μ̃, so the parameterisations really are the same thing. Try your own pair.
The blue arrow is the signal part, a·y0. The purple arrow is the noise part, b·ε. Together they reach the noisy input yj. The warm tick is where the best network would put the clean value. Choose a clean value, a level and a noise.
Toy data of exactly +1 or −1 and the illustrative schedule from Chapter 6. The “best network” is the exact squared-error optimum for this toy, worked out in the next paragraph; a trained network only approaches it.
Where does the warm tick come from? In this toy the clean value is either +1 or −1. Given yj, the chance that it was +1 is 1/(1 + e−2ay/b²), because a Gaussian bump of width b sits around +a and another around −a. The best guess of the clean value under squared error is the average over both: E[y0 | yj] = tanh(a yj / b2). The best noise guess follows: ε* = (yj − a E[y0 | yj]) / b. In the default pair the best guess is 0.85 while the true noise was 0.60, so the loss for this pair is 0.06. A single pair cannot be predicted exactly. The network learns the average over every pair that looks the same, and that average is the best it can do. Early levels (small j) are easy, since the sign of yj tells you the clean value. Late levels are hard, because the signal is nearly gone. Slide j to 100 to see the warm tick sit near 0.
To generate, run the trained network in a loop. Start with pure noise yJ ~ N(0, I). For j = J down to 1, evaluate the same network, compute the mean, and add a fresh bit of noise: yj−1 = μθ(yj, j) + σj ξj, with a new ξj ~ N(0, I) at every step except the last, where nothing is added. Return y0. The hero at the top of the page ran this loop, with 100 network calls. Here is the whole recipe as code.
python # training: one pair, one noise level y0 = sample_data() # clean sample j = randint(1, J) # one level, uniform eps = randn_like(y0) yj = sqrt(abar[j]) * y0 + sqrt(1 - abar[j]) * eps loss = ((eps - net(yj, j)) ** 2).mean() # simplified noise MSE # sampling: the same net, J times y = randn(shape) for j in range(J, 0, -1): mu = (y - beta[j] / sqrt(1 - abar[j]) * net(y, j)) / sqrt(alpha[j]) y = mu + (sigma[j] * randn_like(y) if j > 1 else 0) return y
The cost is the catch. Every sample needs J network calls, and J is often hundreds. The next chapter asks whether all that fresh noise and all those calls are really needed.
Chapter 9
Walk down the same hill with a compass instead of a coin toss, and skip most of the steps
Two hikers walk down a foggy mountain to the valley. The first flips a coin at every step to choose a small sideways move. The second follows a compass and takes no coin tosses at all. Both reach the valley. Both can start from different places and end at different spots in the valley. The question is whether the coin tosses were needed. If the walk is the same without them, the second hiker is cheaper, steadier, and, as we will see, free to take bigger strides.
The DDPM sampler of Chapter 8 is the first hiker. It adds a fresh noise draw ξj at every reverse step: yj−1 = μθ(yj, j) + σj ξj. The lecture asks it plainly: is fresh randomness at every reverse step necessary to generate a distribution? DDIM (denoising diffusion implicit models) asks whether the randomness can come from the initial sample alone.
Why might a different walk work? Look at what training actually fits. It builds one noisy sample at one chosen level, with the marginal q(yj | y0) = N(√ᾱj y0, (1 − ᾱj) I). It never runs a whole generation path. So the training objective pins down how noisy samples look at each level, and says nothing about how a sample at one level connects to the same sample at the next. The lecture draws two such connections, “coupling A” and “coupling B”. They share the same distribution at every level but join the dots in different ways. DDIM keeps the noisy marginals and changes how the levels are connected.
Start from the training construction yj = √ᾱj y0 + √(1 − ᾱj) ε. At generation time we know yj but not y0 or ε. The network gives an estimate of the noise. From that we can solve for an estimate of the clean part:
Now to move to a lower noise level i < j with no fresh noise: keep both estimates and change the proportions. Scale the clean estimate with the signal weight of level i, and reuse the predicted noise with the noise weight of level i:
yi = √ᾱi ŷ0 + √(1 − ᾱi) ε̂, i < j
This is deterministic DDIM, the case the lecture marks η = 0. After moving, evaluate the network again at the new level. Work one move in the toy world of the hero. At level j = 50 the noisy sample is y = 0.80. The best network gives ε̂ = 0.676. The clean estimate is (0.80 − 0.888 × 0.676)/0.460 = 0.435. Move to level i = 25, where the signal weight is 0.823 and the noise weight is 0.568: y25 = 0.823 × 0.435 + 0.568 × 0.676 = 0.358 + 0.384 = 0.742. No coin toss was used. The sample moved from 0.80 to 0.74, a step that took 25 levels at once.
If every step is deterministic, then the whole sampler is one fixed function from the start noise to the final sample: yJ ~ N(0, I), then y0 = Fθ(yJ). The same start always gives the same path. A different start gives a different sample. The lecture shows this with a row of faces made by sliding smoothly from one start noise to another. A random input, not a random step, is enough for varied outputs. And it brings a bonus: because paths are smooth, a start halfway between two starts gives something halfway between their images.
Pick a start near zero, where the data could go either way. Press Run 6 times. DDPM draws fresh noise each run. DDIM does not.
Toy world from the hero: data are +1 or −1, the network is the exact best denoiser, the schedule is the illustrative one from Chapter 6. DDPM runs use fresh random numbers each time you press the button, so its counts change. The DDIM choice of levels is evenly spaced.
Press the button with DDPM chosen and a start of 0.02. The six paths wander and then settle on +1 or −1, a mix that changes every press. Now switch to DDIM. All six paths lie on top of each other and end at the same value, every time. DDIM from one start gives one answer. It is still a generator, because a different start gives a different answer. Move the start to −0.40 and the answer changes to −1.00.
The lecture ends this part with a practical gain. Since DDIM never needed the one-step-at-a-time chain, we can walk a shorter list of levels, J = jK > jK−1 > … > j0 = 0, and evaluate the network only at those levels. The denoiser is the same. No retraining is needed. Fewer evaluations cut the sampling cost, and they can change the sample quality.
The device shows both sides. Keep the start at +0.02, a hard case because the sample sits on the fence between the two data values. With 100 DDIM calls the path ends at +1.00. With 20 calls it still does. With 10 calls it ends at +0.80, short of the data. With 5 calls it ends at +0.08, nearly stuck on the fence. That last sample is a blur: it is neither +1 nor −1. The failure has a symptom (an in-between sample), a measurement (its distance from the nearest data value, 0.92 here), and a fix (take more calls). Over many starts the miss is smaller: the hero measures an average of 0.14 for five DDIM calls. Real image models show the same trade, with their own numbers.
Chapter 10
Draw straight lines from noise to data, teach a network the velocity of each line, and sample by following the average
Drop a leaf on a river. You can predict where it ends up in two ways. You can memorise a table of start points and end points, or you can map the current: at every spot on the water, which way does it push and how hard? The second map is smaller and more useful. Any leaf, dropped anywhere, just follows it.
DDIM showed that a generated sample can follow a deterministic path. The lecture then asks: can we learn the motion directly? The two methods are close cousins. DDIM uses a denoising network to predict a clean and a noise description, and from those it constructs a discrete denoising step. Flow matching uses a velocity network to predict the motion at the current position, which is a direction and a speed, and it integrates a velocity step. The lecture’s summary line: move from predicting a denoised endpoint to learning how the sample moves.
Let rτ be the current sample at generation time τ (tau). Time here runs the other way from Chapter 6’s noise level j: τ = 0 is pure noise and τ = 1 is data. A velocity is a rate of change, a direction and a speed. The network vθ(rτ, τ) outputs the velocity at this position and this time. Following it from noise toward data is a rule called an ordinary differential equation (ODE): the rate of change of r is given by the field.
drτ / dτ = vθ(rτ, τ)
How do we train a velocity network, when we do not know the field? We invent one path per training example. Draw a noise sample z ~ N(0, I) and, independently, a data sample y ~ pdata. Join them with a straight line. A point on that line at time τ ∈ [0, 1] is
The loss is plain regression again. The input is the interpolated sample rτ and the time τ. The target is the paired path velocity y − z:
LCFM = Ez, y, τ [ ∥ vθ(rτ, τ) − (y − z) ∥2 ]
Training builds the intermediate samples directly. It does not integrate the learned field, so there is no long simulation inside the training loop. That is a large practical gain over simulating a chain.
Here is the subtle part, and the slide that matters most. Two different pairs can pass through the same position at the same time. Their targets disagree. The network sees only (r, τ), so it cannot tell the pairs apart. With squared error it learns the average of the targets, the mean velocity: v*(r, τ) = E[ y − z | rτ = r, τ ]. Try it. The faint lines are training pairs. Pick a point, and the device draws the two pairs that cross it, one headed to +1 and one headed to −1.
Horizontal: time τ. Vertical: position r. Faint lines are training pairs. Blue lines are the two pairs through your point. The warm arrow is the mean velocity the network learns.
Toy data of +1 or −1, equally likely. For a pair ending at y through point (τ, r), the noise is z = (r − τy)/(1 − τ), so its velocity is (y − r)/(1 − τ). The weights are the chance that each y produced r: P(y = +1 | r) = 1/(1 + e−2τr/(1−τ)²).
At τ = 0.50 and r = 0.10, the pair that ends at +1 started at z = −0.80 and moves with velocity +1.80. The pair that ends at −1 started at z = 1.20 and moves with velocity −2.20. The weights are 0.60 and 0.40, so the mean velocity is 0.60 × 1.80 + 0.40 × (−2.20) = +0.19. Move to r = 0 at τ = 0.05: both pairs push almost equally hard in opposite ways, and the mean is almost zero. The network sits on a fence and has no strong opinion. A straight line is the training target, but the learned field is a blend, so sampling trajectories can curve even though every training segment is straight.
To generate, start from noise r0 ~ N(0, I) and follow the learned field from τ = 0 to τ = 1. The simplest rule is Euler sampling: ask for the velocity here, take a small step in that direction, then ask again:
rτ+Δτ ≈ rτ + Δτ · vθ(rτ, τ)
With N steps, Δτ = 1/N and the network is called N times. This sampler is the cousin of Chapter 9’s DDIM step count. Choose N below and watch the trajectories. The faint grey curves are the exact flow (many tiny steps). The warm lines are your Euler path.
Drag N from 1 to 24. With very few steps the straight Euler segments cut the corner of the curved flow.
The same toy world and the exact best velocity field, v*(r, τ) = (tanh(τr/(1−τ)²) − r)/(1 − τ), so only the number of steps differs. The grey curves use 200 Euler steps as the stand-in for the exact flow. Forty-eight seeded starting noises.
With one step, every sample moves straight to r = 0.00, the average of the data. That is a blur: the network predicted the mean velocity at τ = 0, and one big step follows it all the way. With two steps the samples reach the two sides but unevenly, with an average miss of 0.43. With four steps the miss is down to 0.07, and with eight it is almost gone. This is the same failure ladder as in Chapter 9: a symptom (blur), a measurement (the average miss), a fix (more steps).
| DDPM and DDIM | Flow matching | |
|---|---|---|
| Path | yj = aj y0 + bj ε, a prescribed Gaussian corruption schedule | rτ = (1 − τ) z + τ y, a chosen noise to data path |
| The network predicts | The noise | The velocity |
| The sampler | Determines the reverse updates | Integrates the learned field |
The lecture’s closing message: path, prediction target and sampling rule are connected design choices. Change one and the others move with it. The last slide asks the question the next lecture takes up: how do diffusion and flow matching fit into the world-model framework?
Chapter 11
Keep the numbers, the terms, the checklist and the core code, then follow the links onward
The lecture has two halves with one shared habit: learn small, cheap pieces and chain them. In Dreamer, an agent learns a value to stand for the missing tail of a short imagined rollout, then trains an actor on imagined returns by sending the slope through the world model. In diffusion, a network learns to remove a little noise, and the ELBO shows that this is a regression on a mean. DDIM removes the fresh noise and lets the sampler take bigger strides. Flow matching learns the velocity of straight noise-to-data paths and follows the average. Go back to the instrument at the top and try each sampler again.
| Item | The fact | Where |
|---|---|---|
| Return | Rt = ∑ γk rt+k+1. Worked: 1, 0, 2, 0, 3 at γ = 0.9 gives 4.59 | Chapter 1 |
| Rollout return | G0 = r̂1 + … + γH−1 r̂H + γH Vψ(xH) | Chapter 2 |
| Value loss | ½(Vψ(xt) − sg(Gt))2, with G held fixed | Chapter 2 |
| Two actor gradients | Score function: −log π · sg(A). Dynamics: −G. At N = 8 the spreads were 1.78 and 0.54 around a true 3.00 | Chapters 3, 4 |
| Three parameter sets | θ world model, ψ value, φ policy, each with its own signal | Chapter 5 |
| Cutoff vs end | Cutoff: G = rewards + tail value. True end: G = rewards only | Chapter 5 |
| Jump to level j | yj = √ᾱj y0 + √(1 − ᾱj) ε. At j = 50 in the toy schedule: 0.46 and 0.89 | Chapter 6 |
| ELBO split | Lrec + ∑ Lj + Lprior; each Lj is a KL between Gaussians, so it is a mean error | Chapter 7 |
| DDPM training and sampling | One level per step, loss ∥ε − εθ∥2; sampling needs J calls with fresh noise except at the end | Chapter 8 |
| DDIM | yi = √ᾱi ŷ0 + √(1 − ᾱi) ε̂; no fresh noise; any subsequence of levels | Chapter 9 |
| Flow matching | rτ = (1 − τ)z + τy; target y − z; sample with r + Δτ vθ | Chapter 10 |
You understand this lecture if you can do each of these from memory:
python # Dreamer: imagined return and the three updates (sketch) def imagined_return(x0, actor, world, value, H, gamma): x, G, disc = x0, 0.0, 1.0 for t in range(H): a = actor(x.detach()) # reparameterised sample x = world.step(x, a) # next latent, differentiable G = G + disc * world.reward(x) disc = disc * gamma return G + disc * value(x) # tail value, discounted by gamma**H G = imagined_return(x0, actor, world, value, H, gamma) actor_loss = -G.mean() # grads reach the actor through the model value_loss = 0.5 * ((value(x_t) - G_t.detach()) ** 2).mean() # Diffusion: DDPM loss, DDIM step yj = sqrt(abar[j]) * y0 + sqrt(1 - abar[j]) * eps loss = ((eps - net(yj, j)) ** 2).mean() def ddim_step(y, j, i): # i < j, no fresh noise e = net(y, j) y0 = (y - sqrt(1 - abar[j]) * e) / sqrt(abar[j]) return sqrt(abar[i]) * y0 + sqrt(1 - abar[i]) * e # Flow matching: loss and Euler sampler z, y, tau = randn_like(y0), y0, rand(batch, 1) r = (1 - tau) * z + tau * y loss = ((vnet(r, tau) - (y - z)) ** 2).mean() r = randn(shape) for n in range(N): r = r + (1.0 / N) * vnet(r, n / N)
Dreamer sits in model-based reinforcement learning. Our model-based RL lesson and world models lesson give the wider picture, and Lecture 10 gives the policy and value ideas this lecture uses. The VAE side of Chapter 6 is in VAE and VQ-VAE and in Lecture 7. For diffusion and flow matching in more depth, see Diffusion, Flow Matching and the series lesson Flow Matching and Diffusion Models, part 2.
Credit. This lesson re-teaches Lecture 11 of CIS 6280: World Models at the University of Pennsylvania (Fall 2026), taught by Jiatao Gu. The course slides are the source for the order and the claims. The slides cite Hafner et al., Dream to Control: Learning Behaviors by Latent Imagination (Dreamer, ICLR 2020), Hafner et al., Learning Latent Dynamics for Planning from Pixels (PlaNet, 2019), Kingma and Welling (VAE, 2014), Ho et al., Denoising Diffusion Probabilistic Models (2020), Song et al., Denoising Diffusion Implicit Models (2020), Lipman et al., Flow Matching for Generative Modeling (2023) and Sutton et al. (policy gradient, 1999). All devices, toy numbers and wording here are our own, and every toy number is labelled illustrative where it appears.
Now press Teach and explain the lecture from memory: why a rollout needs a value tail, how the actor gets its slope through the model, why a diffusion loss is a regression on the mean, what DDIM drops, and why flow matching learns an average velocity. Then go back to the instrument and generate with one step, then twenty.