A robot plans a move by imagining it, one step at a time, and each imagined step starts from the last guess. Each guess is a little off, so the errors pile up. A model only 8% wrong per step ends 5.8 units off after 12 steps.
Learn how a world model encodes what it sees, keeps the past, predicts the next code, and rolls forward on its own.
Press Play, then drag the model gain and the number of steps. Compare the green line, where the model gets the true state each step, with the warm line, where it reads its own guesses. Then we build, piece by piece, the codes, the memory, the distributions and the famous designs that do this.
You need a rough idea of what a probability is and a little algebra. We build the rest from zero.
Based on Lecture 8 of Penn CIS 6280: World Models (Fall 2026), taught by Jiatao Gu, and the papers it cites. Lecture slides. Every idea is re-taught here in our own words and devices; it follows Lecture 7, Latent-Variable and Adversarial Models.
The system is the lecture’s own (slide 63): the true position follows xt+1 = xt + at from x0 = 1, with a = +1 for six steps and then −1 for six. The model believes xt+1 = g xt + at. At g = 1.08 the lecture reports a one-step error of 0.160 and a rollout error of 5.823 at step 12; this page computes the same numbers live.
Chapter 0
Turn a camera frame into a short code the model can predict, and see the four choices hidden inside that step
A robot looks at a table through a camera. Each picture has 64 × 64 pixels, three colour numbers per pixel. That is 12,288 numbers, and 12,288 more arrive a moment later. Now we ask the robot: what will the next picture look like if I push the cup left?
Predicting 12,288 numbers one by one is slow, and most of them carry no news. The table stays the table. The cup moves. A smarter plan is to squeeze each picture into a short list of numbers that keeps what matters, and to predict that list instead. This lecture is about doing exactly that, and about the choices that make it work or fail.
One short paragraph on the course first. Assignment 1 is due Thursday, September 24, and Canvas holds the exact deadline. The September 22 class was tentatively on Zoom, and the September 24 class may be on Zoom too, so check Canvas for both. Office hours this week moved to Friday, September 25, at 5:00 PM.
An observation, written ot, is whatever the agent senses at time t, such as one camera frame. An action, at, is what the agent does, such as “push left”. A world model is a learned model of how the world changes when the agent acts. Lecture 7 told you why one answer is not enough, and how latent variables let a model draw many answers.
A latent world model is a world model that does its predicting in a hidden space. The lecture states it in one sentence: it predicts how an environment evolves under supplied actions, using a learned internal representation. A representation is a rewritten form of the input, such as a short list of numbers. A code, written zt, is the representation of one observation. The code is the thing we predict.
Two earlier ideas meet here. A state-space model says a hidden state explains how observations evolve. Representation learning says an objective, a goal for training, decides what the code keeps. Neither gives you the other for free. The lecture’s warning is plain: an observation code is not automatically a sufficient predictive state. A code can keep the colour of the cup and lose its speed. Chapter 1 returns to this.
The encoder is a network that turns an observation into a code. The decoder turns a code back into an observation. In a variational autoencoder (VAE), the encoder does not output one code. It outputs a distribution over codes, written qφ(z | x), and we sample one code from it. Lecture 7 wrote the observation as x. This lecture writes ot, because observations now come in time order.
The prior p(z) plus the decoder pθ(x | z) define a generative model of single observations. Nothing in that picture says how one observation leads to the next. The lecture is blunt about it: a temporal transition is an additional modeling choice. Most of this lesson is about how to make that choice.
Slide 5 adds a second option. You can train the decoder against a discriminator, a judge network that tries to tell real images from decoded ones. The decoder then learns from reconstruction losses, perceptual losses (which compare how two pictures look to a trained network, not pixel by pixel) plus the judge’s feedback, which tends to give sharper images. The discriminator exists only during training. VQGAN is the lecture’s example of a latent autoencoder with an adversarially trained decoder. Lecture 7 covered the game itself.
What does a code look like? Slide 6 shows two shapes. It can be one continuous vector, such as [0.2, −0.4, 1.1, …]. Or it can be a sequence of patch tokens, [v1, v2, …, vK], where each piece describes one patch of the picture and is itself a vector. The lecture’s rule is: a token is a unit of representation, and it can be a continuous vector. So a “token” here is wider than the word-pieces of a language model.
Some designs go one step further and make every piece a whole number. They keep a codebook: a fixed list of V learned vectors, each with an index. Think of a paint shop that sells only four shades. You bring a colour you like, and the clerk hands you the closest shade on the shelf.
That is vector quantization. The encoder first makes a continuous feature v = E(o). Then we compare v with every codebook entry and pick the nearest one.
The lecture works a toy case with four entries in two dimensions: e1 = (−1.0, 0.0), e2 = (0.0, 1.0), e3 = (0.7, −0.2) and e4 = (1.0, 1.0). The feature is v = (0.6, −0.1). Work it by hand. The squared distance to e3 is (0.6 − 0.7)2 + (−0.1 + 0.2)2 = 0.01 + 0.01 = 0.02. To e1 it is 2.56 + 0.01 = 2.57. To e2 it is 0.36 + 1.21 = 1.57. To e4 it is 0.16 + 1.21 = 1.37. Entry 3 wins, so k* = 3. Now move the feature yourself.
The four pink dots are the codebook. The teal dot is your continuous feature v. The solid green line goes to the winner. Drag the two sliders and find the borders where the winner changes.
The codebook and the starting feature are the lecture’s toy values (slide 7). Real codebooks hold hundreds or thousands of learned entries in many more dimensions; the lecture marks these numbers as toy values too.
Notice what the decoder gets: e3 = (0.7, −0.2), not your v. The gap between them is the price of a code you can count. The payoff is that a sequence of indices is exactly what a language-style model can predict. Repeat the lookup at every feature position and you get a sequence or a grid of code indices. The encoder, the codebook and the decoder learn together from reconstruction and quantization objectives.
Slide 8 sorts the encoder into a small table. The code can be continuous or discrete. The encoding can be deterministic or stochastic. Deterministic and continuous is z = E(o). Deterministic and discrete is the nearest codebook index. Stochastic and continuous draws z ~ qφ(z | o), a distribution over vectors. Stochastic and discrete samples a categorical code, a draw among V labelled options.
The lecture then adds a sentence worth keeping: the encoder distribution and the future-prediction distribution are separate choices. A model can have a noisy encoder and a one-answer predictor, or a one-answer encoder and a noisy predictor. We will meet both.
Course note. Assignment 1 is due Thursday, September 24, and the class on September 22 may be on Zoom, so check Canvas for the exact time and link. Office hours this week moved to Friday, September 25 at 5:00 PM.
The lecture has four parts, and this lesson follows them in order: the design space, autoregressive prediction, seven concrete models, and the failures of rollouts.
Chapter 1
See why one frame cannot tell you which way a ball is moving, and compare the two ways a model can keep the past
You watch a video of a ball, and someone pauses it. The ball sits in the middle of the screen. Where will it be in one second? You cannot say. It might be rolling right. It might be rolling left. One frame holds the position, and the motion lives in the difference between frames.
This is the heart of the lecture’s first section, and it is the reason a world model needs memory. Before we add memory, let us see the one case where we already know exactly what to do.
In Lecture 4 we met the linear Gaussian state-space model (LGSSM). A hidden state st is the full truth about the world at time t, such as the ball’s position and speed. The state changes by a fixed linear rule plus noise, and a sensor sees a noisy linear view of it.
Because everything is Gaussian, the agent’s belief about the state, its best guess with an error bar, stays Gaussian forever. Two numbers describe it: a mean μ and a covariance P. The belief update has two steps, and the lecture lists the formulas.
Step 1, predict. Push the belief through the world model: μ− = Aμ + B a, and P− = A P A⊤ + Q. The error bar grows, because the world adds noise. Step 2, update. A new observation arrives. The innovation ν = o − Cμ− measures how wrong the prediction was. The Kalman gain K = P−C⊤S−1, with S = C P− C⊤ + R, says how much to trust the sensor. Then μ = μ− + Kν and P = P− − K S K⊤.
Here is one pass with plain numbers. Take A = B = C = 1, Q = 0.1, R = 0.5, a start of μ = 0 and P = 1, an action a = 1, and an observation o = 1.4. Predict: μ− = 1 and P− = 1.1. The sensor term is S = 1.1 + 0.5 = 1.6. The innovation is ν = 1.4 − 1 = 0.4. The gain is K = 1.1 / 1.6 = 0.6875. So the new mean is 1 + 0.6875 × 0.4 = 1.275, and the new covariance is 1.1 − 0.6875 × 1.6 × 0.6875 = 0.344. The error bar shrank from 1.1 to 0.344, because the sensor helped. The loop then repeats for the next action and observation.
μ = 1 + 0.6875 × 0.4 = 1.275, P = 1.1 − 0.756 = 0.344 (toy numbers, ours)
Real worlds have nonlinear dynamics and image observations. The lecture splits the new problem into two challenges. The first is the world model itself: how to represent the environment and predict it. In the general form, the next state is drawn from pθ(st+1 | st, at) and the observation from pθ(ot+1 | st+1). We must choose the state, the representation and the predictive distributions.
The second is the belief update: inferring the state from the history of observations. The history Ht = (o0:t, a0:t−1) is every observation so far and every earlier action. The belief now needs an integral over all earlier states, and a Bayes rule for the new observation. Those integrals may have no closed form, and a mean plus a covariance may no longer describe the belief. A note on symbols: the slide writes Ht for the history, and reserves ht for a recurrent memory, a hidden vector a network carries from step to step.
The lecture sums up the job in one line. We must design both the latent world model and how observation history informs predictions.
The ball example is slide 11. History A shows a dot moving right across three frames. History B shows it moving left. The last frame in each history is the same image, a dot in the middle. Their futures differ. A predictor that sees only the last frame cannot tell them apart.
Each row is a history of three frames, then the true next frame (blue ring) and the predictor’s answer (warm dot). Frames the predictor cannot see are dimmed. Slide the number of frames it may use.
Toy numbers, ours: the dot moves 0.15 of the frame width per step, so history A ends at 0.50 and moves to 0.65, and history B ends at 0.50 and moves to 0.35. With two or more frames the predictor adds the last step again; with one frame the best single answer is the middle, 0.50.
Look at what the one-frame predictor does. It sees the dot at 0.50 in both rows, so it must give one answer for two different futures. The safest answer is the middle, 0.50. That is 0.15 wrong in both rows, and it matches neither future. With two frames the speed is visible, and the error drops to zero. The failure is not a flaw in the predictor’s arithmetic. The needed fact was never in its input.
So the model must keep information from the history. The lecture names two views of how. In the first, you describe state evolution. You infer a model state s̃t from the observed history Ht, apply modeled dynamics with the action, and get the next model state s̃t+1. The state is the memory, and it follows a rule you wrote down.
In the second you condition predictions on retained history. You encode the history of codes and actions, z≤t, a<t, feed it to a predictor network that holds an internal state ht, and read out the next code ẑt+1. A hat marks a prediction. Here the network’s own memory carries the past, with no hand-written dynamics.
Both retain information from history, and the lecture notes that the two views can overlap in one learned model. A recurrent network with a latent variable, as in chapters 6 and 7, is both at once. The practical test for any design is the one the device just ran: which facts about the past can the predictor reach?
Chapter 2
Chain one-step guesses into a trajectory, and learn why training with the true past differs from running on your own
A weather forecaster says: it rains tomorrow. Asked about the day after, she does not wait for tomorrow to happen. She starts from her own forecast and predicts again. A third day follows from the second forecast. Each day leans on a guess, and each guess may be a little wrong.
That is a rollout: a trajectory made by running the model forward for many steps without new observations. It is how a world model is used for planning, so it is the thing we must get right. Slides 13 to 17 set up how rollouts work and where training differs from using.
Chapter 1 gave two ways to keep the past, and each has its own rollout. With an explicit model state, you infer a state s̃t from what you observed, then propagate it: the action at takes it to a predicted state s̃t+1, and the next action takes that to s̃t+2. The state carries the memory.
With history-conditioned prediction, you keep a growing list of codes. You start with the observed prefix …, zt. The model predicts one more code, and you append it: …, zt, ẑt+1. Then you predict again from the longer list. The slide adds a clear rule: actions are supplied at each step, earlier actions also stay in the history, and the hat marks a generated code. In both views the instruction is the same. At each step, use the model’s output to continue predicting the future.
A subtle point hides here. The thing the model passes from step to step is its internal state. The thing that supervises the model is a target, such as the next observation, the next code, or a posterior target. The lecture says these need not be the same variable. A model can carry a rich hidden state and be graded only on a small readout of it. Keep that in mind: later models grade different things.
Now the design question of where the code comes from. Suppose an encoder Eφ makes codes z and a predictor Fψ turns them into predictions ẑ, with a prediction loss against a target. There are two ways to train.
In pretrain, then freeze, you train the encoder first, for example as an autoencoder, then lock it. The prediction loss then updates the predictor only. The codes stay fixed, which is stable and cheap. In joint learning, both parts are trainable, and the prediction loss updates the encoder and the predictor together. The codes can bend toward whatever helps prediction, which is powerful and dangerous.
Why dangerous? The slide asks the question and leaves it open: a constant representation is easy to predict. If the encoder outputs the same code for every input, the predictor guesses it perfectly and the loss is zero. Chapter 5 gives the answer, and shows what each model uses to prevent it.
The lecture now defines the core idea. A rollout starts from the observed seed ẑ0 = z0. Every prediction uses earlier outputs and the supplied actions. Prediction can be deterministic, a direct output, or probabilistic, a draw from a predicted distribution.
The lecture names this pattern: it is autoregressive. “Auto” means self, and “regressive” means predicting from earlier values. The model reuses its earlier outputs to predict later ones. Language models do exactly the same with words.
Here is the catch. During training you own the recording. You know the true code at every step, so you can hand the model the recorded past. That is teacher forcing: the next step always receives the recorded zt+1, the teacher’s answer, whatever the model just said. The loss compares each prediction with the recorded next code.
During a rollout you own no recording of the future. The first step reads the observed prefix, but the second step must read the model’s own generated code. The lecture’s summary is exact: teacher forcing feeds recorded history, rollout feeds generated history, and future observations are unavailable.
Step through a toy of this. The true codes are 0, 2, 4, 6, 8, 10, which grow by 2. Our model has learned a slightly wrong rule, “add 2.3 to the input”.
Each column is one prediction. Rows show the input the model reads, its answer, the recorded target, and the error. Switch between training and rollout, then add steps.
Toy numbers, ours: the true code at step t is 2t, and the model answers input + 2.3. This stands in for any model with a small, steady error; the lecture’s slide 17 shows the same two input flows without numbers.
In training mode every step is 0.3 off, and the error never grows, because each step starts from the truth. In rollout mode the first step is also 0.3 off. But the second step starts from a code already 0.3 too high, adds its own 0.3, and lands 0.6 off. By step 5 the rollout is 1.5 off, five times worse, although the model and its rule are identical. The hero at the top of this page shows the same effect at a different scale.
This gap is the training-rollout mismatch. The model is graded on one task, predicting from truth, and used on another, predicting from itself. Chapter 9 gives the remedies.
Chapter 3
Score a prediction with the negative log likelihood, for a discrete code and for a Gaussian, and see where squared error comes from
A model has to say what comes next, and it can answer in two ways. It can name one thing: the next code is 7. Or it can give odds: 7 is likely, 3 is possible, 11 is a long shot. The lecture calls the first answer deterministic and the second probabilistic, and it shows how to train each. We start with the case where the codes are whole numbers, as in Chapter 0’s codebook.
Suppose the codebook holds V entries, and the model must pick the index of the next one. The network reads the context, which is the code history plus the action, and outputs V raw scores called logits. One score per code, any real number. A softmax turns scores into probabilities: exponentiate each score so it is positive, then divide by the total so they add to 1.
This loss is the negative log likelihood (NLL). Likelihood means: how much probability did the model give to what actually happened? We take the logarithm so that products of many steps become sums, and we flip the sign so that smaller is better. Work it for four codes with logits 2.0, 1.0, 0.1 and −1.0. The exponentials are 7.39, 2.72, 1.11 and 0.37, which sum to 11.58. The probabilities are 0.64, 0.23, 0.10 and 0.03. If the true next code is the second one, the loss is about −ln 0.23 = 1.45. If it was the first, the loss is −ln 0.64 = 0.45. A model that backs the right answer pays little.
Now use the distribution. You can sample: draw k̂ from the categorical distribution, so code 1 comes up 64% of the time and code 4 comes up 3% of the time. Or you can take the argmax: always pick the largest probability. Sampling is stochastic and the argmax is deterministic. The lecture says it directly: prediction can be stochastic or deterministic. Sampling keeps the other futures alive in a rollout. The argmax always follows the single most likely path.
What if the code is a list of real numbers? There is no finite list of options to put odds on. The standard move is to predict a Gaussian, the bell curve. The network outputs two things for the next code y = zt+1: a location μθ(c), the middle of the bell, and a spread Σθ(c), its width. The slide’s sentence is: predict location and spread from context.
The loss keeps the same idea. It is the NLL of the recorded next code y* under the bell: ℒ = −log pθ(y* | c). As before, you can sample ŷ ~ N(μ, Σ) for a stochastic step, or return the mean ŷ = μ for a deterministic one.
Pick the kind of code. Move the sliders so the model puts its probability on the blue truth, and watch the loss fall. Then press Sample to draw one answer and Best guess for the deterministic answer.
Toy numbers, ours: six logits 2.2, 1.4, 0.7, 0.1, −0.5 and −1.2, multiplied by the sharpness; the Gaussian truth is y* = 1.0. The formulas are the lecture’s (slides 18 and 19); with sharpness 0 all six codes get 1/6.
Try the Gaussian by hand first. Take y* = 1.0, μ = 0.4 and σ = 0.5. The NLL of a one-number Gaussian is (y − μ)2 / (2σ2) + ½ ln(2πσ2). The first term is 0.36 / 0.5 = 0.72. The second is ½ ln(1.571) = 0.226. The total is 0.946. Now slide σ to 1.0: the first term drops to 0.18 and the second rises to 0.919, so the total is 1.099. The loss has two forces. A narrow bell punishes any miss hard. A wide bell pays a fixed fee for being vague.
Now hold the spread fixed, σ2I. Then the second term is a constant C, and the NLL becomes ‖y − μθ(c)‖2 / (2σ2) + C. That is a scaled mean squared error (MSE), the plain “square the miss” loss. So training a deterministic network with MSE is the same as training a Gaussian with a fixed spread and keeping only its mean. If the spread is learned, the lecture notes that residual weighting and log-scale terms also matter.
What does MSE aim at? Over all the futures that could follow the same context, the squared error is smallest at their average. The slide states it: the best mean is μ*(c) = E[y | c], the conditional expectation. The lecture adds the warning that this mean need not be a typical future. If a ball can go to −1 or to +1 with equal chance, the MSE answer is 0, a place the ball never goes. Lecture 7’s blurry point predictor was exactly this.
Notice a split in how the output is used. During training you never sample. You only ask how much probability the model gave to the recorded value, and you push that number up. During a rollout there is no recording, so you must choose a value to feed back in. The model gives odds, and something outside the model turns odds into one code.
The choice has a cost either way. Argmax always follows the single most likely path, so it is repeatable but can lock into a loop. Sampling keeps other futures alive, but it may draw an unlikely code, and that code then becomes the next input. Take a model that is right about its odds and gives its best code 0.64 at each of three steps. A sampled rollout follows the argmax path only 0.64 × 0.64 × 0.64 = 26% of the time. The other 74% of rollouts wander off it.
The spread σ works the same way for continuous codes. It is the model’s own statement of how much to trust its mean. A model that misses by 0.6 pays 0.946 with σ = 0.5, as in the example. The same miss with an overconfident σ = 0.2 costs 0.36 / 0.08 + ½ ln(2π × 0.04) = 4.5 − 0.69 = 3.81. So the loss rewards a model that knows how wrong it tends to be. That honest spread is what lets a rollout show a fan of futures instead of one thin line.
Chapter 4
Let the network output a weighted blend of Gaussians, train it by scoring the true next code, and sample from it to roll forward
A robot hand moves toward a cup, and a post stands in the way. In the recordings the hand goes around the post. Sometimes it passes on the left, and sometimes on the right. Both are good. The path through the middle hits the post.
Chapter 3 ended on the problem. A network trained with squared error returns the average of the two paths, which runs through the post. A single Gaussian does no better, because one bell has one hump, and the hump would sit in the middle too. We need an output that can say: either here, or there.
The lecture’s answer is a mixture: a weighted sum of several Gaussians. Each component is one bell that describes one region of possible outcomes. Each weight says how likely that region is. With M components the density is a sum over them.
For the robot hand, two components do the job. One bell sits left of the post, one sits right, and each carries weight 0.5. The middle gets almost no density, so the model never proposes a path through the post.
A mixture density network (MDN) is an ordinary neural network whose last layer outputs these parameters. The lecture’s summary is exact: an MDN predicts parameters, and it does not output one chosen future. It reads the context c, which is the code history plus the current action, and returns three groups of numbers.
The weights π1:M come from a softmax over M logits, so they are positive and sum to 1, as in Chapter 3. The means μ1:M are unconstrained, because a centre can be anywhere. The scales σ1:M must be positive, so the network outputs a raw number and applies a softplus, the smooth function ln(1 + ex), which is always above zero. The slide adds a small ε. Why? A bell that shrinks to a needle has an enormous density at its centre, and the loss would chase that. The ε keeps every bell at least a little wide.
Count the numbers. For a code with 1 coordinate and M = 3, the network outputs 3 logits, 3 means and 3 raw scales: 9 numbers. For a code with 32 coordinates, each component needs 32 means and 32 scales (if the bells are axis-aligned), plus one logit, so the count grows quickly. Choose M as a design setting: too small and a real region goes unmodeled, too large and some bells spike on single examples.
To train, we use the NLL from Chapter 3, now on the mixture. Take the recorded next code y* = zt+1. Evaluate every component at y*, multiply each by its weight, add, and take −log.
ℒ(c, y*) = −log [ ∑k πk(c) N(y*; μk(c), σk2(c)) ]
The slide stresses that training requires no component sampling. You never choose a component. Gradients flow through the sum, so every bell is nudged in proportion to how much it explained y*. Work one case. Let there be two components, weights 0.5 and 0.5, means −1 and +1, and scales 0.3 and 0.3. The true next code is y* = 1.1. The right-hand bell has density (1 / (0.3 × 2.507)) × exp(−0.5 × (0.1 / 0.3)2) = 1.330 × 0.946 = 1.258. The left-hand bell is 2.1 / 0.3 = 7 scales away, so its density is near zero. The mixture gives 0.5 × 1.258 = 0.629, and the loss is −ln 0.629 = 0.46. A single Gaussian with mean 0 and scale 1.0 gives density 0.218 at the same point, and a loss of 1.52. The mixture pays a third as much.
The blue ticks are 40 recorded next codes for one context: two clusters. Choose how many bells the network may use. The fit runs live. Then press Sample 20 to pick a bell, then a value.
Toy data, ours and seeded: 40 values from two bells at −1 and +1 with spread 0.25. The fit uses 60 steps of expectation maximization, a standard way to fit a mixture; a real MDN reaches a similar fit by gradient descent on the loss above. The NLL is the average over the 40 values, in nats.
To roll forward, the model needs one code. It samples in two stages. First draw a component, k ~ Cat(π(ct)), which picks “left of the post” or “right of the post”. Then draw the code from that bell, ẑt+1 ~ N(μk, Σk). Then update the context, take the next supplied action, and repeat. Because the first draw commits to a branch, each rollout is a single plausible future.
The lecture closes the section with a pointer. Richer alternatives, diffusion and flow models, come in later lectures. The MDN is the simple member of that family, and it is the one the 2018 World Models system used.
Chapter 5
Study three models that predict the next feature directly, and find what stops their codes from collapsing to a constant
Think of a cook who tastes a stew and writes one line in a notebook: “salty, thick, hot”. Next to it she writes what she expects after adding water: “less salty, thinner, hot”. She never writes the full recipe or draws the pot. She predicts in her own short notes. The notes are the features, and the next three models do exactly this: they predict the next feature directly.
The lecture pauses for a recap. A latent world model has three parts in a row. First, encode each observation into a code, ot → zt, either continuous or discrete, deterministic or stochastic. Second, retain predictive state, a context ct built from the code and action history, (z≤t, a<t) → ct. A model state, a belief or an internal memory carries it. Third, predict under an action, (ct, at) → ẑt+1, as a direct output or as a distribution plus a selection rule.
Training compares predictions with observed codes or observations. Rolling forward reuses the predicted state or code for the next action, and decodes to observations when the model has a decoder. The lecture names the four design choices: the representation, the retained state, the prediction rule and the training objective. Slide 26 then asks how seven real models answer them, and walks through each. This chapter takes the first three: TD-MPC, V-JEPA 2-AC and LeWorldModel.
TD-MPC (Hansen, Wang and Su, 2022) is the simplest. Its encoder is deterministic: zt = Eθ(ot), and the input may include recent frames. Its dynamics network is deterministic too: ẑt+1 = Fθ(zt, at). A rollout reuses the predicted latent at the next step, so every step stays in latent space.
The training loss is a latent consistency loss: the squared L2 distance, the sum of squared differences, between the prediction and the code of the real next frame.
The target comes from a target encoder, a slow-moving copy of the online encoder. It follows the online weights by an exponential moving average (EMA): θ̄ ← (1 − τ)θ̄ + τθ. With an illustrative τ = 0.01, a target weight of 1.00 facing an online weight of 2.00 moves to 0.99 × 1.00 + 0.01 × 2.00 = 1.01. The copy drifts slowly, so the target does not jump around under the predictor’s feet.
Now the warning on slide 30. The consistency loss alone permits collapse. If the encoder outputs one constant for every frame, E(o) = c, and the dynamics returns it, F(c, a) = c, then the loss is exactly zero. So the full TD-MPC objective also includes reward prediction and value learning. The rewards force the code to keep what matters for the task. The slide’s videos show the latent transition and the agent running (the Dog Run task), and the lecture’s note is that predictions stay in latent space while the task video shows the behavior in the environment.
V-JEPA 2 (Assran et al., 2025) is a video model, and the lecture focuses on its action-conditioned stage, V-JEPA 2-AC. Its encoder is a pretrained network with deterministic features, frozen during this stage. The predictor keeps a history of features, actions and robot poses inside itself, and predicts the next feature deterministically.
The loss uses L1 error, the sum of absolute differences, in two parts. The teacher-forced term predicts each next feature from the recorded history. The two-step rollout term feeds the first prediction back and checks the second one. So L = Lteacher forcing + Ltwo-step rollout, and both compare with the frozen encoder’s features of the real frame. This is Chapter 2’s remedy for the training-rollout gap in a small dose. The robot then uses feature-space predictions to choose actions. A frozen encoder cannot collapse, because nothing in this stage trains it.
LeWorldModel (Maes et al., 2026) goes the other way. It learns the encoder and the predictor jointly, end to end from pixels, with deterministic continuous features. The history lives inside the predictor, ẑt+1 = Fψ(zhistory, ahistory). The loss is MSE prediction plus a regularizer called SIGReg.
SIGReg answers Chapter 2’s open question, what stops a constant code. It takes many random one-dimensional projections of the codes in a batch, and pushes each projection toward a standard bell, N(0, 1). A constant code fails that test at once. Across samples, the embeddings are encouraged toward N(0, I). The lecture adds a careful caveat: this does not make the prediction p(zt+1 | history, action) Gaussian. It constrains how the codes spread across the data, not what the next code looks like. A separate visualization decoder, not part of training, lets you look at predicted frames. Control uses the latent predictions.
Eight inputs get eight codes on the line. The scale s multiplies every code. Drag s toward zero, or press Train to let gradient descent decide. Then turn the spread penalty on and train again.
Illustrative toy, ours: the predictor misses by 0.3 on the scale of the codes, so the prediction loss is 0.09 s2. The penalty is 0.5 (s2 − 1)2, which asks the codes to have spread 1; real SIGReg tests random projections of the batch against N(0, 1) and is more general. The collapse itself is the lecture’s (slide 30).
Watch what the loss alone does. Its curve is a bowl with the bottom at s = 0, where all eight codes sit on one point and the loss is zero. The model has “solved” prediction by predicting nothing. With the penalty on, the bottom moves to s = 0.95, and the codes keep their spread. The minimum is a little under 1, because the prediction loss still tugs the codes inward a bit.
Chapter 6
Walk through vision, memory and dreaming in the model that made latent world models famous, in the order it is trained
Imagine you have driven the same track a thousand times. Now close your eyes. You can still “see” the next bend coming, and you can steer through it in your head. You do not replay pixels. You hold a feel for the road, and your feel moves forward as you imagine turning the wheel.
In 2018 David Ha and Jürgen Schmidhuber built a machine that does this. Their paper, World Models, let an agent learn a compact model of a game, then practise inside the model’s dream. Lecture 7 introduced the idea. Our world-models Gleam tells the whole story. Here we open the machine and follow how it is trained, step by step, with the lecture’s own diagrams in mind.
The system has a vision part (called V) and a memory part (called M). A small controller (C) reads their outputs and picks the action. The lecture’s slides cover V and M, because they are the world model. They hold every design choice the lecture asks about.
V is a variational autoencoder, as in our VAE lesson. It looks at one frame ot and gives a distribution over codes, qφ(zt | ot), then samples a stochastic code zt ~ qφ(· | ot). The slide shows driving frames from CarRacing next to their VAE reconstructions. The lecture’s note is worth keeping: the VAE encodes and reconstructs observed frames, and temporal prediction is the MDN-RNN’s job. V knows nothing about time.
M is a recurrent network, an RNN. It reads a code and an action and updates a hidden vector, the memory ht. The update is deterministic: ht = fθ(ht−1, zt−1, at−1). The roles are split cleanly. The code zt encodes the current frame. The memory ht summarizes earlier codes and actions. (If you want to see how an RNN carries a memory, our recurrent-network lesson builds one.)
From ht, M predicts the next code with a mixture density network, the head from Chapter 4. That MDN is the prior pθ(zt | ht): its belief about the code before it sees the frame. So the memory update is deterministic and the code update is probabilistic. The slide’s rule is the one to remember: while observing, sample the frame VAE posterior; during rollout, sample the MDN prior.
Concretely, a camera frame is an array of pixels, say 64 × 64 × 3. V shrinks it to a code of tens of numbers, say z: 32. M keeps a memory a few hundred numbers wide, say h: 256, and its MDN outputs weights, means and scales for a few components (Chapter 4). These sizes are illustrative, to show the shape of the data; they are not the lecture’s. The point is the squeeze. The thing M predicts is a few dozen numbers, not twelve thousand.
The system does not train V and M together. The slide lists two stages, and a third use.
Stage 1: train V. Collect frames from play. Train the VAE to reconstruct each frame, with the usual KL term from Lecture 7 (a score of how far the code’s distribution sits from a plain prior) that keeps the code close to that prior. No time, no actions.
Stage 2: freeze V, fit M. The lecture’s wording is: freeze V; fit M to recorded latent sequences. Run the frozen encoder over a recorded episode to turn every frame into a code. For each step, M reads the history z≤t, a≤t and must give high probability to the recorded next code. The loss is the MDN negative log likelihood from Chapter 4.
ℒM = −log pψ(zt+1 | z≤t, a≤t)
Notice that the frozen encoder appears twice on the slide: once for the frame at time t, and once for the frame at t + 1, whose code becomes the observed target. M scores the recorded next code. Because V is frozen, the target does not move while M learns.
Use: imagined rollouts (slide 44). Now cut the camera. Start from a real code and memory. Sample ẑt from the MDN prior, feed it and the next action into the RNN, get the next memory, sample again. The latent and the memory continue by themselves. If you want to look, send each ẑt through the VAE decoder to get a picture ôt. The slide labels it clearly: the decoded observation is a model output. It is not a real frame.
Pick a phase. Teal outlines learn. Blue outlines are frozen. Grey parts are not used in that phase. Press Dream one step in phase 3 to count imagined steps.
The three phases and their losses are the lecture’s (slides 41, 43 and 44). The box layout is ours, and the step counter only counts; it runs no network.
Why stage the training? Each stage has one clear job and a fixed target. V needs only pictures, and M never chases a code that is still changing. The price is that V learns what looks important, not what predicts well. If the car’s speed is a tiny detail in the picture, V may blur it away, and M can never get it back. The joint models of Chapter 5 and the RSSM of Chapter 7 exist to fix that.
Chapter 7
Let the state posterior see the history, match it to a history-based prior, and imagine by sampling that prior
You wake up and, before you open the curtains, you guess the weather. Yesterday was grey, so you guess rain. That guess is your prior: what you expect before you look. Then you open the curtains and see sun. Your new belief is a posterior: your expectation, corrected by what you saw. A good guesser has a prior that lands close to the posterior, so the correction is small.
Chapter 6’s system gave each frame a code from the frame alone. It never used yesterday’s grey sky to read today’s picture. The model in this chapter, the recurrent state-space model (RSSM), does. It comes from PlaNet (Hafner et al., 2019) and the Dreamer family reuses it. The lecture compares it with World Models on one slide, and that comparison is our map.
The RSSM keeps the same two ingredients as before: a deterministic RNN memory and a probabilistic latent. It names them ht and ut. The memory updates by ht = fθ(ht−1, ut−1, at−1). The pair s̃t = (ht, ut) is the model state. The memory carries history in a way that can be copied exactly. The latent ut carries what is random.
There are two ways to get ut, and they are the center of the model. A frame encoder first turns the picture into features, et = Eφ(ot). Then:
Slide 47 puts the two systems side by side. World Models 2018 draws zt ~ qφ(· | ot): the current frame only. The RSSM draws ut ~ qφ(· | ht, Eφ(ot)): the history plus the current observation. History helps the reading. If you know a car was braking, a blurry frame is easier to read.
Slide 48 shows imagining without observations. The encoder branch is cut. You sample ut ~ pθ(ut | ht) from the prior and continue with ht+1 = fθ(ht, ut, at). The lecture’s sentence: during imagination, sample the prior and continue; the observation branch is unavailable. This is the same two-mode rule as in Chapter 6, now with history on both branches.
Now the question: how is the prior taught to be good? Slide 49 gives one loss per time step.
ℒt = Eq[ −log pθ(ot | ht, ut) ] + β DKL( qφ(ut | ht, et) ‖ pθ(ut | ht) )
The first term is reconstruction. Draw u from the posterior, decode, and score the real frame. The second term is a KL divergence, a measure of how far one distribution sits from another, scaled by a weight β. It pulls the posterior toward the prior and, because both are trained, it pulls the prior toward the posterior. So the prior learns to predict, from history alone, the code that the real frame would have produced. That is exactly what imagination needs. The lecture’s phrase: joint temporal training, where the posterior is matched to a history-dependent prior.
Compare the 2018 recipe. There the VAE KL pulled each frame code toward a fixed standard prior, and the memory model was fitted afterward in a second stage. Here observation modeling and temporal prediction train together, with the temporal prior in the KL.
For a feel of the numbers, take one-number Gaussians. Let the prior be N(0, 1) and the posterior N(0.8, 0.5). The KL of the posterior from the prior is ln(1 / 0.5) + (0.52 + 0.82) / (2 × 12) − 0.5 = 0.693 + 0.445 − 0.5 = 0.64 nats. If the real frame surprises the model more, so the posterior sits at 2.0, the KL grows to 0.693 + 2.125 − 0.5 = 2.32 nats.
The purple curve is the prior, from history alone. The blue curve is the posterior, after seeing the frame. Slide the surprise, then press Train the prior a few times.
Illustrative toy, ours: one-number Gaussians and the closed-form KL, ln(σp / σq) + (σq2 + (μq − μp)2) / (2 σp2) − 1/2. Each press of Train moves the prior half way to the posterior, standing in for many gradient steps. The loss form is the lecture’s (slide 49).
Train the prior and watch the KL fall to zero. When the prior equals the posterior, imagination samples exactly what observation would have shown. Press Imagine 12 before and after training. Before, the dreamed values scatter around the wrong centre. After, they land where the posterior would.
Dreamer builds agents on this state. The lecture lists the world-model choices across versions. Dreamer V1 to V3 are recurrent state-space models. V1 uses a Gaussian stochastic state. V2 uses a categorical state, a choice among labelled codes as in Chapter 3, and a trick called KL balancing that weighs the two directions of the KL differently. V3 uses a categorical RSSM with more robust training. Dreamer 4 changes the architecture to a Transformer world model: video frames pass through a causal tokenizer into continuous video tokens, and block-causal dynamics map tokens plus supplied actions to the next tokens. Shortcut forcing trains the dynamics to denoise future representations in few generation steps. The lecture’s summary: V4 changes the world-model architecture, and latent imagination remains the shared idea.
Chapter 8
Turn each frame into discrete codes and predict them with a Transformer, then count what that costs
Read this line: “the cat sat on the”. You expect the next word to be “mat”. A language model does the same thing, one word at a time, and it does it very well. Now suppose a video game screen were also a sentence. Each frame is a short string of words from a small dictionary, and a game is a long story told in those words. If so, a language model could predict the next frame.
That is the move of this chapter. Chapter 0 gave us the dictionary, a codebook. Chapter 3 gave us the loss for picking the next word. Two models put them together and replace the RNN with a Transformer, a network that reads a whole list of tokens at once and learns which earlier tokens matter for the next one.
IRIS (Micheli, Alonso and Fleuret, a 2022 preprint, ICLR 2023) has the title Transformers are Sample-Efficient World Models. It has four parts in a row, and the lecture draws each of them.
First, a discrete tokenizer: an encoder plus a codebook, as in Chapter 0, turns each frame ot into K codes, kt,1, …, kt,K. The encoder is deterministic: each patch is assigned to its nearest codebook entry. Second, an autoregressive Transformer reads the code history and the actions. Its state is the code and action history itself, held inside the sequence model, with no separate memory vector. Third, it outputs a categorical distribution over the next codes, and we sample or take the argmax, exactly as in Chapter 3. Fourth, an image decoder looks up each code’s vector in the codebook and decodes them to the next frame ôt+1.
The losses are separate. The autoencoder learns from reconstruction and codebook terms. The Transformer learns from token cross-entropy, which is the discrete negative log likelihood of Chapter 3 applied to each next code. The slide notes the training loop repeats updates of both the autoencoder and the Transformer, rather than training them once in sequence.
Generation runs in two orders, and the lecture points them out. The first order is across time: frame t, then the action at, then frame t + 1. The second is within each frame: code 1, then code 2, up to code K, each conditioned on the earlier ones. The slide’s figure shows a recorded trajectory above and a model rollout below that uses the recorded actions, with the first frames given as conditioning. An agent trained on these imagined trajectories then plays Atari games.
Photon-1 (David Li and Jonathan Li, an Induction Labs technical report, July 2026) is titled Scaling Video Pretraining with Imagination Models. The lecture gives the size: 106 billion parameters with 5 billion active, 575 million frames, and 960 discrete tokens per frame. Its structure matches IRIS, and the slide lists the parts.
The encoder is a differential FSQ. FSQ, finite scalar quantization, is a cousin of Chapter 0’s lookup that rounds each number of the feature to a small set of levels, so no learned codebook is needed. The word “differential” means the report codes frame changes, not whole frames. The state is the token history inside the Transformer, and the prediction is probabilistic autoregressive next-token. The learning is video pretraining first, then later action supervised fine-tuning (SFT) and reinforcement learning (RL). One slide shows predicted ball positions in billiards after a fine-tune that rendered them.
Slide 58 lines the two up. They share a recipe: discrete visual representations plus autoregressive next-token prediction. They differ in training.
| Part | IRIS | Photon-1 |
|---|---|---|
| Visual representation | VQ image codes | FSQ codes of frame differences |
| Dynamics data | Action-labeled Atari interactions | Video pretraining without action labels |
| Actions | Condition imagined next frames | After SFT: predict next state, then action |
| Image decoder | Autoencoder reconstruction; rollout rendering | Auxiliary visualization only |
The lecture adds a fair caution. Photon-1 is a scaled extension of this modeling family, and the representation and the training stages also change. So it is not simply IRIS made bigger.
Tokens bring a price. Predicting a frame code by code is serial: code 2 waits for code 1. If a frame has K codes and you imagine H frames, you need about H × K prediction steps. Take 10 frames with Photon-1’s 960 codes each: 9,600 serial steps. The lecture’s remedy for this cost is frame or block prediction, where the model outputs all the features of a frame in one go. Compression, fewer tokens, helps too. Play with the counts below.
The bars use a log scale: each tick is ten times more steps. Choose codes per frame, then slide the number of imagined frames.
The 960 codes per frame is the lecture’s number for Photon-1; 16 and 256 are illustrative picks. The count H × K is the lecture’s (slide 66). Real steps differ in cost, so the lecture says to compare actual latency too.
The slide pairs each cost with a repair and an honest trade. Many serial token steps call for fewer tokens or frame prediction, and compression may lose information. Repeated computation over a long context calls for caching or context compression, which costs memory or forgets history. Many iterative sampling steps call for distillation or shortcut methods, which need extra training and come in later lectures.
Chapter 9
Line the seven models up, then name the ways a rollout goes wrong and the repair for each
A mechanic opens the hood of seven different cars. The engines look alike at first. Each has pistons, a fuel line and a cooling loop. But the details differ: where the fuel mixes, how the heat leaves, what happens when it gets cold. To pick a car for a trip, she needs a chart that lines the parts up. This chapter is that chart for world models, and then a list of the ways each engine fails on the road.
Slides 60 and 61 hold two tables. We join them here, because they answer the four design questions of Chapter 5: what the representation and history are, how the next step is predicted, how the model is learned, and what keeps the codes useful.
| Model | Representation and history | Next-step prediction | How it is learned | What constrains the codes |
|---|---|---|---|---|
| TD-MPC | Stacked frames, then a compact latent state | Deterministic latent transition | Joint, task-oriented | Consistency plus reward and value signals |
| V-JEPA 2-AC | Frozen features plus action and pose history | Point prediction (L1 training) | Pretrain, then freeze the encoder | Fixed pretrained target features |
| LeWorldModel | Jointly learned features plus causal history | Point prediction (MSE training) | Joint encoder and predictor | Predictive objective plus SIGReg |
| World Models (2018) | Frame VAE code plus RNN memory | MDN over the next code | VAE, then MDN-RNN | Reconstruction plus framewise VAE KL |
| RSSM / Dreamer V1 to V3 | Memory plus stochastic model state | Prior; posterior when observations arrive | Joint temporal inference and dynamics | Observation modeling plus prior and posterior agreement |
| IRIS | Discrete image codes plus causal history | Categorical next-token probabilities | Separate losses; repeated updates of both modules | Reconstruction through discrete codes |
| Photon-1 | Discrete frame-change codes plus causal history | Categorical next-token probabilities | Video pretraining, then action learning | Encoder training details not specified in the report |
Read down the columns. Three models predict one point. Two give a distribution, through a mixture or through a prior and posterior. Two give categorical odds over codes. Each model also answers the collapse question in its own way: fixed targets, task signals, a spread penalty, a KL term, or reconstruction. The lecture ends the pair of tables on one sentence: a learned representation becomes useful for world modeling through its temporal and action-conditioned role. A code is good when it helps predict the next step under an action, not when it merely rebuilds a picture. The lecture also warns that these are model-component comparisons, not complete agent descriptions.
You met this in Chapter 2. Training reads recorded inputs, z1, z2, and aims at z3. A free-running rollout reads a seed and then its own outputs, ẑ1, ẑ2, and produces ẑ3. The remedy on the slide: train on multi-step predictions that feed generated outputs back as inputs. V-JEPA 2-AC does a little of it, with teacher forcing plus a two-step rollout loss. The slide’s footer points to two further ideas, scheduled sampling and diffusion forcing, which also mix generated inputs into training.
The hero at the top is this slide. Let us derive why the rollout drifts. The truth is xt+1 = xt + at. The model is x̂t+1 = g x̂t + at. Define the error et = x̂t − xt. Subtract: et+1 = g x̂t − xt. Add and remove g xt: the right side is g(x̂t − xt) + (g − 1) xt. So:
et+1 = g et + (g − 1) xt
There are two terms. The second, (g − 1) xt, is the fresh mistake of this step. It is all there is if the model reads the true state, because then et = 0. The first, g et, is the old error passed forward and multiplied by g. With g = 1.08, the old error grows 8% each step on top of the new one. Run it: e1 = 0.08 × 1 = 0.08. Then e2 = 1.08 × 0.08 + 0.08 × 2 = 0.246. Keep going to e12 = 5.82, while the one-step error at step 12 is only 0.16. The lecture’s remedies: multi-step consistency in training, a validated rollout horizon, and state updates whenever a real observation arrives.
Two different problems share a slide. In the first, information is missing: the same observation arises from different histories, as in Chapter 1. The cure is relevant history and memory. In the second, the query lies outside the data coverage, a scene the model never saw. The cure is to expand coverage and to validate new conditions. The lecture’s sharp sentence: a richer output distribution cannot recover information that the representation discarded. A mixture on top of a code that lost the speed still has no speed to draw from. It adds that longer context and ensemble disagreement (several models trained apart, whose answers differ when the input is unfamiliar) can help, but neither guarantees reliable out-of-distribution prediction.
When the future is truly ambiguous, one rollout is one draw. The slide shows four imagined trajectories from the same first frame (IRIS, Figure 2). Use probabilistic dynamics and several samples. But the lecture adds a warning that this device tests: more randomness does not repair model bias. Evaluate the predicted distribution, not only one sample.
Blue shows where the real system goes, a band of one spread either side. The warm lines are sampled rollouts of the model. Add samples and watch the average. Then raise the bias.
Illustrative toy, ours. The truth moves +1 per step with a random wobble of 0.3 per step. The model moves 1 plus its bias per step, with its own noise. The idea, that sampling does not fix bias, is the lecture’s (slide 65).
With bias zero, the average of the samples closes in on the true mean as you add samples, and the fan has about the width of the blue band. With bias 0.05, every step is 5% too fast, and after 12 steps the average sits 0.6 beyond the truth, whatever the sample count. More draws shrink the noise in the average. They never remove a bias that every draw shares.
Chapter 8 counted H × K serial steps for token-by-token rollout. The repairs trade speed for something else: compression can lose information, caching uses memory or forgets history, and distillation needs more training. Always compare real latency, because steps differ in cost.
Chapter 10
Review the lecture on one page: the numbers, an evaluation checklist, the core code, and where the course goes next
You can now say, from memory, what Lecture 8 built. A latent world model encodes each observation into a code, keeps a predictive state from the history, and predicts the next code under a supplied action. A rollout feeds those predictions back in, so training on recorded inputs (teacher forcing) differs from running on your own outputs. The prediction can be one point, a categorical distribution, a Gaussian or a mixture, and the loss is the negative log likelihood. Seven models answer the design questions differently, and each has its own way to stop the codes from collapsing. Every rollout still has to face compounding error, missing information, unfamiliar inputs, many futures and the serial cost of tokens.
| Item | Value or meaning | Where |
|---|---|---|
| Toy codebook lookup | v = (0.6, −0.1) is nearest to e3 at squared distance 0.02 | 0, slide 7 |
| Rollout error at gain 1.08 | g = 1.08: one-step error 0.160, rollout error 5.823 at step 12 | Hero, 9, slide 63 |
| Error recursion | et+1 = g et + (g − 1) xt | 9, slide 63 |
| Discrete loss | ℒ = −log πk* | 3, slide 18 |
| Fixed-spread Gaussian | NLL = ‖y − μ‖2 / (2σ2) + C, so the best mean is E[y | c] | 3, slide 20 |
| Mixture loss | −log ∑k πk N(y*; μk, σk2) | 4, slide 23 |
| TD-MPC target | EMA copy, θ̄ ← (1 − τ)θ̄ + τθ, with stop gradient | 5, slide 29 |
| V-JEPA 2-AC loss | L1 teacher forcing plus L1 two-step rollout | 5, slide 34 |
| RSSM loss | −log p(o | h, u) + β KL(q ‖ p) | 7, slide 49 |
| Photon-1 size | 106B parameters, 5B active, 575M frames, 960 tokens per frame | 8, slide 57 |
| Serial cost | H frames × K codes is about HK steps | 8, 9, slide 66 |
The lecture ends with a discussion slide: choose one model and say which checks apply. Here are its five checks, each with the Chapter that explains why.
| Check | Concrete test | Why |
|---|---|---|
| Data split | Hold out entire episodes, not adjacent frames. | Neighbouring frames look alike, so a random split leaks the answer. |
| Rollout horizon | Same initial evidence and actions; compare one-step and H-step rollout. | Chapter 9: one-step error hides compounding. |
| Probabilistic output | Use held-out NLL when available; check calibration or coverage (do events given 70% odds happen about 70% of the time, and does the stated range contain the truth as often as it claims). | Chapters 3 and 4: score the whole distribution. |
| Representation scale | Raw latent MSE is not comparable across different encoders. | Chapter 5: shrinking the codes shrinks the loss. |
| Use conditions | Test new scenes and actions separately; measure generation latency. | Chapter 9: coverage and serial cost. |
Try it on one model. For IRIS you would hold out whole Atari episodes, compare the 1-step and 20-step rollouts from the same first frames, report token cross-entropy on held-out episodes, and time the generation. For TD-MPC you could not compare its latent MSE with LeWorldModel’s, since the two encoders have different scales. You would compare the control results and the rollouts decoded to the same space.
Three small pieces carry the whole lecture: the mixture loss, a teacher-forced training step, and a dream that feeds the model its own samples. This is a sketch for one code coordinate at a time, in PyTorch style, not a full implementation.
python import math, torch, torch.nn.functional as F def mdn_nll(logits, mu, raw_sigma, y): # logits, mu, raw_sigma: (B, M) y: (B, 1), the recorded next code log_pi = F.log_softmax(logits, dim=-1) # weights: positive, sum to 1 sigma = F.softplus(raw_sigma) + 1e-3 # positive scales, with a floor log_n = -0.5*((y - mu)/sigma)**2 - torch.log(sigma) - 0.5*math.log(2*math.pi) return -torch.logsumexp(log_pi + log_n, dim=-1).mean() # score every bell; no sampling def train_step(model, z, a): # z: (B, T+1, D) recorded codes h, loss = model.init_state(z.size(0)), 0.0 for t in range(z.size(1) - 1): h, p = model.step(h, z[:, t], a[:, t]) # input is the RECORDED z_t loss = loss + mdn_nll(*p, z[:, t+1]) # target is the recorded z_{t+1} return loss / (z.size(1) - 1) @torch.no_grad() def dream(model, z0, actions): # no camera: the model reads itself h, z, out = model.init_state(z0.size(0)), z0, [z0] for a in actions: h, (logits, mu, raw_sigma) = model.step(h, z, a) k = torch.distributions.Categorical(logits=logits).sample() # pick a bell sigma = F.softplus(raw_sigma.gather(-1, k[:, None])) + 1e-3 z = mu.gather(-1, k[:, None]) + sigma * torch.randn_like(sigma) # draw from it out.append(z) # z feeds the next step return torch.stack(out, dim=1)
Find the three ideas in the code. The first function never samples (Chapter 4). The second feeds recorded codes (Chapter 2, teacher forcing). The third feeds its own samples, which is exactly why Chapter 9’s failures appear.
This lecture is the middle of a bridge. Lecture 7 gave us latent variables, the ELBO and a first look at World Models. Lecture 4 gave us the state-space and belief ideas that Chapter 1 reused. The lecture’s last line names the next step: use predicted futures for planning, then learn behavior with world models. Lecture 9 takes up planning and control, and Lecture 10 turns to policies and value with world models.
Credit: this lesson re-teaches Lecture 8 of Penn CIS 6280: World Models (Fall 2026) by Jiatao Gu. The structure follows the lecture’s own, and every device and example is ours except where a note says the number is the lecture’s. The slides are on the course site, and the papers named in each chapter are the primary sources.
Now press Teach and explain, from memory, how a latent world model is built and run: the code, the memory, the prediction rule, the loss, and the rollout. Then go back to the instrument at the top, set the gain to 1.15, and say why the warm line leaves the green one.