Flow & Diffusion Models

Chapter 1: Generation Is Sampling

Step one is purely about representation. A computer cannot store "a dog" — it stores numbers. So every object we ever want to generate, we first flatten into a vector of real numbers, a point in some space R^d (d-dimensional space):

So our target object is a point z ∈ R^d. Now: what makes a good z? The slides line up four dog images — "useless," "bad," "wrong animal," "great!" — and ask the key question: those are subjective words. Can we make "good" objective?

The trick: let frequency define quality

Here is the move. Imagine the gigantic distribution of all images on the internet. A crisp, well-lit dog photo is very likely to appear there; a smear of random static is essentially impossible. So we declare:

With that definition, generation becomes sampling: to "generate an object," draw a fresh sample from the data distribution.

Read that as: "z is sampled from p-data." A great dog photo is a high-probability draw; we want our algorithm to land in those high-density regions.

Spell out the dataset precisely: it is a finite collection

That is the only grip we have on p_data. Training will mean: squeeze as much information as we can out of these samples so that, at the end, we can produce more samples that look like they came from the same source.

Conditioning on a prompt

"A photo of a dog" vs. "a photo of a cat" vs. "a landscape" are different distributions. So we generalize: a condition variable y (the prompt, a class label, a text caption) selects a conditional data distribution p_data(· | y). Generating from a prompt means sampling that conditional distribution:

Chapter 2: The Plan — Push Noise Into Data

We can't sample p_data directly. But there is one distribution we can always sample, trivially, as many times as we like: a standard Gaussian. So here is the entire architecture of every flow and diffusion model:

Read it left to right: start at a random Gaussian point X₀ at time t = 0; transform it; arrive at a data sample X₁ at time t = 1. The whole game is to build that arrow in the middle. The defining choice of flow/diffusion models: the transformation is not one big jump — it is a continuous journey through time, governed by a differential equation.

Two questions now stand between us and a working generator, and they are the rest of this lecture:

1. What mathematical object describes "transport points continuously through time"? Answer: a differential equation — an ODE for smooth deterministic motion (Chapters 3–5), or an SDE for noisy motion (Chapters 6–8).
2. How do we run it on a computer? Answer: numerical simulation — tiny time-steps (the Euler method), Chapter 4.

Chapter 3: ODEs, Vector Fields & Flows

"Transport a point continuously through time" is the job of an ordinary differential equation (ODE). Three objects describe the same idea from three angles. Get all three and the rest of the course is downhill.

1. The trajectory

A trajectory is the path a single point takes. It is a function of time:

For each time t between 0 and 1, X_t is where the point is. (We use the time window [0,1] by convention — t = 0 is noise, t = 1 is data.)

2. The vector field — the arrows

A vector field u assigns, to every location x and every time t, a velocity — an arrow saying "if you are here at this time, move this way, this fast":

Think of u_t(x) as a wind map, or the current in a river that changes over the day. It is the only thing we get to design — later, a neural network is this vector field.

3. The ODE itself — "follow the arrows"

The ODE is the rule connecting the two: at every instant, the trajectory's velocity must equal the arrow at its current location. "Velocity" is the time-derivative, so:

The first line says "go where the arrow at your feet points." The second pins down where you start. Without a starting point the path is not determined — the same wind map produces a different journey from a different origin.

So: vector fields define ODEs, whose solutions are flows. Three names, one underlying object. The vector field is the wind; the ODE is the law "drift with the wind"; the flow is the resulting map that warps all of space, like a fluid carrying a grid of dye.

Do solutions even exist? (Yes — relax)

A careful person asks: given a vector field, is there always a trajectory, and is it the only one? The Picard–Lindelöf theorem answers yes, under mild conditions:

Why you can stop worrying: we will parameterize u with a neural network, and neural networks always have bounded derivatives. So in every case we care about, the flow exists and is unique. This theorem is good news you can file away, not an obstacle.

Worked example: the linear field, solved exactly

Pick the simplest non-trivial field, a contraction toward the origin: u_t(x) = −θx for some θ > 0. Claim: the flow is

Let's prove it by checking the two defining properties of a flow — every step shown:

Plug in numbers (we'll reuse these in the code lab): θ = 1, x₀ = 2. Then ψ_t(2) = 2·exp(−t). At t = 1: 2·exp(−1) = 2⁄e ≈ 0.7358. That exact number is our ground truth for testing a numerical integrator.

Chapter 4: Simulating an ODE — Euler's Method

The linear field had a closed-form flow. Almost nothing else does — a neural-network vector field has no formula for its flow. So we simulate: approximate the continuous journey by many tiny straight steps. The simplest, most intuitive scheme is the Euler method.

The idea is one line of common sense. You are at X_t. The arrow there says "move in direction u_t(X_t)." Take a small step of size h in that direction:

Here h = 1/n is the step size and n is the number of steps. Repeat from t = 0 up to t = 1. That's it — "small step in the direction of the arrow," over and over.

Worked example: Euler on the linear field, by hand

Field u(x) = −x (so θ = 1), start x₀ = 2, integrate to t = 1. The exact answer (Chapter 3) is 0.7358. Watch Euler approach it as we halve the step:

Halving h roughly halved the error — Euler is a first-order method (error shrinks proportionally to h). That trade-off — accuracy vs. number of steps — is the central practical knob in diffusion sampling, where each step is an expensive neural-network call.

Now build the integrator yourself. This is the literal engine of every sampler in the course — the step function you write here is reused, unchanged, all the way to Lecture 4.

Chapter 5: Flow Models — an ODE With a Neural-Net Engine

We now assemble the first real generative model. Take the ODE machinery and make two changes that turn it from "a wind map" into "an image generator."

Putting it together, a flow model is the ODE:

and the goal of training is to make the endpoint land on the data distribution:

Algorithm 1 — Sampling from a flow model

This is the entire inference procedure. Memorize its shape; everything in the course either trains the network inside it or makes its steps cheaper.

# Sample from a flow model with the Euler method
# Input: trained vector-field network u_theta(x, t); number of steps n
t = 0.0
h = 1.0 / n                      # step size
x = sample_gaussian()            # X_0 ~ p_init = N(0, I)
for i in range(n):
    x = x + h * u_theta(x, t)    # one Euler step along the learned arrows
    t = t + h
return x                        # X_1, a sample of the data distribution

Read it against Algorithm 1 in the slides: draw a Gaussian, then loop n Euler steps, return the endpoint. The only difference from Chapter 4's hand example is that u_theta is a trained network instead of −θx. The "Generate" button in Chapter 2 ran exactly this loop with a toy field.

Chapter 6: Randomness — Brownian Motion

Flow models are deterministic after the start. Diffusion models add randomness at every step. To build them we need the canonical source of continuous randomness: Brownian motion (also called the Wiener process, hence the symbol W — after Norbert Wiener, who taught at MIT).

Picture a stochastic process: a trajectory that is random. Run it twice from the same start and you get two different paths, because the dynamics themselves roll dice. Brownian motion is the continuous-time limit of a random walk — the path a speck of pollen traces as water molecules kick it.

How do we simulate it? Discretize time into steps of size h. Each step, add a Gaussian kick whose variance is h (matching "variance = elapsed time"). Since standard normal noise ε has variance 1, scaling by √h gives variance h:

Now simulate it yourself and confirm the signature law Var[W_t] = t — the property that defines Brownian motion.

Chapter 7: SDEs — ODEs With a Noise Term

A stochastic differential equation (SDE) is just an ODE with Brownian noise stirred in at every step. To see exactly where the noise slots in, we first rewrite the ODE without derivatives (because for a random, jagged path the derivative no longer exists in the usual sense).

Rewriting the ODE as small steps

Start from d⁄dt X_t = u_t(X_t). Use the definition of the derivative as a difference quotient and rearrange — every step shown:

where R_t(h) is an error term that vanishes as h → 0. This just restates Euler: "each instant, take a small step of size h in direction u_t." Nothing new — but now there are no derivatives, so we can add randomness.

Adding the noise

Amend the update: take the deterministic step and a Brownian kick, scaled by a diffusion coefficient σ_t ≥ 0 that controls how much noise to inject:

This is the Euler–Maruyama update — the SDE analogue of Euler. Since the Brownian increment W_t+h − W_t = √h·ε (Chapter 6), in code it reads:

Mathematicians write the same thing in the compact symbolic form (just shorthand for the limit of the update above):

Here u_t is the drift (the deterministic pull) and σ_t is the diffusion (the noise strength). Two facts make SDEs easy to live with:

You already wrote the deterministic x + h*drift step (Chapter 4) and the √h Brownian kick (Chapter 6). The next code lab (Chapter 8) snaps them together into Euler–Maruyama — the complete diffusion sampler.

Chapter 8: Diffusion Models & the Ornstein–Uhlenbeck Process

A diffusion model is a generative model built from an SDE: random Gaussian start, a learned drift u^θ_t, a noise schedule σ_t, simulated with Euler–Maruyama. To build intuition for how drift and noise interact, we study the cleanest non-trivial SDE — the Ornstein–Uhlenbeck (OU) process.

The remarkable result: the OU process converges to a Gaussian whose variance is set by the tug-of-war between pull and push:

Sanity-check the extremes. More noise σ → wider stationary spread (σ² on top). Stronger pull θ → tighter spread (2θ on the bottom). And if σ = 0 (no noise), the variance is 0 — every path collapses deterministically to the origin, recovering the contracting flow from Chapter 3. Same equation, dial in between.

Now assemble everything: implement the OU drift and the full Euler–Maruyama step, then verify the stationary variance σ²/(2θ).

ODE sampling vs. SDE sampling

The same target can often be reached by an ODE (deterministic, σ = 0) or an SDE (stochastic, σ > 0). ODE sampling is reproducible and usually needs fewer steps; SDE sampling injects fresh noise each step, which can improve sample diversity and quality and "self-correct" errors. Lecture 3 shows the two are deeply linked through the score function. For now: flow = ODE, diffusion = SDE, and ODE is the σ = 0 corner of the SDE world.

Chapter 9: Connections & Cheat Sheet

You built the entire engine of modern generative AI from first principles — with no neural network yet. Here is the whole lecture on one page.

Cheat sheet

What's next in this series

• Lecture 2 — Flow Matching. The training problem: how do you choose the vector field so the endpoint lands on p_data? Conditional probability paths and the flow matching loss.
• Lecture 3 — Score Matching & Guidance. The score function (∇log p), how it links ODE and SDE samplers, and classifier-free guidance for prompts.
• Lecture 4 — Architectures & Latent Spaces. U-Nets, Diffusion Transformers, and working in a VAE's latent space (Stable Diffusion 3, Movie Gen).
• Lecture 5 — Discrete Diffusion. Carrying these ideas to text and language models.

Related lessons on this site

• Diffusion Models (Gleam) — the DDPM view of the same machine.
• Flow Matching (Gleam) — a focused take on the training target (Lecture 2's topic).
• Diffusion (Architecture Atlas) — bird's-eye overview.
• SDEs & Continuous Diffusion (Deep-Dive) — the rigorous continuous-time treatment.