Flow Matching

Chapter 0: Why — Who Draws the Arrows?

Where Lecture 1 left us. We built the engine: a vector field u_t(x) of arrows in space, an ODE that says "follow the arrows," and the Euler integrator that drives a Gaussian noise point along those arrows from t = 0 to t = 1. Press a button, watch noise flow into a target shape. But there was a cheat: we drew the arrows by hand. For a toy ring of clusters that's easy. For "a photo of a dog" in 196,608 dimensions, nobody can hand-draw the field. Lecture 2 answers the question we dodged: how do you LEARN the arrows from data?

Recall the flow model from Lecture 1. Start at a random Gaussian point, follow a neural-network vector field u^θ_t(x) for unit time, and you want the endpoint to be a fresh data sample:

X₀ ∼ p_init → d⁄dt X_t = u^θ_t(X_t) → X₁ ∼ p_data (the goal)

Here θ are the network weights and p_init is the easy standard Gaussian N(0, I_d). An untrained network produces garbage — random arrows transport noise to more noise. Training means finding weights θ so the endpoint distribution actually equals the data distribution p_data.

The whole lecture in one sentence. We will construct a specific "correct" target vector field — one we can prove turns noise into data — and then train the network by plain regression to copy it. The twist: the correct target is an intractable average we can never compute, yet we will find a tractable stand-in with the exact same gradient, so regressing on the stand-in trains the network as if we had the real thing. That stand-in objective is Flow Matching.

The construction has a beautiful skeleton called the Flow Matching matrix — a 2×3 grid we will fill in cell by cell over the next chapters:

The Flow Matching matrix (our roadmap).

	Probability Path	Vector Field	FM Loss
Conditional per single data point	p_t(x \| z) — Ch 2	u_t(x \| z) — Ch 3	L_CFM — Ch 5
Marginal across all data	p_t(x) — Ch 1	u_t(x) — Ch 4	L_FM — Ch 5

The top row is tractable (closed-form formulas you can evaluate). The bottom row is what we actually want but cannot compute. Flow Matching is the bridge: train on the top row, get the bottom row for free.

One sentence of vocabulary now, because it organizes everything. "Conditional" means "for one fixed data point z" — e.g. build a path that ends exactly at this one cat photo. "Marginal" means "averaged over the whole data distribution" — the thing that ends at any plausible cat photo. Conditional objects are simple because they target a single point; marginal objects are what generate real samples but require summing over all data.

Honest scope check. Today we learn the vector field (the ODE / flow case). Lecture 3 adds the score and turns this into the full SDE / diffusion training story plus guidance. Everything here is "simulation-free" — remarkably, we never run the Euler integrator during training. That is the punchline, and it is what makes Flow Matching cheap enough to train Movie Gen.

In Lecture 1 we could already sample from a flow model. What exactly is missing — what does Lecture 2 add?

A faster Euler integrator with fewer steps. A way to train the vector field — to find network weights so the hand-drawn arrows are replaced by learned arrows that transport noise to the data distribution. A way to store the training images so we can retrieve them. A bigger Gaussian for the initial distribution.

Chapter 1: A Probability Path — the Road From Noise to Data

The flow model only pins down the two endpoints: at t = 0 we have noise, at t = 1 we have data. But what should the distribution look like at t = 0.5? Lecture 1 never said. Flow Matching's first move is to choose what happens in between — to lay down a road through the space of distributions. That road is a probability path.

Definition — a probability path p_t. A probability path is a smoothly time-varying family of distributions, one for each time t in [0, 1], that interpolates from the initial (noise) distribution to the data distribution:

p₀ = p_init (noise) p₁ = p_data (data)

Think of each p_t as a snapshot — a fog of probability mass — and the path as a movie of that fog morphing from a featureless Gaussian blob into the structured data cloud.

An analogy that will pay off: imagine a weather map of fog density over a landscape, filmed over a day. At dawn (t = 0) the fog is a single smooth Gaussian hill. By dusk (t = 1) it has gathered into the exact shapes of your data — five tight clusters, say. The probability path is that whole time-lapse film.

The subtle point that trips everyone up — read twice. A probability path specifies only the marginals: what the fog looks like at each instant. It says nothing about how an individual particle moves. Two completely different sets of arrows (vector fields) can produce the exact same time-lapse of fog. The path is "what distribution at each time"; the vector field is "which way each particle drifts." Choosing the path first, then finding arrows that realize it, is the entire strategy of Flow Matching.

Why introduce a whole path instead of just the endpoints?

Because the endpoints alone are a hopeless target. "Find arrows that move this Gaussian to that data cloud" is a global, tangled constraint — you'd have to reason about the entire transport at once. But if you also dictate every intermediate snapshot, the problem becomes local in time: at each instant you only need arrows that nudge the current fog into the next instant's fog. Local problems are the ones we can solve with calculus and closed forms.

So the recipe is now:

Step 1

Pick a probability path p_t from noise to data (this chapter & next).

↓

Step 2

Find the vector field whose arrows make particles follow that path (Ch 3–4).

↓

Step 3

Train a network to copy that vector field by regression (Ch 5).

One catch we will confront head-on in Chapter 2: we still cannot directly write down a path that ends at the full data distribution p_data, because we don't know p_data — we only have samples. The fix is to first build an easy path that ends at a single data point, then average. That is the conditional/marginal split, next chapter.

A probability path — fog morphing from noise to data

Each frame is a snapshot p_t of the distribution (a 1-D density curve here for clarity). Drag t from 0 to 1: at t = 0 the density is a single broad Gaussian (noise); as t grows it morphs into the structured target (a two-bump "data" density). This is the marginal path — what we eventually want. Notice it specifies only the shape at each time, not how any one particle travels.

time t0.00

A probability path p_t is said to "specify only the marginals." What does that mean, and why does it matter?

It means the path also fixes each particle's trajectory uniquely. It means the path only works for marginal data, not conditional. It means the path fixes the distribution at each instant (the fog snapshot), but NOT how individual particles move — many different vector fields can realize the same path. It means the endpoints are undefined.

Chapter 2: The Conditional Path — One Data Point at a Time

We hit a wall at the end of Chapter 1: we can't write a path ending at the full data distribution, because we don't know it. Flow Matching's escape is almost too simple. Pick a single data point z from the training set, and build an easy path that ends exactly at z. That is a conditional probability path, written p_t(x | z) — "the distribution at time t, given that we are heading toward the data point z."

Definition — conditional probability path p_t(x | z). For each fixed data point z, a family of distributions over t in [0,1] such that

p₀(· | z) = p_init (noise) p₁(· | z) = δ_z (a spike at z)

Here δ_z is the Dirac delta — the simplest distribution imaginable: sampling from it always returns z, with zero spread. So a conditional path starts as the broad noise Gaussian and collapses to a single point z by t = 1.

Why is this so much easier? Because the destination is a single point, not a complicated cloud. There's an obvious, hand-buildable path: start with a wide Gaussian centered near the origin, and shrink it down while sliding its center onto z. That's the Gaussian conditional path, and it is the path used by essentially every state-of-the-art model.

The Gaussian conditional path. Introduce two noise schedules — smooth functions α_t and β_t of time, with the boundary values α₀ = 0, α₁ = 1 and β₀ = 1, β₁ = 0. Define

p_t(· | z) = N(α_t z, β_t² I_d)

A Gaussian whose mean is α_t z (slides from 0 toward z as α_t: 0→1) and whose standard deviation is β_t (shrinks from 1 toward 0 as β_t: 1→0).

Read the symbols carefully — α_t is the data-mixing dial (how much of z is in the mean) and β_t is the noise level (the spread). Check the endpoints by plugging in:

Endpoint check.
At t = 0: mean = α₀ z = 0·z = 0, std = β₀ = 1. So p₀(·|z) = N(0, I_d) = p_init. ✓ Pure noise, independent of z.

At t = 1: mean = α₁ z = 1·z = z, std = β₁ = 0. A Gaussian with zero spread centered at z is the spike δ_z. ✓ Collapsed onto the data point.

How to sample a point on this path (the reparameterization)

This is the workhorse formula of the whole lecture. A Gaussian with mean α_t z and standard deviation β_t can be sampled by taking standard noise ε (drawn from N(0, I_d)) and shifting/scaling it:

x_t = α_t z + β_t ε, ε ∼ N(0, I_d)

In words: a noisy data point is a blend of the clean data z (weighted α_t) and pure noise ε (weighted β_t). At t = 0 it's all noise (α = 0, β = 1 ⇒ x = ε); at t = 1 it's all data (α = 1, β = 0 ⇒ x = z). In between it's a interpolation. This single line is what we run millions of times during training.

Why β_t² and not β_t in the distribution? A normal distribution is parameterized by its variance (spread-squared), so the covariance is β_t² I_d. But when you sample by scaling standard noise ε, you multiply by the standard deviation β_t (the square root of the variance). Same object, two faces — this is exactly the √h vs. h subtlety from Lecture 1's Brownian motion. Mixing them up is the most common bug.

Now the conditional path for a single z. To get the path for all data, average over z — that's Chapter 4. For now, here is a single conditional path, animated.

A conditional path collapsing onto one data point z

The orange star is the chosen data point z. The cloud is a sample from p_t(·|z) = N(α_t z, β_t² I). Drag t: at t = 0 it's a broad Gaussian centered at the origin (all noise, doesn't know z); as t grows the cloud slides toward z (mean α_t z) and tightens (std β_t), collapsing to a spike at z by t = 1. Schedule used: α_t = t, β_t = 1−t.

time t0.30

Now implement the sampling formula yourself and confirm both endpoints — this is Code Lab 1.

In the Gaussian conditional path x_t = α_t z + β_t ε, what are the roles of α_t and β_t at the two endpoints?

α is the spread and β is the mean. α_t weights the data z (0 at t=0 → 1 at t=1) and β_t weights the noise ε (1 at t=0 → 0 at t=1): so t=0 is pure noise, t=1 is the clean point z. Both are constants equal to 1. They are the network weights being trained.

Chapter 3: The Conditional Vector Field — Arrows That Realize the Path

We have a conditional path: a movie of the fog collapsing onto z. Chapter 1 warned us that a path only says what distribution at each time, not how particles move. Now we find the arrows. We want a conditional vector field u_t(x | z) such that if you start a particle at a noise point and follow these arrows, the particle's distribution at time t is exactly p_t(· | z).

What "realizes the path" means precisely. The conditional vector field u_t(x | z) is any vector field whose ODE, started from noise, keeps the particle distributed like the conditional path:

X₀ ∼ p_init, d⁄dt X_t = u_t(X_t | z) ⇒ X_t ∼ p_t(· | z) for all t.

The miracle of the Gaussian path: we can write this field down in closed form by hand. No learning needed for the conditional field — it's pure algebra.

The trick: build the conditional FLOW first, then differentiate

Recall from Lecture 1 that the flow ψ_t(x₀) is the map "where does a particle starting at x₀ end up at time t," and that the vector field is the time-derivative of the flow along its own trajectory. For the Gaussian path we can guess the flow directly. We want a map that takes a standard-noise start x₀ and produces a sample of p_t(·|z) = N(α_t z, β_t² I). But that's exactly the reparameterization from Chapter 2 with x₀ playing the role of ε:

ψ_t(x₀ | z) = α_t z + β_t x₀

Check it: if X₀ ∼ N(0, I), then α_t z + β_t X₀ is Gaussian with mean α_t z and variance β_t² — precisely p_t(·|z). ✓ So this map carries the noise distribution onto the conditional path. Now extract the vector field by differentiating the flow in time.

Deriving the field, every step shown

By the definition of a flow, the velocity of a trajectory equals the field evaluated at the trajectory's location. Write x = ψ_t(x₀ | z) for the current location. We need two pieces: the time-derivative of the flow, and a way to express x₀ in terms of the current x.

Step 1 — differentiate the flow in time. Use α̇_t and β̇_t for the time-derivatives of the schedules (a dot means d⁄dt). Differentiating ψ_t(x₀|z) = α_t z + β_t x₀:

d⁄dt ψ_t(x₀|z) = α̇_t z + β̇_t x₀

Step 2 — solve the flow for the start point. From x = α_t z + β_t x₀, isolate x₀:

x₀ = (x − α_t z) ⁄ β_t

Step 3 — substitute and collect terms. Put the expression for x₀ into Step 1. The velocity, now written as a function of the current location x, is the conditional vector field:

u_t(x | z) = α̇_t z + β̇_t · (x − α_t z) ⁄ β_t

Step 4 — gather the z and the x. Expand and group:

u_t(x | z) = ( α̇_t − (β̇_t⁄β_t) α_t ) z + (β̇_t⁄β_t) x

That is the closed-form conditional Gaussian vector field. Stare at its two pieces — the meaning is the whole point:

Reading the formula. The arrow at location x is a sum of two pulls:

A data-pull toward z: the term ( α̇_t − (β̇_t⁄β_t) α_t ) z drags the particle in the direction of the target data point z.
A radial term in x: the term (β̇_t⁄β_t) x scales the current position — since β_t shrinks (β̇_t < 0), this contracts the cloud, squeezing out the noise.

Drop the data-pull and the particle never learns where z is — it just contracts to the origin. Both terms are essential.

"Isn't a conditional field useless?" A fair worry: every trajectory of u_t(·|z) ends at the same point z. So if you sampled with it you'd just regenerate one training example over and over — no diversity, no generation. Correct — the conditional field alone is useless for sampling. Its job is to be a tractable building block. Chapter 4 averages these per-point fields over the whole dataset to get the marginal field, which does generate fresh, diverse data. Hold this thought.

Let's confirm the formula is right the way an engineer should: numerically. If u_t(x|z) really transports the path, then the time-derivative of a sample x_t = ψ_t(x₀|z) (computed by finite differences) must equal u_t evaluated at that sample. That's Code Lab 2.

The conditional Gaussian vector field has two terms. What does the term proportional to z do, and what goes wrong if you drop it?

It is the data-pull that drags the particle toward the target z; without it the field only contracts toward the origin and never reaches z. It is the step size; without it the ODE runs backward. It is pure noise injection; without it the path becomes deterministic. Nothing — it is a redundant term you can always drop.

Chapter 4: The Marginal Vector Field — Averaging Over Data

We have, for each data point z, a tractable conditional field u_t(x | z) that aims a particle straight at z. Individually useless (every trajectory lands on the same z). Now the master stroke: average the conditional fields over the whole dataset to get a field that generates the real data distribution. This is the marginal vector field, and the theorem that it works is the marginalization trick.

First, the marginal path it follows

The marginal probability path p_t(x) is what you get by first picking a random data point z, then sampling its conditional path:

z ∼ p_data, x ∼ p_t(· | z) ⇒ x ∼ p_t (sampling)

As a density, that is a weighted average of the conditional densities — an integral over all data points weighted by how likely each is:

p_t(x) = ∫ p_t(x | z) p_data(z) dz

Check the endpoints: since every conditional path starts at p_init and ends at δ_z, the average starts at p_init (Chapter 1's noise) and ends at the average of spikes over the data — which is exactly p_data. ✓ So p_t is a genuine noise-to-data path. We can sample from it (pick z, add noise) but we cannot evaluate its density — that integral is intractable. Remember this asymmetry; it drives everything.

The Marginalization Trick (Theorem). Define the marginal vector field as the conditional fields averaged with a special weighting:

u_t(x) = ∫ u_t(x | z) · [ p_t(x | z) p_data(z) ⁄ p_t(x) ] dz

Then this field follows the marginal path: starting from noise and following u_t gives X_t ∼ p_t for all t — and in particular X₁ ∼ p_data. So u_t converts noise into data. This is the field we want our network to learn.

What is that weird weighting term?

The bracket is the secret sauce. By Bayes' rule, the quantity p_t(x | z) p_data(z) ⁄ p_t(x) is exactly the posterior p_t(z | x) — "given that I observe a noisy point x at time t, how likely is it that it came from data point z?" So the marginal field is a posterior-weighted average of velocities:

The intuition, in words. Stand at a noisy point x. Every data point z would pull you in a different direction u_t(x | z) — straight toward itself. The marginal field is the weighted average of all those pulls, where each pull's weight is "how plausible is it that this noisy x is a noised version of z." Nearby, plausible data points get a big vote; far-fetched ones get almost none. The result is an arrow that points toward the plausible data, which is precisely what we need to denoise.

An analogy: x is a blurry photo. Each clean training image z "claims" the blurry photo could be its corrupted version, and offers a direction to sharpen toward itself. The marginal field is the consensus direction, weighting each claim by how believable it is. Early (t near 0) the photo is almost pure noise, so many images claim it weakly — the arrow is gentle and broad. Late (t near 1) the noise is small, one image dominates the posterior, and the arrow points sharply at it.

Why we can't just use this formula directly. The marginal field needs p_t(x) in the denominator — the intractable integral we just flagged. And it needs an integral over the entire data distribution for every single x. So although we have a formula, we cannot compute it. This is the crisis. Chapter 5 resolves it: we never compute u_t; we train against the cheap conditional field instead, and prove that's equivalent.

Conditional vs. marginal field — one pull vs. the consensus

Three data points (orange stars). Conditional shows the field for ONE chosen z — every arrow points straight at it (a single pull). Marginal shows the posterior-weighted average over all three — arrows point toward whichever data point is most plausible from that location, so the field splits the plane into basins. Toggle the mode and drag t: early t gives soft, broad arrows; late t snaps each region toward its nearest data point.

mode

time t0.50

The marginal vector field is a posterior-weighted average of conditional fields. What is each conditional field's weight at a noisy point x?

A constant 1⁄N for every data point. The distance from x to the origin. The posterior p_t(z | x) — how plausible it is that the noisy point x came from data point z; nearby/likely z get a large weight, far/unlikely z almost none. The network weights θ.

Chapter 5: The Flow Matching Loss — Regress on the Tractable Twin

We want our network u^θ_t(x) to equal the marginal field u_t(x). The obvious loss is mean-squared error: penalize the squared distance between the network's arrow and the target arrow, averaged over random times and random points on the path.

The Flow Matching loss (what we wish we could optimize).

L_FM(θ) = E_{t, x} [ ‖ u^θ_t(x) − u_t(x) ‖² ]

where t is drawn uniformly on [0,1] and x is drawn from the marginal path p_t. The double-bars-squared is the squared length of the difference vector; E means average. Problem: the target u_t(x) is the intractable marginal field from Chapter 4. We literally cannot evaluate this loss.

The fix is audacious: replace the intractable target with the tractable conditional field — and also sample x by first picking a data point z. Define the Conditional Flow Matching loss:

The Conditional Flow Matching loss (what we actually optimize).

L_CFM(θ) = E_{t, z, x} [ ‖ u^θ_t(x) − u_t(x | z) ‖² ]

where t is uniform, z ∼ p_data (a real training point), and x ∼ p_t(·|z) (just add noise: x = α_t z + β_t ε). Every term here is computable. No intractable integral, no density evaluation — just sample a data point, sample noise, and regress.

But here's the puzzle that stops everyone: we want the network to match the marginal field, yet we are training it to match the conditional field. The conditional field is "useless" — it points at one fixed data point. How can copying it possibly give us the marginal field? The answer is a small miracle.

The theorem that makes Flow Matching work. The two losses differ only by a constant that does not depend on θ:

L_FM(θ) = L_CFM(θ) + C, so ∇_θ L_FM = ∇_θ L_CFM

The gradients are identical. Since gradient descent only ever looks at the gradient, minimizing the tractable L_CFM drives the weights to exactly the minimizer of L_FM — the network converges to the marginal field. The constant C is invisible to the optimizer.

Proof — why the two losses share a gradient

It's just algebra on the squared norm. Expand ‖a − b‖² = ‖a‖² − 2 a·b + ‖b‖² with a = u^θ_t(x) and b the target field. Do it for both losses and compare the three pieces:

Expanding L_FM:

L_FM = E[ ‖u^θ‖² ] − 2 E[ u^θ · u_t(x) ] + E[ ‖u_t(x)‖² ]

The first term uses only x ∼ p_t; the third term has no θ at all — call it a constant C₁.

The crucial middle term. Write the average over the marginal path as an integral, substitute the marginalization-trick definition of u_t(x) (Chapter 4), and the intractable p_t(x) in the numerator cancels the one in the denominator:

E_{x∼p_t}[ u^θ · u_t(x) ] = E_{z, x∼p_t(·|z)}[ u^θ · u_t(x | z) ]

The left side uses the marginal field; the right uses only the conditional field. That swap is the heart of the proof.

Expanding L_CFM:

L_CFM = E[ ‖u^θ‖² ] − 2 E[ u^θ · u_t(x|z) ] + E[ ‖u_t(x|z)‖² ]

The first two terms are now identical to L_FM's first two (using the swap above). Only the third terms differ — and both third terms are constants in θ. Therefore L_FM − L_CFM = C, a constant. The gradients match. □

What this buys us (the engineering payoff). Training reduces to a plain supervised regression with three lines of sampling:

draw a real data point z; draw a random time t; draw noise ε;
form the noisy point x = α_t z + β_t ε;
minimize ‖ u^θ_t(x) − u_t(x | z) ‖² by gradient descent.

No ODE is ever simulated during training — this is the celebrated simulation-free property. It's why Flow Matching scales to Stable Diffusion 3 and Movie Gen.

Data-flow check (Concept + Realization). Per training step: z is a data vector in R^d; ε is noise in R^d; x = α_t z + β_t ε is in R^d. The network u^θ_t(x) takes x (R^d) plus the scalar t and outputs a velocity in R^d — same shape as x. The conditional target u_t(x|z) is computed from the closed form of Chapter 3 — cheap arithmetic on z, x, and the schedules. The loss is one scalar; backprop updates θ. Inference (Lecture 1's Euler sampler) is a separate phase that runs the trained field. Training never touches it.

Why is it valid to train the network on the conditional field u_t(x|z) when what we actually want is the marginal field u_t(x)?

Because the conditional and marginal fields are literally equal. Because the marginal field is easier to compute than the conditional field. Because the two losses differ only by a θ-independent constant, so they have the same gradient — minimizing the tractable conditional loss drives the network to the marginal field anyway. Because gradient descent ignores the loss and uses only the data.

Chapter 6: Gaussian Paths Concretely — Schedules and the Loss

Time to make everything numeric. We pick concrete noise schedules α_t and β_t, plug them into the conditional field, and watch the famously simple Flow Matching loss fall out.

Noise schedules, the contract. α_t and β_t are any smooth, monotonic functions with

α₀ = 0, α₁ = 1 (data dial: off → on) β₀ = 1, β₁ = 0 (noise dial: on → off)

Different choices give different paths but the same endpoints. Two popular families: the linear / CondOT schedule α_t = t, β_t = 1 − t (Chapter 7), and the square-root schedule α_t = t, β_t = √(1 − t) (used in the lab notebook).

The conditional field in schedule form

Drop the schedule derivatives into Chapter 3's formula. With α̇_t and β̇_t the time-derivatives:

u_t(x | z) = ( α̇_t − (β̇_t⁄β_t) α_t ) z + (β̇_t⁄β_t) x

This is fully closed-form — given the schedules and their derivatives (both known analytically), you can evaluate it for any x, z, t with elementary arithmetic. That tractability is the engine of the conditional loss.

The conditional loss, simplified by sampling x = α_t z + β_t ε

Here's the trick that makes the loss beautiful. Instead of sampling x and then computing u_t(x|z), substitute the reparameterization x = α_t z + β_t ε directly into the conditional field and watch it collapse. Plug x = α_t z + β_t ε into the formula above:

Substituting and simplifying:

u_t(x|z) = ( α̇_t − (β̇_t⁄β_t) α_t ) z + (β̇_t⁄β_t)( α_t z + β_t ε )

The (β̇_t⁄β_t)α_t z terms cancel (one is subtracted, one added), and (β̇_t⁄β_t)·β_t ε = β̇_t ε. What survives is stunningly simple:

u_t(x|z) = α̇_t z + β̇_t ε

So when you generate the training point as a noise+data blend, the target velocity is just the same blend of schedule derivatives: how fast the data dial is opening times z, plus how fast the noise dial is closing times ε. The Conditional Flow Matching loss becomes:

L_CFM(θ) = E_{t, z, ε} [ ‖ u^θ_t(α_t z + β_t ε) − (α̇_t z + β̇_t ε) ‖² ]

Worked number (linear schedule). Take α_t = t, β_t = 1 − t, so α̇_t = 1 and β̇_t = −1. The target becomes α̇_t z + β̇_t ε = (1)z + (−1)ε = z − ε — a constant pointing from the noise to the data, independent of t! Concretely with z = 3, ε = 0.5: the target velocity is 3 − 0.5 = 2.5, no matter the time. The network just has to learn to output "data minus noise" at every noisy point it sees.

The loss does NOT go to zero — and that's correct. A given noisy point x can be explained by many (z, ε) pairs, each with a different target z − ε. The network can only output one arrow per x, so it learns the average of those targets — which is exactly the marginal field. The irreducible spread of conditional targets means the minimum loss is positive. If your training loss is approaching zero, something is wrong (overfitting or a bug). Expect it to plateau at a healthy nonzero value.

Schedules & the conditional velocity target

Top: the two schedules α_t (data dial, teal) and β_t (noise dial, orange) over time — toggle linear vs. square-root. Bottom: for fixed z and ε, the conditional velocity target α̇_t z + β̇_t ε over time (purple). For the linear schedule it's the constant z − ε; for the square-root schedule it blows up near t = 1 (because β̇_t diverges) — a real practical wrinkle.

schedule

When you sample the training point as x = α_t z + β_t ε, the conditional velocity target simplifies to what?

Just x itself. α̇_t z + β̇_t ε — a blend of the schedule derivatives with the data and the noise; for the linear schedule this is the constant z − ε. Always zero, because the loss must reach zero. The network weights θ.

Chapter 7: Straight Paths — Linear / Optimal-Transport Interpolation

The linear schedule deserves its own chapter because it is the cleanest possible path and the basis of rectified flow — the formulation behind Stable Diffusion 3. It also generalizes beautifully to interpolating between any source and target, not just a Gaussian and a data point.

The straight-line (CondOT) path. With α_t = t and β_t = 1 − t, the conditional sample is a linear interpolation of noise and data:

x_t = (1 − t) ε + t z

At t = 0 it's the noise ε; at t = 1 it's the data z; in between it slides along the straight segment connecting them. "CondOT" stands for conditional optimal transport: a straight line is the shortest path, the optimal-transport route between the two points.

This is just the reparameterization with the linear schedule, rewritten to emphasize the geometry: each particle travels in a perfectly straight line from its noise start to its data target, at constant speed.

The velocity is constant — the difference vector

Differentiate the straight segment x_t = (1 − t) ε + t z in time:

d⁄dt x_t = −ε + z = z − ε

The velocity does not depend on t — it is the constant difference between data and noise. So the conditional Flow Matching loss for the straight path is the simplest objective in all of generative modeling:

L_CFM(θ) = E_{t, z, ε} [ ‖ u^θ_t( (1 − t) ε + t z ) − (z − ε) ‖² ]

Reading it. Sample noise ε and data z, place x somewhere on the segment between them at a random time t, and teach the network to predict the constant velocity z − ε that carries x from ε to z. The network learns "which way is data, which way is noise" everywhere in space. Sampling then walks any new noise point along the learned straight-ish field to data.

Why straight paths are coveted — fewer sampling steps. Recall from Lecture 1 that Euler's error comes from the field curving during a step. If the true trajectories are straight lines, Euler is nearly exact even with big steps — you can sample in a handful of network calls instead of hundreds. That's the entire promise of rectified flow: straighten the paths, slash the inference cost. The learned marginal field isn't perfectly straight (it's an average of straight conditional fields), but it starts close, and "reflow" procedures iterate to straighten it further.

Generalizing: interpolate any source and target

Nothing forced the source to be a Gaussian. Replace ε ∼ p_init with a sample from any easy-to-sample source distribution, and z from the target. The straight-line path and its constant velocity z − (source) carry on unchanged:

x_t = (1 − t) x_source + t x_target, velocity = x_target − x_source

This is how Flow Matching becomes a general distribution-to-distribution transport learner — image-to-image translation, paired data, and more, not just noise-to-data. The recipe never changes: interpolate, regress on the difference vector.

Straight-line conditional trajectories (noise → data)

Blue dots (left) are noise samples ε; orange stars (right) are data points z, paired up. Drag t: each particle slides along the straight segment x_t = (1−t)ε + t z from its noise start to its data target, at constant velocity z − ε. The thin gray lines are the full straight paths. This is the CondOT / rectified-flow geometry — the reason such fields need few Euler steps.

time t0.00

Build the general transport yourself — not noise-to-data, but one arbitrary distribution to another, with the same two-line recipe: interpolate, regress on the difference vector.

For the straight-line (CondOT) path x_t = (1−t)ε + t z, what is the conditional velocity target, and why is this path attractive for fast sampling?

The target is x_t; it is fast because it ignores the data. The target is the constant z − ε (data minus noise); because conditional trajectories are straight lines, Euler is nearly exact with few large steps — cheap inference. The target is the noise ε alone; it is fast because no data is needed. The target grows with t; it is fast because the path is curved.

Chapter 8: Showcase — Train a Vector Field on a 2D Toy

Everything assembles here. We will train a tiny vector field by minimizing the Conditional Flow Matching loss on a 2D toy — noise to a ring of five Gaussian clusters, exactly the lab notebook's target — and watch the loss fall and samples land on the target. This is the full Flow Matching pipeline in pure numpy.

The training loop (Algorithm). Repeat for many steps:

sample a mini-batch of data points z from the dataset;
sample a random time t (uniform) and noise ε for each;
form noisy points x = α_t z + β_t ε (here the linear schedule, so x = (1−t)ε + t z);
predict velocities with the field and regress them onto the targets z − ε;
take a gradient step on the squared error.

Notice what's absent: no ODE simulation. We never roll out a trajectory during training — simulation-free, as promised.

The lab notebook uses a multi-layer perceptron with weights tuned by Adam over thousands of steps. To keep the in-browser lab fast and transparent, we train a compact featurized linear field: the network's velocity is a linear function of a fixed set of features of (x, t) — position, time, and a few radial-basis "bumps" in space. It's a real learned field (it has weights fit by gradient descent on the CFM loss), just small enough to train in a second.

Why a featurized linear model still teaches the real lesson. The marginal field is genuinely nonlinear in space, so a plain linear-in-x field couldn't capture it. By expanding x into nonlinear features (Gaussian bumps at several anchor points, plus time), a linear-in-features model becomes a flexible function approximator — the same trick as kernel methods. The training objective, the targets z − ε, the time/noise sampling, and the simulation-free property are all exactly what a deep network uses. Only the function class is smaller.

Below: first, the interactive showcase widget — watch a field train live and pull noise onto the ring. Then Code Lab 3, where you implement the loss and the training step yourself.

Live: train a field, watch noise flow to the ring

Press Train: a small featurized field is fit by gradient descent on the CFM loss (target ring = five clusters, orange). The loss curve falls and plateaus at a healthy nonzero value (as Chapter 6 warned). Then press Sample to Euler-integrate fresh Gaussian noise through the learned field — the blue cloud should land on the orange ring. Reset to start over.

In the training loop, what is the correct training pair — the noisy point x and the velocity target — for the linear schedule, and why does dropping the noise break it?

x = z and target = 0; dropping noise just speeds things up. x = t·z and target = z; it works fine without noise. x = (1−t)ε + t z (a noisy point on the noise→data segment) and target = z − ε (the velocity); without the noise the field is trained only on the data side and to point at z rather than transport noise, so sampling from real Gaussian noise misses the ring. x = ε and target = ε; the data is irrelevant.

Chapter 9: Connections & Cheat Sheet

You just derived — and implemented — the training algorithm behind Stable Diffusion 3 and Movie Gen, from a hand-drawn-arrows problem to a one-line regression. Here is the whole lecture on a page.

The spine. Choose a probability path from noise to data. Build it cheaply by conditioning on one data point z (the tractable top row of the matrix). Find the closed-form conditional vector field that transports that conditional path. The true generator is the marginal field — the posterior-weighted average of conditional fields — which is intractable. But the Flow Matching theorem says regressing the network onto the cheap conditional field has the same gradient as regressing onto the marginal field. So train on the conditional loss; get the marginal field for free. Simulation-free.

Cheat sheet

Probability path. p_t interpolates p₀ = p_init to p₁ = p_data; specifies marginals only, not particle motion.

Conditional path (Gaussian). p_t(·|z) = N(α_t z, β_t² I); sample x = α_t z + β_t ε. Schedules: α₀=0, α₁=1, β₀=1, β₁=0.

Conditional field. u_t(x|z) = (α̇_t − (β̇_t⁄β_t)α_t) z + (β̇_t⁄β_t) x. On the path it equals α̇_t z + β̇_t ε.

Marginal field (marginalization trick). u_t(x) = average of u_t(x|z) weighted by the posterior p_t(z|x); follows p_t, so X₁ ∼ p_data. Intractable to evaluate.

FM theorem. L_FM = L_CFM + C ⇒ same gradient. Train on the tractable conditional loss.

Linear / CondOT. α_t=t, β_t=1−t ⇒ x_t = (1−t)ε + t z, velocity z − ε (constant). Straight paths → few Euler steps (rectified flow / SD3).

Training (Algorithm 3). Sample z, t, ε; form x = α_t z + β_t ε; minimize ‖u^θ_t(x) − (α̇_t z + β̇_t ε)‖². No ODE simulated — simulation-free. Loss plateaus at a nonzero value.

Where this sits in the series

Lecture 1 — Flow & Diffusion Models. Built the sampler: ODEs, SDEs, vector fields, flows, Euler & Euler–Maruyama. We trained nothing — the arrows were given. Review it here.
Lecture 2 — Flow Matching (this one). Trained the vector field: conditional/marginal paths, the marginalization trick, the CFM loss, straight paths.
Lecture 3 — Score Matching & Guidance. The score (gradient of the log density) gives the SDE/diffusion training story — the same matrix with a score column — plus classifier-free guidance for prompts. The lab adds a conditional score loss right next to the flow loss you wrote here.
Lecture 4 — Architectures & Latent Spaces. U-Nets, Diffusion Transformers, and training in a VAE latent space (SD3, Movie Gen).
Lecture 5 — Discrete Diffusion. Carrying these ideas to text and language.

Related lessons on this site

Flow Matching (Gleam) — a complementary take on the same training target.
Diffusion Models (Gleam) — the DDPM view of the noising/denoising machine.
Diffusion (Architecture Atlas) — bird's-eye overview.
SDEs & Continuous Diffusion (Deep-Dive) — rigorous continuous-time treatment.

The Feynman test. Open the Lecture 2 slides. Can you now explain, from memory: the conditional vs. marginal split; why the conditional path is easy (it ends at one point); the closed-form conditional Gaussian field and what its two terms do; the marginalization trick and why the marginal field is a posterior-weighted average; why we regress on the conditional field even though we want the marginal one (same gradient); the simplified target z − ε for the linear schedule; and why the loss never reaches zero? And can you write the training loop in ten lines of numpy? If yes, you own Flow Matching.

One sentence: what is the single insight that makes Flow Matching trainable at scale?

That we can store the entire data distribution as a lookup table. That the marginal vector field has a simple closed form we evaluate directly. That regressing the network onto the tractable conditional field has the same gradient as regressing onto the intractable marginal field — so a one-line, simulation-free regression learns the true generator. That we must simulate the ODE at every training step.