Learned & Differentiable Filters — When the Fusion Rule Itself Is Learned

Chapter 1: Differentiable Filtering — The Filter Is Just a Computation Graph

Everything in this lesson hinges on one realization, and once you see it the rest is detail. Here it is: every operation inside a Kalman filter is differentiable. That sentence sounds dry. It is, in fact, the key that unlocks the whole field.

Let's unpack it. Recall the Kalman filter's machinery from Lesson 5. Each tick it does a handful of arithmetic operations — multiply a matrix, add a vector, invert a small matrix, multiply again. Specifically, the predict step is x̂_pred = F · x̂ (multiply the state by the dynamics matrix); the gain is computed from covariances; and the update is x̂ = x̂_pred + K · (z − H · x̂_pred) (correct the prediction by the gain times the innovation). Every one of those steps is multiplication, addition, and matrix inversion — and every one of those is a smooth, differentiable function of its inputs.

From "evaluate the filter" to "train the filter"

Think of the Kalman filter not as a fixed algorithm but as a computation graph: a diagram of boxes (operations) wired together by arrows (data flowing forward). Sensor measurements flow in at the left; the state estimate comes out at the right; in between sit the predict box, the gain box, the update box. This is exactly the shape of a neural network — a graph of differentiable operations — except that in a vanilla KF every box is fixed analytic math, with no free parameters to learn.

Now the move that makes everything possible: replace some of those fixed boxes with neural networks, whose weights are learnable. Maybe the box that produces R becomes a small net. Maybe the box that computes the gain becomes an RNN. The rest of the graph — the predict step, the update equation — stays as fixed, trusted math. Because the whole graph is still differentiable end to end, you can:

This is the entire recipe. The filter trains itself by gradient descent on a task loss, because the task loss can flow backward through every differentiable operation to reach the learnable parts. The classical filter was computed; the learned filter is trained. Same skeleton, but now the unknown organs grow from data.

A gradient intuition you can hold in your hand

Let's make the backprop concrete with the simplest possible case — a one-step scalar filter where the only learnable thing is the gain K. The update is:

where x̂_pred is the prediction, z is the measurement, and the term (z − x̂_pred) is the innovation — the surprise, how much the measurement disagreed with the prediction. Suppose we know the true state x_true, and our loss is the squared error:

To train K by gradient descent we need ∂L/∂K — how the loss changes if we nudge the gain. By the chain rule, that is ∂L/∂x̂ times ∂x̂/∂K. The first factor is 2(x̂ − x_true) (just differentiate the square). The second factor — how the estimate moves when we move K — is exactly the innovation (z − x̂_pred), because that is what K multiplies. So:

Now plug in numbers. Say the prediction was x̂_pred = 10, the measurement was z = 14 (innovation = +4), the current gain is K = 0.5, so x̂ = 10 + 0.5×4 = 12. And the truth was x_true = 13.5. The estimate (12) undershot the truth (13.5) — we did not trust the measurement enough. The error is x̂ − x_true = 12 − 13.5 = −1.5. The gradient is:

A negative gradient on K means: increasing K would decrease the loss. The math just told us to trust the measurement more — raise the gain — which is exactly right, because we undershot a measurement that was pointing the right way. A gradient step K ← K − η · ∂L/∂K with a small learning rate η = 0.02 gives K ← 0.5 − 0.02×(−12) = 0.5 + 0.24 = 0.74. The gain crept up from 0.5 toward more trust in the sensor — learned from a single example of the filter being wrong. Do this over thousands of examples and K settles at the value that minimizes tracking error on your data, with no Q or R ever specified.

The whole differentiable update in code

A differentiable filter is, at heart, the same arithmetic you already know — the only new thing is that we keep track of the gradient so we can train. Here is the scalar update plus its gradient, in plain numpy, no autograd library needed:

python
def update_and_grad(x_pred, z, K, x_true):
    # forward: the standard KF update is just two multiplies and an add
    innov = z - x_pred                 # the innovation (surprise)
    x_hat = x_pred + K * innov         # corrected estimate
    loss  = (x_hat - x_true)**2        # task loss: squared tracking error
    # backward: chain rule, dL/dK = dL/dx_hat * dx_hat/dK
    dL_dxhat = 2 * (x_hat - x_true)      # gradient of the square
    dxhat_dK = innov                   # x_hat moves by 'innov' per unit K
    dL_dK    = dL_dxhat * dxhat_dK     # the gradient that trains K
    return x_hat, loss, dL_dK

x_pred, z, x_true = 10.0, 14.0, 13.5
K, eta = 0.5, 0.02
for step in range(8):
    x_hat, loss, g = update_and_grad(x_pred, z, K, x_true)
    K = K - eta * g                    # gradient descent on the GAIN
    # K creeps from 0.50 -> 0.74 -> ... toward the value that nails x_true

Notice there is no Q, no R, no covariance propagation anywhere — the gain is learned directly from the tracking error. That single idea, scaled up so the gain comes from an RNN instead of one scalar, is KalmanNet (Chapter 4). But first, the gentlest hybrid of all: keep the whole classical filter and learn only the noise.

Chapter 2: Learned Q and R — Keep the Whole Filter, Learn Only the Noise

The simplest hybrid keeps everything you know about the Kalman filter exactly as it is, and learns only the two things you almost never know for sure: the noise covariances Q (how much the dynamics surprise you) and R (how noisy the sensor is). This is the on-ramp — the smallest possible step from classical to learned, and the place to build intuition before the bolder methods.

Recall why Q and R matter. They are the trust dials of the Kalman filter. R says "how much do I believe the sensor"; Q says "how much do I believe my own prediction." The gain — the famous Kalman gain K — is computed from them. Get them right and the filter is optimal. Get them wrong and the filter is still confident, just wrong: too small an R and it chases sensor noise; too large an R and it ignores good measurements and drifts. And as we said in Chapter 0, you usually do not know the true Q and R. So you tune. Learning them is just tuning, done automatically by gradient descent on data.

Why a wrong R wrecks the gain — worked, with numbers

Let's see, concretely, how a mis-specified R poisons the gain, and therefore the estimate. The steady-state scalar Kalman gain for a static quantity (measure the same value repeatedly with prediction variance P and measurement variance R) is:

Read it as the trust dial: if the measurement is very precise (R small), K → 1 (trust the sensor fully); if the measurement is garbage (R huge), K → 0 (ignore it, keep the prediction). Now suppose the true sensor variance is R_true = 4, and our prediction variance is P = 4. The correct gain is:

Half-and-half: a balanced, sensible blend. Now suppose we mis-specified R — we assumed the sensor was far better than it is, R_assumed = 0.25 (sixteen times too small):

The wrong R inflates the gain from 0.5 to 0.94 — the filter now trusts the sensor 94%, even though half of what the sensor reports is noise. The estimate will jitter wildly, chasing every noisy measurement, because we told it the sensor was nearly perfect. Flip it the other way — assume R_assumed = 100 (the sensor is far worse than reality) — and K = 4/104 = 0.038: the filter now ignores a perfectly good sensor and drifts on its prediction. A wrong R, in either direction, breaks the filter while it reports full confidence. This is exactly the failure a learned R fixes: instead of guessing R, find the R that makes the tracking error smallest on real data.

The data flow: what is learned, what stays classical

Be precise about the boundary, because this precision is what separates a hybrid filter from a black box. In learned-Q/R:

So 95% of the filter is the textbook Kalman filter, untouched. Only two matrices are free parameters. You run the (fully differentiable) filter forward on logged trajectories, compute the tracking loss against ground truth, and backpropagate the loss to Q and R. Because Q and R must stay positive-definite (a covariance can't be negative), in practice you learn their logarithms or a factorization — a small technical wrinkle that keeps them valid. The conceptual story is simply: two numbers that used to be hand-tuned are now fit by gradient descent.

Learned-R in code

Learning R for a static-measurement filter is a tiny gradient loop. We measure how badly the filter tracks at the current R, estimate the gradient by finite difference, and step R downhill:

python
import numpy as np

def filter_loss(R, z, x_true, P=4.0):
    # run the steady-state KF with this R, return mean squared tracking error
    K = P / (P + R)                       # classical gain — trust dial set by R
    x = 0.0; err = 0.0
    for zi in z:
        x = x + K * (zi - x)              # standard update, gain from R
        err += (x - x_true)**2
    return err / len(z)

np.random.seed(0)
x_true = 5.0; R_true = 4.0
z = x_true + np.sqrt(R_true) * np.random.randn(400)   # noisy measurements

R = 20.0                                 # a bad initial guess (5x too big)
for step in range(60):
    eps = 1e-2
    g = (filter_loss(R+eps, z, x_true) - filter_loss(R-eps, z, x_true)) / (2*eps)
    R = max(0.05, R - 3.0 * g)         # gradient descent toward the best R
# R converges near 4.0 -> the learned R recovers the true sensor noise

The loop never knows R_true; it only knows the measurements and the truth, and it lets the tracking error pull R toward the value that explains the data. The result lands near 4 — the filter discovered its own sensor noise. That is learned Q/R: the classical filter, fully intact, with its two stubborn dials fit automatically. The next methods keep more of the magic but learn bolder pieces.

Chapter 3: BackpropKF — Let the Filter Teach a Perception Net

Learned Q/R kept the sensor model and learned the noise. BackpropKF — Haarnoja and colleagues, NeurIPS 2016 — flips the emphasis: keep a standard, differentiable Kalman filter, but put a learned perception front-end in front of it, and train the whole stack end-to-end. It answers a problem the classical KF never could: what do you do when your "sensor" is a camera and there is no equation mapping pixels to a measurement?

Picture tracking a red ball bouncing around a video. The state is the ball's position and velocity — that part the KF handles fine, with simple dynamics. But the measurement is a 200×200 image. There is no clean H matrix that turns a grid of pixels into "the ball is at (x, y)." That mapping — pixels to position — is exactly what a convolutional neural network (CNN) is good at and what no human can write as an equation. So BackpropKF puts a CNN where the measurement model used to be.

The crucial twist: the CNN outputs the measurement AND its uncertainty

Here is the elegant part. A naive approach would have the CNN spit out a position (x, y) and hand it to the filter as a measurement. But the filter needs more than a number — it needs to know how much to trust that number, i.e. the measurement covariance R. And the right R is not constant: when the ball is in clear view, the CNN's position estimate is sharp (small R, trust it); when the ball is occluded behind an obstacle or motion-blurred, the CNN is guessing (large R, don't trust it). The uncertainty is image-dependent.

So BackpropKF's CNN outputs both: a measurement z (where it thinks the ball is) and an uncertainty R (how sure it is), per frame. The data flow:

The filter is untouched classical math — predict, gain, update, exactly as in Lesson 5. The only learned box is the CNN front-end. But because the whole graph is differentiable, the tracking loss at the very end (estimated trajectory vs. true trajectory) flows all the way back through the Kalman filter and into the CNN's weights.

How a bad vs. good learned R changes the track — worked

Let's make "the filter teaches the net" quantitative. Suppose on a blurry, occluded frame the CNN's position guess is genuinely off: it reports z = 8.0 when the truth is 5.0, an error of 3. The prediction (from the ball's motion) is a good x̂_pred = 5.2, very close to truth. The gain is K = P/(P+R) with P = 1. Compare two values of the learned R:

On this bad frame, a CNN that knew it was unsure (reported R = 25) let the filter nearly ignore the bad measurement and coast on its good prediction — error 0.31. A CNN that was over-confident (R = 0.1) forced the filter to swallow the bad measurement whole — error 2.75. During training, the second case produces a big loss, and backprop pushes the CNN to raise its R on frames like that. After training, the CNN has learned to output R = 25 on blurry frames and R = 0.1 on clear ones — calibrated uncertainty, taught entirely by the downstream filter. That is the payoff of end-to-end training through the filter.

BackpropKF's lasting lesson is architectural: a learned perception module and a classical filter are not rivals — they compose. The CNN does what CNNs do (turn pixels into a noisy estimate with an honest error bar); the KF does what KFs do (fuse a sequence of noisy estimates over time using their error bars). Training them jointly, end to end through the differentiable filter, makes the perception net produce exactly the kind of measurement the filter can use best. Next, we keep even less classical and learn the very heart of the filter — the gain.

Chapter 4: KalmanNet — Replace the Gain Computation With an RNN

This is the centerpiece of the lesson, and the cleanest expression of "the structure is the regularizer." KalmanNet — Revach and colleagues, IEEE Transactions on Signal Processing, 2022 (arXiv:2107.10043) — keeps the Kalman filter's predict and update structure exactly, but rips out the one part that needs the things you don't know — Q, R, and the covariance propagation — and replaces it with a small RNN that learns the Kalman gain directly from data.

To see why this is the perfect surgical cut, look again at what a Kalman filter actually does each tick. It has three jobs: (1) predict the next state from the dynamics; (2) compute the gain K, which requires propagating the covariance forward through the Riccati equations using Q and R; (3) update the prediction by K times the innovation. Of these, the predict and update are simple and rely on the parts of the model you usually do know (roughly how the state evolves, roughly how the sensor reads it). The painful part — the part that needs Q, R, and the entire covariance machinery you can't specify — is computing the gain. So KalmanNet learns exactly that one painful part, and leaves everything else as trusted classical math.

The data flow — walk it carefully

Here is precisely what is classical and what is learned, tick by tick. This is the most important diagram in the lesson:

Read steps 2–4 and you are reading the standard Kalman update verbatim — the innovation, the gain, the correction. The only thing that changed is where the gain came from: a learned RNN instead of the Riccati covariance recursion. The gain is exactly the trust-dial from Lesson 5; here it is learned instead of computed. Everything that made the Kalman filter interpretable — you can still point at the prediction, the innovation, the gain, the corrected estimate — survives. You have not built a black box; you have built a Kalman filter whose one unknown organ is grown from data.

Why feed the RNN innovation features, and why an RNN?

Two design choices deserve a word. First, why the innovation? Because the innovation — how wrong the prediction was — is precisely the signal that tells you whether to trust the model or the measurement. A run of large innovations means "my model is off, lean on the sensor" (raise the gain); a run of small innovations means "my model is nailing it, the sensor's noise is just jitter" (lower the gain). KalmanNet feeds the RNN features derived from the innovation and from the recent state/observation differences — the same quantities the classical gain implicitly depends on, handed to the network so it can learn the mapping the Riccati equation used to compute.

Second, why an RNN and not a plain feed-forward net? Because the optimal gain depends on history. In a real Kalman filter, the covariance — and hence the gain — evolves over time, accumulating information from every past measurement. The RNN's hidden state is the learned stand-in for that covariance: a memory that summarizes the past well enough to set the right gain now. KalmanNet does not track an explicit covariance matrix — the RNN's recurrent memory implicitly encodes it. That is the whole trick that lets it drop Q, R, and the Riccati machinery.

The payoff — tracking under model mismatch, where the textbook KF fails

Now the reason anyone cares. The classical Kalman filter is optimal only when the model is correct. Feed it a mismatched model — wrong dynamics, wrong noise — and it is no longer optimal; it can lag, drift, or diverge while reporting full confidence. KalmanNet, having learned the gain from real trajectories rather than from a possibly-wrong model, adapts to the true behavior of the system. Across the experiments in the paper, KalmanNet tracks accurately under model mismatch where the textbook KF degrades — and it does so while remaining far more data-efficient and interpretable than a black-box RNN, because most of the filter is still the trusted classical skeleton.

Let's quantify the mismatch story with a single gain. Suppose the true process noise is large (the system is more jittery than your model assumed): Q_true = 10, while you assumed Q_assumed = 1, with R = 1 and prediction-driven P. The classical gain uses your wrong Q. Roughly, the steady-state gain scales with how much process uncertainty you believe: with the assumed Q = 1 the filter thinks its prediction is trustworthy and sets a low gain (say K ≈ 0.27), so it under-trusts measurements and lags behind the truly-jittery state. The correct gain for the true Q = 10 is much higher (K ≈ 0.73) — trust the measurements more, because the prediction is actually unreliable. The classical KF, locked to the wrong Q, uses 0.27 and lags. KalmanNet, having learned its gain from data where the system really did jitter, outputs the higher gain that the truth demands — it never knew Q at all, it just learned to react to the large innovations the jitter produced. That is the advantage, in one number: the learned gain finds the value the wrong model could not.

KalmanNet's loop in code — a pluggable gain

The cleanest way to feel KalmanNet is to write a Kalman filter whose gain comes from a pluggable function, then swap in either the analytic gain or a tiny learned/adaptive one. The filter structure is identical; only the gain source changes. Here is a mismatched-noise tracking task where the adaptive gain wins:

python
import numpy as np
np.random.seed(0)

# --- a 1-D tracking task with MODEL MISMATCH ---
# The system is actually very jittery (true process noise large),
# but our model ASSUMES it is smooth (assumed Q tiny). Classic mismatch.
N = 200
true_Q, assumed_Q, R = 10.0, 0.2, 1.0
x = 0.0; truth = []; meas = []
for k in range(N):
    x += np.sqrt(true_Q) * np.random.randn()        # real (jittery) dynamics
    truth.append(x)
    meas.append(x + np.sqrt(R) * np.random.randn())  # noisy measurement
truth = np.array(truth); meas = np.array(meas)

def analytic_gain(P):
    # classical gain from the ASSUMED (wrong) Q -> stays too low -> lags
    P_pred = P + assumed_Q
    K = P_pred / (P_pred + R)
    return K, (1 - K) * P_pred

def learned_gain(P, innov_hist):
    # stand-in for KalmanNet's RNN: raise the gain when innovations
    # are persistently large (the data says "your prediction is unreliable")
    recent = np.mean(np.abs(innov_hist[-10:])) if innov_hist else 1.0
    K = recent / (recent + np.sqrt(R))     # big innovations -> trust the sensor more
    return np.clip(K, 0.05, 0.97), P  # no Q/R/Riccati needed

def run(gain_fn, learned=False):
    x_hat, P, innov_hist, est = 0.0, 1.0, [], []
    for z in meas:
        x_pred = x_hat                      # predict (random-walk dynamics)
        innov  = z - x_pred                 # innovation = surprise
        K, P = (gain_fn(P, innov_hist) if learned else gain_fn(P))
        x_hat = x_pred + K * innov          # SAME classical update, learned K
        innov_hist.append(innov); est.append(x_hat)
    return np.array(est)

est_kf = run(analytic_gain)
est_kn = run(learned_gain, learned=True)
print("classical KF (wrong Q) RMSE :", np.sqrt(np.mean((est_kf-truth)**2)))
print("learned-gain      RMSE :", np.sqrt(np.mean((est_kn-truth)**2)))
# learned-gain RMSE is markedly lower: it raised the gain to match the real jitter

The two runs share the identical predict and update lines — x_hat = x_pred + K * innov — the only difference is the gain source. The classical gain, computed from the wrong (tiny) assumed Q, stays stubbornly low and the estimate lags the jittery truth. The learned gain watches the innovations, sees they are persistently large, and raises K to trust the measurements — recovering the truth without ever knowing Q or R. That is KalmanNet in miniature: same skeleton, learned gain, wins under mismatch.

Here is the compact version of the whole idea — a Kalman update with a pluggable gain, the heart of the method in five lines:

python
def kf_step(x_pred, z, gain):
    innov = z - x_pred                # classical: the surprise
    K     = gain(innov)               # PLUGGABLE: analytic OR learned RNN
    return x_pred + K * innov          # classical: the update equation
# gain = lambda i: 0.27           -> textbook KF (fixed, from assumed Q/R)
# gain = rnn.predict_gain         -> KalmanNet (learned from data)

Chapter 5: Differentiable Particle Filters — Making Resampling Learnable

So far we have learned pieces of the Kalman filter, which assumes a single Gaussian belief. But Lesson 10 taught us the particle filter — the tool for when the belief is multi-modal or the dynamics are wildly nonlinear, where you represent the belief as a cloud of weighted samples (particles). Can we learn the unknown pieces of a particle filter too — its motion model, its measurement model — end to end? The answer is yes, but there is one stubborn obstacle, and overcoming it is the whole content of this chapter.

Recall the particle filter's loop: propagate each particle through the motion model; weight each particle by how well it explains the measurement; then resample — draw a fresh set of particles, keeping the heavy-weighted ones and discarding the light ones, so the cloud concentrates where the belief is strong. That resampling step is what keeps the filter from degenerating into one particle hoarding all the weight. It is essential.

The obstacle: resampling is not differentiable

Here is the problem. Resampling is a hard sampling step — a discrete, all-or-nothing decision: this particle survives (copied), that one dies (deleted). And a discrete decision has no useful gradient. Think about it: if you nudge a particle's weight up by a hair, the resampling output usually doesn't change at all (the same particles survive) — gradient zero — until the nudge crosses a threshold and a particle suddenly flips from "dead" to "alive" — gradient infinite/undefined. A function that is flat everywhere except for jumps cannot pass a smooth gradient backward. So the backpropagation chain breaks at the resampling step. You cannot train the motion and measurement models through it, because the gradient can't get past it.

The fix: soft / differentiable resampling

Jonschkowski and colleagues (RSS 2018), and Karkus and colleagues around the same time, found the fix: replace the hard resample with a soft, differentiable one that lets gradients flow. The intuition is to relax the all-or-nothing decision into a smooth blend. Instead of "this particle is fully copied or fully deleted," a soft resample produces new particle weights as a smooth, differentiable function of the old ones — nudging an input weight now produces a small, smooth change in the output, so the gradient passes through. (One common trick mixes the resampled distribution with a small amount of the uniform distribution, which makes the operation smooth and keeps the gradient alive; others use a soft, temperature-controlled weighting.)

With the resampling made differentiable, the entire particle filter becomes one differentiable computation graph — propagate, weight, soft-resample, repeat — and you can train its learnable components end to end:

So the differentiable particle filter keeps the particle filter's structure — the cloud of samples, the propagate/weight/resample loop — as the inductive bias, and learns the motion and measurement models from data, with the soft resample as the bridge that lets the loss reach them. It is the same philosophy as KalmanNet (keep the Bayesian skeleton, learn the unknown parts) carried into the world of multi-modal, nonlinear beliefs.

Chapter 6: Showcase — The Textbook KF vs the Learned Filter Under Mismatch

This is the payoff — the simulation that makes the whole lesson click. We put the textbook Kalman filter and a learned-gain filter side by side on the same tracking task, and then we break the model. The textbook KF was tuned for a system that moves a certain way; the learned filter learned its gain from how the system actually moves. Drag the mismatch slider and watch their fortunes diverge: at zero mismatch they tie, but as the true dynamics drift away from the assumed model, the textbook KF lags and drifts — over- or under-confident — while the learned filter adapts and stays glued to the truth.

The object follows a true trajectory driven by some real process noise. The textbook KF assumes a fixed noise level (its tuning). When you raise the mismatch, the true process noise grows beyond what the KF assumed, so the KF's gain — locked to its wrong assumption — is too low: it under-trusts the measurements and falls behind the increasingly jittery truth. The learned filter, whose gain reacts to the size of the innovations it actually sees, raises its gain to match the real jitter and keeps tracking. Press Run to animate, drag mismatch to widen the gap, and watch the error bars below.

Play with it. At 0% mismatch the two filters are nearly indistinguishable — when the model is correct, the classical Kalman filter is already optimal, and the learned gain simply recovers the same value. There is no magic to extract when the model is right; this is the honest part of the story. But push the mismatch up and the divergence is stark: the textbook KF's estimate (blue) lags behind the jittery truth, its error bar swelling, while the learned filter (teal) raises its gain and clings to the truth, its error bar staying low. The learned filter's advantage is exactly proportional to how wrong the model is.

This single picture answers "why bother with learned filters?" If you had a perfect model you wouldn't — the Kalman filter is unbeatable there. You bother because perfect models are rare. The learned filter is insurance against the mismatch that real terrain, real sensors, and real weather inevitably introduce — and the showcase shows the insurance paying off precisely when you need it. (No quiz here — the simulation is the test. Convince yourself by breaking it.)

Chapter 7: Use Cases & Real Products — Where Learned Filters Earn Their Keep

Learned and differentiable filters are not a parlor trick — they are answers to recurring, painful problems in deployed systems. The common thread: anywhere the classical filter's model is unknown, mismatched, or just exhausting to tune, and where you have logged trajectories to learn from. Let's walk the landscape.

Tracking and navigation under unknown noise or partial models

The headline use case, and the one the papers benchmark. A radar or camera tracker following maneuvering targets faces dynamics no fixed model captures — a vehicle accelerates, turns, stops, all with noise statistics that shift with conditions. KalmanNet-style learned filters shine here: they learn the gain from real target trajectories, so they keep tracking through maneuvers where a KF tuned for constant-velocity motion would lag. The same applies to navigation when the sensor noise is unknown or time-varying — a cheap IMU whose bias drifts with temperature, a GNSS receiver whose noise spikes in urban canyons. Rather than hand-modeling each regime, the learned filter absorbs it from data.

Robotics state estimation with learned dynamics

This is where differentiable filters are most natural. A legged robot's contact dynamics, a soft robot's deformation, a manipulator interacting with unknown objects — these have dynamics that are genuinely hard to write down analytically. Differentiable filters let you learn the motion model (the f in x̂_pred = f(x̂)) directly from logged robot trajectories, while keeping the Bayesian filtering structure that fuses it with sensors over time. Differentiable particle filters in particular have been used for robot localization — learning the measurement model that scores a particle against a camera image, end-to-end, in environments where a hand-built sensor model would be brittle.

Sensor fusion where Q and R are hard to tune

Every practicing engineer who has tuned a Kalman filter knows the pain: days spent nudging Q and R, getting it right for one scenario only to have it fail in another. Learned-Q/R (Chapter 2) directly automates this. For multi-sensor fusion stacks — IMU + wheel odometry + GPS + LiDAR, each with its own noise that varies by surface, speed, and weather — learning the covariances (even making them input-dependent, as BackpropKF does) replaces weeks of manual tuning with a training run on logged drives. The interpretability stays: you can still inspect the learned R and sanity-check it against physical expectations.

The low-data regime

A subtle but important product niche: where you have too little data for a black-box approach but too wrong a model for a pure classical one. A new robot platform, a new deployment site, a rare failure mode — you have hundreds of trajectories, not millions. This is exactly the hybrid's home turf (Chapter 0). The filter structure supplies the prior, so a few hundred trajectories suffice to learn the missing pieces. Black-box deep estimators simply can't be trained on that little data; learned filters can.

The unifying business case across all of these: you ship a system that works in conditions you never explicitly modeled, trained on data you already log. That is why learned filters are moving from papers into perception and navigation stacks — they convert the unglamorous, never-ending labor of model-building and tuning into a training run.

Chapter 8: Practical Application — When to Reach for a Learned Filter, and How

Knowing the methods is half the battle; knowing when to use one and how to train it is the other half. This chapter is the decision guide and the recipe.

When to reach for a learned filter — and when not to

Be honest about this, because over-reaching is the most common mistake. Learned filters are insurance against model mismatch and tuning pain — they are not free improvements over a good classical filter. The decision:

The golden rule: if a tuned classical filter already works, do not replace it. The Kalman filter is optimal under its assumptions and comes with decades of theory and guarantees. Learned filters earn their place only where those assumptions break — and they buy flexibility at the cost of the guarantees, the need for data, and the risk of failing outside the training distribution (Chapter 9).

What to learn — the ladder of commitment

You don't have to go all-in. The methods form a ladder from "barely learned" to "mostly learned," and you should climb only as high as your problem demands:

Climb the minimum distance. Each rung learns more and so needs more data and gives up more interpretability/guarantees. If your model is right and only the noise is unknown, learning Q/R is enough — don't jump to KalmanNet and pay for flexibility you don't need.

How to train — two loss philosophies

There are two ways to define the training loss, and the choice matters:

In practice both require ground-truth trajectories (from a motion-capture rig, a high-grade reference sensor, or simulation). The end-to-end view is what lets BackpropKF train a perception net through the filter without ever labeling per-frame uncertainty — the task loss supplies the signal. The data requirement is the real cost: you need enough representative trajectories that the learned component sees the conditions it will face at deployment. Hundreds to thousands of trajectories is typical for these hybrid methods — far less than a black-box estimator, far more than zero.

Keeping interpretability — a practical gift

One concrete practical advantage worth exploiting: because the skeleton stays classical, you can still inspect the learned filter. In KalmanNet you can pull out the learned gain K_t at every step and plot it — does it rise when the system maneuvers and fall when it's quiet, as a sensible gain should? In learned-Q/R you can read the learned R and check it against physical expectations of your sensor. This interpretability is not just intellectual comfort; it is how you debug and certify a learned filter, which is the subject of the next chapter. A black-box RNN gives you none of this — another reason the structured hybrid is the practical sweet spot.

Chapter 9: Debugging & Failure Modes — How a Learned Filter Betrays You

A learned filter inherits a new family of failure modes that classical filters never had — the failure modes of learning. The cruel part is that several of them are silent: the filter keeps producing confident numbers while quietly being wrong, because the learned component left the world it was trained in and nobody told the estimate. This chapter catalogs the betrayals and, for each, how to detect it.

Failure 1: Overfitting to the training distribution (the big one)

This is the signature failure, and it is the exact mirror image of the classical filter's strength. A learned filter trained on certain dynamics can overfit — it tracks beautifully on data that resembles its training set and fails on out-of-distribution (OOD) dynamics it never saw. The painful irony: the classical filter's hand-written model would have generalized. A Kalman filter built on physics works in conditions the engineer never tested, because the physics is true everywhere. A learned gain only knows the conditions in its training data. Push the robot onto terrain it never trained on, and the learned filter can be worse than the humble classical one.

How to detect it: hold out OOD test trajectories (different speeds, terrains, noise levels than training) and measure tracking error on them specifically. A large train-vs-OOD gap is the fingerprint of overfitting. Never report only in-distribution accuracy — it hides this failure entirely.

Failure 2: Loss of consistency / uncertainty miscalibration

The classical Kalman filter gives you a covariance you can trust — it tells you not just the estimate but how sure to be, and that uncertainty is consistent (it matches the actual error). A learned filter often breaks this. KalmanNet, for instance, learns a gain directly — it does not necessarily produce a meaningful covariance at all. So you may get a good estimate with no honest error bar, or a reported uncertainty that doesn't match reality (over-confident or under-confident). For a downstream system that reasons about risk (a planner deciding whether it's safe to proceed), a miscalibrated uncertainty is dangerous — it trusts a number that lies about its own reliability.

How to detect it: this is exactly what consistency checks are for — the NEES (Normalized Estimation Error Squared) and NIS (Normalized Innovation Squared) tests, the subject of the next lesson (sf-21). They statistically test whether the filter's reported uncertainty matches its actual errors. A learned filter that passes tracking accuracy but fails NEES/NIS is telling you its confidence is a fiction. Always run these on a learned filter before trusting its uncertainty.

Failure 3: Training instability through the filter

Backpropagating through many time steps of a recurrent filter is backpropagation through a long chain, and long chains have the classic pathologies: exploding or vanishing gradients. The gradient of the loss with respect to an early-step parameter is a product of many per-step factors; if those factors are mostly >1 the gradient explodes (training diverges, weights blow up to NaN), if mostly <1 it vanishes (early steps never learn). The filter's own feedback can amplify this. Symptoms: loss that spikes to infinity or NaN, or a model that simply won't improve no matter how long you train.

How to detect / fix: watch the gradient norms during training — sudden spikes mean exploding gradients (apply gradient clipping), flatlined early-layer gradients mean vanishing (the recurrent architecture or truncated backprop-through-time helps). A loss curve that won't descend or periodically explodes is the tell.

Failure 4: Silent failure when the learned component leaves its regime

This is the most dangerous because it combines the others into a quiet catastrophe. The learned gain or model was trained on a region of state/observation space; in deployment the system wanders outside that region (a sensor degrades in a new way, the dynamics enter an unseen regime). The learned component now extrapolates — and neural networks extrapolate unpredictably, often confidently and wrongly. A classical filter degrades gracefully and predictably outside its design point; a learned one can fail abruptly and silently, still emitting plausible-looking estimates.

How to detect it: monitor the inputs to the learned component at runtime — are the innovation features within the range seen during training? An out-of-range input is an early warning that the learned gain is now guessing. Combine with the NIS check (Failure 2): a sudden run of statistically large innovations means the filter's predictions stopped matching reality — the canary for "the learned component has left its regime." Some deployments keep a classical filter running in parallel as a sanity reference and alarm when the two diverge.

Chapter 10: Connections, References & Cheat-Sheet

Learned and differentiable filters are the bridge between the two halves of this series — the classical Bayesian filters of the first half and the modern, data-driven frontier of the last. The idea that ties it together: fusion where the combination rule is learned but constrained by Bayesian structure. The structure is the regularizer; the learning fills the gaps the structure leaves. Keep both halves of that sentence and you have the whole field.

Where this sits in the series

Everything here is built directly on the classical filters you already met. Revisit them with new eyes — now you can see which part each learned method replaces:

• Lesson 5 — The Kalman Filter: the skeleton every method in this lesson keeps. The Kalman gain you learned there is exactly what KalmanNet learns; the Q and R you tuned there are what learned-Q/R fits. Re-read it and notice the trust dial — that is the heart of all of this.
• Lesson 7 — The Extended Kalman Filter: the EKF linearizes a nonlinear model you must still provide. Learned filters offer an alternative when you can't provide it — learn the behavior instead of linearizing a guess.
• Lesson 10 — The Particle Filter: the structure that differentiable particle filters keep. The resampling step you learned there is exactly the non-differentiable obstacle Chapter 5 had to soften.
• Estimation & Learning, Merged: the broader capstone on how estimation theory and machine learning fuse — the wider context for this lesson's hybrid philosophy.
• Robust Estimation: the classical answer to bad data (outlier rejection, heavy-tailed noise) — a useful contrast to the learned answer of letting a net infer per-input uncertainty (BackpropKF).

What's next — can we trust a learned filter? (Lesson 21)

This lesson gave you the power to learn a filter; the next gives you the tools to know whether you can trust it. Lesson 21 — Consistency & Debugging is the capstone on filter trust: the NEES and NIS consistency tests that statistically check whether any filter's reported uncertainty matches its true errors. We flagged these repeatedly in Chapter 9 as the way to catch a learned filter's miscalibration and its silent departures from the training regime. A learned filter that tracks well but fails NEES/NIS is lying about its own confidence — and Lesson 21 teaches you to catch that lie. Learn the filter here; learn to trust it next.

Cheat-sheet — the four methods at a glance

Cheat-sheet — the load-bearing ideas

• The spectrum: model-based (KF: needs model, data-efficient, interpretable) ↔ hybrid (learn the unknown parts, keep structure) ↔ data-driven (RNN: flexible, data-hungry, opaque). Hybrid wins in the partial-model / limited-data regime.
• The enabling idea: every KF operation is differentiable, so the filter is a differentiable computation graph — a task loss backpropagates through predict/gain/update to train neural components.
• KalmanNet's gain is exactly the trust dial from Lesson 5 — learned, not computed. The RNN's hidden state stands in for the covariance.
• BackpropKF: the filter teaches the perception net to be honestly calibrated — over-confident frames spike the tracking loss, and backprop raises the net's uncertainty there.
• Soft resampling turns the particle filter's non-differentiable staircase into a ramp, so gradients flow and the models train end-to-end.
• The risk: learned filters overfit the training distribution and fail (often silently) out of it — the exact place a hand-written physical model would have generalized. Always run OOD tests and NEES/NIS consistency checks (Lesson 21).

References

1. Revach, G., Shlezinger, N., Ni, X., Escoriza, A. L., van Sloun, R. J. G., and Eldar, Y. C. "KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics." IEEE Transactions on Signal Processing, vol. 70, pp. 1532–1547, 2022. The centerpiece — replaces the Kalman-gain computation with an RNN, tracking under model mismatch without knowing Q/R. arXiv:2107.10043
2. Haarnoja, T., Ajay, A., Levine, S., and Abbeel, P. "Backprop KF: Learning Discriminative Deterministic State Estimators." Advances in Neural Information Processing Systems (NeurIPS), 2016. A differentiable Kalman filter with a learned CNN front-end that outputs measurement and uncertainty, trained end-to-end. arXiv:1605.07148
3. Jonschkowski, R., Rastogi, D., and Brock, O. "Differentiable Particle Filters: End-to-End Learning with Algorithmic Priors." Robotics: Science and Systems (RSS), 2018. Soft/differentiable resampling so a particle filter with learned models trains end-to-end. arXiv:1805.11122
4. Shlezinger, N., Whang, J., Eldar, Y. C., and Dimakis, A. G. "Model-Based Deep Learning." Proceedings of the IEEE, vol. 111, no. 5, pp. 465–499, 2023. The broad survey placing learned filters within deep unfolding, learned optimizers, and structured hybrid methods. arXiv:2012.08405
5. Karkus, P., Hsu, D., and Lee, W. S. "Particle Filter Networks with Application to Visual Localization." Conference on Robot Learning (CoRL), 2018. A companion differentiable particle filter for end-to-end visual localization. arXiv:1805.08975