One map for the whole journey — every method, how they connect, and where they came from.
Twenty-one lessons is a lot of territory. You started by asking why two flawed sensors beat one good one, and you ended staring at transformers that fuse camera, LiDAR, and radar into a bird's-eye-view of a city street. Somewhere in between you derived the Kalman gain by hand, watched particles swarm a non-Gaussian belief, and learned why an information filter can be torn apart across a robot swarm and stitched back together with nothing but addition. This page is the single sheet of paper that holds all of it — the map you pin above your desk so the whole field fits in one glance.
It is, deliberately, lighter on long derivations than the lessons it points to. The value here is not new math; it is structure. Three things, in three views:
If you remember a single sentence from twenty-two lessons, make it this one:
And underneath the throughline is the discovery from Lesson 1 that reframes everything: different sensors fail in different places. A camera goes blind in the dark exactly where LiDAR shines; GPS dies in the tunnel exactly where the IMU keeps counting. Fusion is worth the trouble because the failure conditions don't overlap. Every architecture in this map is, at bottom, an arrangement for letting healthy information vote down the parts that have gone dark.
The twenty-four concepts in the graph fall into four families, and they are not arbitrary buckets — they are four stages of generality. Each family answers a harder version of the same question than the one before it.
Family 1 — Combine measurements. The seed. You have two numbers (or two estimates) of the same quantity, each with a known uncertainty, taken at the same instant. How do you merge them? Answer: weight each by its precision — inverse-variance weighting — and the fused estimate is sharper than either input. This is Lessons 1–3: why fuse, where to fuse (the architecture / JDL question), and the two-measurement blend that every later method secretly contains. Master this and the Kalman gain is just this idea wearing a coat.
Family 2 — Recursive Bayesian filters. Now the world moves and measurements arrive over time. You can't re-fuse everything at every step. The Bayes filter is the recursive engine: predict the state forward with a motion model, then update it with each new measurement — carrying a belief, not a history. Everything else in this family is a way to make that engine tractable: the Kalman filter (linear-Gaussian, optimal, closed-form), the EKF and UKF (when the world is non-linear), the information filter (the same belief flipped inside-out into an additive form), the particle filter (when the belief isn't Gaussian at all), and covariance intersection (when you don't know how your estimates are correlated). This is Lessons 4–11 — the classical heart of the field.
Family 3 — Multi-sensor systems. A real robot doesn't run one filter on one clean signal. It must decide which measurement came from which object (data association — gating, GNN, JPDA, MHT), it must turn raw inertial readings into motion (IMU / inertial navigation, with its inexorable drift), it must bound that drift against an absolute reference (INS/GNSS coupling via the error-state KF), and before any of it works the sensors must be calibrated and time-synchronized — the unglamorous prerequisite that silently breaks everything when it's wrong. This is Lessons 12–15: the engineering that turns a filter into a system.
Family 4 — Perception fusion & the learned frontier. The modern era fuses rich, high-rate perception — whole images, whole point clouds — with inertial data. Visual-inertial odometry (MSCKF, VINS, IMU preintegration) and LiDAR-inertial / factor-graph systems (LIO-SAM, FAST-LIO2, iSAM2) reframe fusion as optimizing a graph of constraints rather than running a single recursive filter. Then perception goes end-to-end: multimodal driving fusion (camera + LiDAR + radar, early/mid/late) and transformer / BEV fusion (BEVFusion, TransFusion, BEVFormer) learn the fusion rule from data, and learned / differentiable filters (KalmanNet, BackpropKF) fold the classical recursion inside a neural network. This is Lessons 16–20.
Running alongside all four families is a fifth thread that isn't a family so much as a discipline: debugging and consistency (Lesson 21 — NEES/NIS, innovation whiteness, robust costs). It connects to every filter because every filter can lie to you about how sure it is, and a fusion estimate that is overconfident is more dangerous than one that admits it's lost. In the graph it's drawn as a diagnostic spine touching the methods it audits.
Here is the whole lineage compressed — read top to bottom and you are reading the order the field was built in, and roughly the order the lessons teach it:
When you face a real fusion problem, the map collapses into three questions — the same three from Lesson 1, now powerful enough to pick a method:
Those three questions are the legend for everything below. The graph in Tab 1 is just these answers, drawn.
"What I cannot create, I do not understand — and what I cannot understand, I cannot teach."
This is the centerpiece. Every method in the series is a node, arranged in layers from foundations at the top to the learned frontier at the bottom. Solid lines are prerequisites ("learn this first"); dashed lines mean one idea evolved into or generalizes another. The teal diagnostic spine on the right audits the filters it touches.
Hover a node to light up its connections. Click a node to open its detail card below — its essence, a link to its lesson, and its key reference. On a phone, tap a node in the list that appears beneath the map.
Click any concept in the graph above to read its one-line essence, jump straight to its lesson, and find the paper it came from. Start anywhere — the Bayes filter is the trunk that most of the classical methods branch from.
The same twenty-four concepts, grouped by layer. Tap any one to open its detail card above (and scroll up to the graph).
The methods didn't appear at random — each answered a problem of its moment. The timeline below runs from Kálmán's 1960 paper to Barfoot's 2024 second edition, color-coded by era: classical (the linear-Gaussian foundations and tracking), robotics / probabilistic (particle filters, SLAM, inertial-visual systems, factor graphs), and deep / learned (differentiable filters and end-to-end perception fusion).
Click a milestone marker to read what it was, why it mattered, and which lesson it connects to. On a phone the timeline becomes a vertical scroll.
Click any marker on the timeline to read its story. Start at 1960 — the year Rudolf Kálmán published the recursive estimator that the next sixty years would extend, linearize, sample, and eventually learn.
The full bibliography for the series, in two groups: the textbooks and courses you'd keep on the shelf, and the seminal papers behind each lesson. Where a paper has an open arXiv copy or a stable DOI, it's linked.