Flip the Kalman filter inside out — track inverse covariance instead of covariance — and the messiest step in all of fusion, combining many sensors, collapses into a single act of grade-school addition. This is why every sensor network on Earth secretly thinks in information, not in spread.
Picture a warehouse with twelve cameras bolted to the ceiling. A forklift rolls across the floor and you want to know exactly where it is. No single camera can tell you: one sees only a corner, another is half-blocked by a shelf, a third catches the forklift edge-on so it can pin down the side-to-side position but is hopeless about distance. Each camera holds a sliver of the truth. Your job is to pool all twelve slivers into one sharp estimate — and to do it fast enough to run live, twelve times a second.
You already know one way to do this. Back in Lesson 3 we fused two sensors with the inverse-variance rule: weight each by how precise it is, blend, done. And in Lesson 5 the Kalman filter generalized that to a moving target, tracking a mean (our best guess of the forklift's position) and a covariance (how uncertain that guess is, a fuzzy cloud around the mean). The Kalman filter is gorgeous. But ask it to fuse twelve sensors and it starts to sweat.
Here's why. The Kalman measurement update, written in the covariance form, has a step where you invert a matrix — you flip the whole uncertainty cloud over — and you must do it once for the batch of sensors, juggling the dimensions of all of them at once. Matrix inversion is the expensive, fragile, hard-to-distribute operation in the whole pipeline. With two sensors it's fine. With twelve sensors scattered across a network, each wanting to chip in its little observation, the covariance form turns into a bookkeeping nightmare: who inverts what, in what order, and how do you add a thirteenth camera tomorrow without re-deriving everything?
There is, and it is almost too simple to believe. Stop tracking the uncertainty cloud (covariance) and start tracking its inverse. The inverse of "how uncertain am I" is "how certain am I" — how much hard information you hold. And when you bookkeep in information instead of in spread, fusing a new sensor stops being a matrix inversion and becomes… addition. Camera 1 brings some information. Camera 2 brings some more. The total information you hold is just the sum:
That is the entire pitch of this lesson. The same Kalman filter, rewritten to track inverse-covariance, is called the Information Filter. Its killer feature is that combining independent sources — the act we call fusion — is pure addition, no inversion in sight. This is exactly why decentralized and distributed sensor networks (the architectures from Lesson 2) love the information form: every node simply adds its information into a shared pool, in any order, and the pool ends up identical to what a single all-knowing central filter would have computed.
Let's see the accumulation before we name anything. Below, a target sits somewhere in the room (the true position is the small cross). Sensor nodes around the edge each report a noisy, partial observation. Click Add a sensor to bring nodes online one at a time, and watch the estimate's uncertainty — drawn as an ellipse — tighten as information piles up.
The small cross is the true target. Each sensor node on the rim contributes a sliver of information. Press Add a sensor to bring nodes online; the fused estimate (dot) sharpens and its uncertainty ellipse shrinks as the slivers ADD up. Press Reset to start from total ignorance (a huge cloud).
Watch the very first add: the estimate snaps from "anywhere in the room" to a fat blob. Each further add carves the blob smaller, and crucially, the ellipse tightens most in the direction that sensor measures well. The blob doesn't shrink uniformly — it shrinks where the new evidence speaks. That directional tightening is the information form doing its accounting, and the entire computation behind each click is one addition.
It is worth feeling why addition is so much better than what the covariance form forces on you, because it explains the entire architecture of modern sensor networks. Suppose three sensors want to contribute. In the covariance (Kalman) world, the combined update is something like "take the prior cloud, take each sensor's cloud, and solve a coupled system that inverts a matrix mixing all of them together." The sensors are entangled in the math: you cannot finish sensor 1's contribution without knowing about sensors 2 and 3, because they share the inversion.
In the information world, the three contributions are independent terms in a sum. Sensor 1 computes its information and ships the number. Sensor 2 does the same, on its own, knowing nothing about sensor 1. The pool adds them. There is no shared inversion to coordinate, no required ordering, no central choke point. This is the difference between three accountants who must sit in one room passing a single ledger around (covariance) and three accountants who each mail in a receipt that the office simply tallies (information). The second scales; the first does not.
This lesson rebuilds the Kalman filter you know in this flipped coordinate system, one piece at a time. We'll define information as inverse-variance (Chapter 1), promote it from a single number to a vector and matrix (Chapter 2), prove that the measurement update is pure addition — and that it gives the exact same answer as Lesson 3's inverse-variance fuse (Chapter 3), confront the one step where the information form is awkward rather than elegant (Chapter 4, prediction), and finally show how this single property — "fusion is addition" — powers decentralized sensor networks (Chapter 5) and a live showcase (Chapter 6). The four recurring chapters — Use Cases, Practical, Debugging, Connections — close it out.
Before matrices, before filters, let's nail the single idea everything else rests on, using one humble scalar. We have a quantity we're unsure about — say the forklift's x-position — and we describe our uncertainty with a variance, written σ² (sigma-squared). Variance is the spread of our belief: a big σ² means a wide, vague cloud ("could be anywhere in a 10-meter range"); a small σ² means a tight, confident spike ("it's at 4.2 m, give or take a centimeter").
Now flip it. Define information — we'll call it Ω (the capital Greek omega, the standard symbol) — as one over the variance:
That's the whole definition. Information is the reciprocal of spread. Where variance measures how vague you are, information measures how sharp you are. They point in opposite directions:
| Your belief is… | Variance σ² | Information Ω = 1/σ² |
|---|---|---|
| razor-sharp ("4.20 m ± 1 cm") | tiny (0.0001 m²) | huge (10000) — you know a lot |
| fuzzy ("somewhere around 4 m ± 1 m") | moderate (1 m²) | moderate (1) |
| vague ("4 m ± 5 m, who knows") | large (25 m²) | small (0.04) — you know little |
| total ignorance ("anywhere on Earth") | ∞ | 0 — you know nothing |
Read the bottom row carefully, because it is the single most useful consequence of the flip and we will lean on it repeatedly: zero information means total ignorance. If you know nothing, your variance is infinite (the cloud fills the universe), and infinite variance inverts to exactly zero information. That is a finite, well-behaved number — zero — not the messy infinity it replaces. In the covariance world, "I know nothing" is a singular, uncomputable infinity. In the information world it is the cleanest number there is. We'll cash this in hard in Chapter 8: it lets you start a filter with no prior at all.
You might ask: why introduce a whole new symbol for "one over variance"? Because the moment you do, a deep fact about fusion becomes visible. Recall from Lesson 3 the strange and beautiful result: when you fuse two independent estimates of the same quantity, their precisions add. Let's re-derive it, now naming the precisions "information."
Two sensors measure the forklift's x. Sensor A says x ≈ 100 with variance σ²A = 4 m². Sensor B says x ≈ 104 with variance σ²B = 16 m² (four times sloppier). Convert each to information:
Now the magic. The fused information is just the sum of the two informations:
And to read off the fused variance, flip back:
Three things to notice, each load-bearing. First, the fused variance 3.2 m² is smaller than either input (4 and 16) — fusing always sharpens, because you can only gain information by adding more. Second, the fusion step itself — 0.25 + 0.0625 — was pure addition; no inversion, no weighting fractions to normalize. Third, the only inversions were at the edges: convert each variance to information once (1/4, 1/16), add in the middle, convert back once (1/0.3125) at the end. The expensive operation got pushed to the boundary and the cheap operation (addition) does the actual fusing. That asymmetry is the entire reason the information form exists.
Information Ω tells us how sharp we are, but not where the peak sits. For that we need a partner quantity, and we'll build it properly in Chapter 2. For now, a preview using the scalar case: the fused mean is the information-weighted blend, which works out to the same inverse-variance weighting from Lesson 3. Quick check with our numbers — weighting each sensor's reading by its information and normalizing:
The fused estimate, 100.8 m, leans toward sensor A (the sharper one) exactly as it should — identical to the answer the Kalman/inverse-variance fuse gave in Lesson 3. The numerator Ω·x is doing something important: it's the reading scaled by how much we trust it. We'll christen that quantity the information vector in the next chapter, because it, too, will turn out to add.
Below, drag the variance slider and watch information move in the opposite direction. As you make the belief sharper (small variance, tall narrow curve), the information needle climbs; make it vague (large variance, short wide curve) and information falls toward zero. The two are locked in a seesaw: one up means the other down.
The curve is your belief about the target's position — tall and narrow means confident, short and wide means vague. Drag the variance slider. Watch the information Ω = 1/σ² readout move the opposite way. Slide variance toward its max to approach total ignorance (information → 0).
One number was enough for a forklift's x-position. But a real target lives in two or three dimensions, and its uncertainty has shape — the edge-on camera from Chapter 0 was sharp side-to-side and blind in depth, so its uncertainty cloud was a long thin ellipse, not a circle. To capture shape we need to promote our two scalars (σ² and Ω) into a matrix pair. This chapter builds the information vector and information matrix — the exact duals of the mean and covariance you know from the Kalman filter.
The Kalman filter carries two quantities to describe its belief:
The Information Filter carries two different quantities that encode the exact same belief, just flipped:
Make it concrete with a real 2×2. Suppose our belief about a target's [x, y] position has covariance
This says: variance 4 m² in x (fairly sharp, ±2 m), variance 25 m² in y (vague, ±5 m), and the zero off-diagonals mean x and y are uncorrelated. The cloud is an ellipse wide in y, narrow in x. Invert it — for a diagonal matrix you just reciprocate each diagonal entry — to get the information matrix:
Read Ω directly: 0.25 units of information about x, 0.04 about y. The sharp direction (x) carries the most information; the vague direction (y) carries little. The information matrix is a map of "how much I know, broken down by direction." A big diagonal entry = a sharp, well-pinned axis; a small one = a fuzzy axis. The edge-on camera from Chapter 0 would have a big entry along the direction it can see and a near-zero entry along the depth it can't — its information matrix is almost rank-deficient, holding information in one direction only. (Hold that thought; it returns in Chapter 9 as a debugging hazard.)
Now the vector. Say the mean of this belief is μ = [10, 20] (target at x = 10, y = 20). The information vector is Ω times μ:
So ξ = [2.5, 0.8]. By itself this vector looks meaningless — it is the mean scrambled by the information — and that's the point of the next chapter: ξ is the form in which evidence adds. To get the human-readable mean back, undo the scramble by multiplying by Ω−1 = Σ:
We recovered [10, 20] exactly — the round trip μ → ξ → μ is lossless, confirming ξ and Ω together hold the very same belief as μ and Σ. The conversion is the only place an inversion lives, and we do it just twice: once to enter the information world, once to leave it.
| Concept | Moment form (Kalman) | Information form (this lesson) | Conversion |
|---|---|---|---|
| Center of belief | mean μ | information vector ξ | ξ = Σ−1μ = Ωμ |
| Shape of belief | covariance Σ | information matrix Ω | Ω = Σ−1 |
| Sharp direction | small variance (tight) | large Ω entry | reciprocal |
| Total ignorance | Σ → ∞ (singular) | Ω = 0 (clean!) | — |
The information matrix has a beautiful geometric reading: its eigenvectors point along the ellipse's axes, and bigger eigenvalues mean sharper (shorter) axes. Below, edit the two diagonal entries of Ω and watch the uncertainty ellipse respond. Crank the x-information up and the ellipse squeezes thin in x; crank it down toward zero and the ellipse stretches out (you know less and less about x). This is the picture of "information as sharpness, broken down by direction."
The ellipse is the uncertainty cloud for a belief whose information matrix is diagonal with the entries you set. More information in a direction = thinner ellipse in that direction (you know more, so less spread). Slide either Ω entry toward zero and watch the ellipse balloon — ignorance returning along that axis.
This is the chapter the whole lesson is built around. We now prove, with arithmetic you can do on a napkin, that fusing a measurement in the information form is nothing but addition — and that the answer it produces is exactly, to the last decimal, the inverse-variance fuse you learned in Lesson 3. The elegance is not a coincidence; it falls straight out of the algebra of multiplying Gaussians, which in the information coordinates becomes adding the two information pieces.
When a sensor reports an observation, it carries its own information: a measurement information matrix and a measurement information vector. Call them Ωz and ξz (the z is for the measurement, traditionally written z). The Information Filter measurement update is breathtakingly simple:
That is the entire update. Add the new information to what you already hold. No Kalman gain to compute, no matrix to invert, no fraction to normalize — in the covariance form the update is a tangle of K = ΣHT(HΣHT+R)−1 with an inversion baked in; in the information form it is a plus sign. Every drop of evidence you collect is simply tallied onto the running total.
Let's fuse the two sensors from Chapter 1, this time as a full measurement update, and confirm we land on the same 100.8 m / 3.2 m² we got before. Start from total ignorance — no prior, which the information form handles trivially as Ωold = 0, ξold = 0.
Sensor A: reads xA = 100 with variance 4, so its information is ΩA = 1/4 = 0.25 and its information vector is ξA = ΩA·xA = 0.25×100 = 25.
Sensor B: reads xB = 104 with variance 16, so ΩB = 1/16 = 0.0625 and ξB = 0.0625×104 = 6.5.
Now run the update twice — add A, then add B — onto the zero prior:
Two additions for the information, two for the vector. To read the human answer, convert back at the very end:
There it is: x̂ = 100.8 m, σ² = 3.2 m² — identical to the inverse-variance fuse of Lesson 3 and the weighted-average of Chapter 1, computed here with nothing but additions in the middle and two conversions at the boundary. Same answer, easier road. This is the proof the lesson promised: the information-add fuse is the inverse-variance fuse, just done in coordinates where the fuse is addition.
Addition is commutative: 25 + 6.5 = 6.5 + 25. So it doesn't matter whether sensor A reports before sensor B or after — the pool ends up identical. It also doesn't matter if they report in one batch or one at a time across an hour. This is not a minor convenience; it is the property that lets a sprawling sensor network fuse correctly with no coordination. Each node adds its information whenever it likes, in whatever order the network happens to deliver it, and the total is always right. We'll build a whole architecture on this in Chapter 5.
The same commutativity gives you a free undo: if a sensor's reading turns out to be bad, you subtract its information back out (Ωnew = Ωold − Ωbad) and the pool returns exactly to its pre-sensor state. Try removing a measurement from a Kalman covariance update and you'll find no clean way to do it. In the information form, retracting evidence is as easy as adding it.
The information measurement update is the shortest update in all of estimation. Here it is, with the two-sensor fuse and the convert-back baked in — the entire chapter in a dozen lines:
python import numpy as np def add_measurement(Omega, xi, z, var): # a scalar sensor reading z with variance var contributes: Omega_z = 1.0 / var # its information xi_z = Omega_z * z # its information vector (info-scaled reading) return Omega + Omega_z, xi + xi_z # FUSE = ADD. that's it. # start from total ignorance: zero information Omega, xi = 0.0, 0.0 Omega, xi = add_measurement(Omega, xi, 100, 4) # sensor A -> (0.25, 25.0) Omega, xi = add_measurement(Omega, xi, 104, 16) # sensor B -> (0.3125, 31.5) # convert back to read the human answer (the ONLY inversion): var_fused = 1.0 / Omega # -> 3.2 m^2 mean_fused = var_fused * xi # -> 100.8 m (same as Lesson 3's inverse-variance fuse)
Run it in your head: each add_measurement call does one reciprocal (the sensor's variance → information), one multiply, and two adds — then hands back the new running totals. Fuse a third sensor by calling it a third time. The only division on the way out is the final 1.0/Omega. Compare this to the Kalman update's gain computation and you see why people who fuse many sensors reach for the information form by reflex.
Below, fuse sensors one at a time and watch the pool's information climb (additively) and its variance shrink. Each sensor you add nudges the estimate and sharpens the belief — and the running total of information is, visibly, just a sum.
Press Add sensor to fuse one more noisy reading of the same true value (250). The bars show each sensor's individual information stacking into the total — the stack height is literally the sum. The readout shows the fused mean and shrinking variance. Subtract removes the last sensor (information un-adds cleanly).
Every filter has two heartbeats: predict (roll the belief forward through the motion model — the target moved, so push the estimate along and let uncertainty grow) and update (fold in a new measurement — sharpen the belief). We just saw that the information form makes update trivially cheap: pure addition. Honesty demands the other side of the ledger. The information form makes predict awkward and expensive — exactly the reverse of the Kalman filter, where predict is cheap and update is the costly one. There is no free lunch; the information form moved the cost, it didn't delete it.
In the Kalman filter, prediction is intuitive. The target moves by the motion model (a matrix F), so the mean moves the same way (μ → Fμ), and the uncertainty cloud both transports and inflates, because motion is never perfectly known — you add process noise Q:
Both lines are direct: multiply by F, add Q. Easy. The cloud grows because predicting the future loses information — you become less sure of where the target is until the next measurement arrives. That's the right behavior: drive a car blindfolded for a second and your uncertainty about its position grows.
Now flip those two lines into information coordinates and the trouble surfaces. The predicted information matrix is
Count the inversions: you must invert Ω (to get back to covariance Σ), do the motion transport and add the process noise, then invert the whole thing again to return to information form. Two matrix inversions per prediction step, sandwiching the noise addition. The information form had to climb out of information coordinates, do the easy thing covariance-land does (transport + inflate), and climb back in. The thing it made cheap (update) it paid for by making the other thing (predict) expensive.
Let's walk a 1-D prediction to see the round trip in the flesh. Belief: information Ω = 0.25 (so variance σ² = 4). The target is stationary (F = 1) but we're unsure of its motion, so process noise Q = 1 m² per step. Predict:
The information fell from 0.25 to 0.2 — prediction destroys information (you know less about where the target is now). And to compute that drop we had to leave information space (1/0.25 = 4), do the easy inflation there (4 → 5), and come back (1/5 = 0.2). In 1-D those are scalar reciprocals — trivial. But in n dimensions each "inversion" is a full n×n matrix inverse, costing on the order of n³ operations, twice, every prediction step. For a high-dimensional state (a full SLAM map can have thousands of dimensions), that is the dominant cost of the whole filter.
Below, set a starting information and a process-noise level, then press Predict repeatedly. Each press rolls the belief forward one step: the information bar shrinks (you know less) and the variance grows. Crank Q up and the information drains faster — a noisier motion model loses information quicker. This is the price the information form pays for its addition-cheap updates.
Set the process noise Q. Each Predict press: invert → inflate by Q → invert back, draining the information bar. Higher Q drains it faster. Press Add measurement to fuse a reading (information jumps back up by addition — the cheap step). Watch predict and update pull in opposite directions.
Now we cash in the property from Chapter 3 — fusion is commutative addition — and watch it build an entire architecture. Back in Lesson 2 we met three fusion architectures: centralized (one computer ingests every sensor), decentralized (each sensor node has its own brain and they share results), and distributed (nodes cooperate with no central authority). The decentralized and distributed designs are robust — no single point of failure — but they raised a hard question: how do independent nodes pool their estimates correctly without a central referee? The information form answers it in one word: add.
The setup: N sensor nodes are scattered around a target. Each node makes its own local observation and, crucially, computes its own little contribution in information form — an information matrix and vector. To get the globally optimal fused estimate (the one a single all-seeing central filter would compute), the network does exactly this:
Step 2 is the miracle. The global fused estimate — the optimal one — is literally the sum of every node's local information. No node needs to know what the others measured. No node needs to act first. The "pool" can be a central tallier, or a token passed around a ring, or each node broadcasting its number to all the others — the architecture doesn't matter, because addition is commutative and associative. The math is indifferent to the topology.
Make it real with three nodes each measuring a target's x-position with different precision. (We'll keep it 1-D so you can check every step by hand; the matrix case is identical with bold letters.)
| Node | Reading xi | Variance σ²i | Information Ωi = 1/σ²i | Info vector ξi = Ωixi |
|---|---|---|---|---|
| 1 (close, sharp) | 50.0 | 1 | 1.000 | 50.00 |
| 2 (mid-range) | 52.0 | 4 | 0.250 | 13.00 |
| 3 (far, vague) | 47.0 | 10 | 0.100 | 4.70 |
Each node ships its two numbers (Ωi, ξi). The pool adds them — no coordination, any order:
Convert back to read the answer the whole network agrees on:
The fused estimate is 50.15 m with variance 0.741 m². Notice two payoffs. First, the fused estimate leans toward node 1 (the sharp one at 50.0) and barely listens to the vague node 3 at 47.0 — the information weighting did that automatically. Second, the fused variance 0.741 is smaller than node 1's variance of 1 — even the best single node is improved by pooling the others. Three nodes, three additions, and the network reached the globally optimal estimate with zero central coordination. That is decentralized fusion.
The whole scheme rests on a single assumption — that the nodes' information contributions are independent. Addition is only correct when you're adding genuinely independent pieces of evidence. If two nodes secretly share information (they both used the same prior, or one node's estimate was passed to the other and back), then adding both double-counts that shared piece, and the pool becomes falsely overconfident — the information matrix says you know more than you really do. This is the central hazard of information-form fusion, it has a name (the double-counting or data-incest problem), and the fix is the subject of Lesson 11. For now, plant the flag: add only independent information; correlated sources need a different tool (covariance intersection, the Lesson 11 topic). We'll dissect the failure in Chapter 9.
Below, place sensor nodes around a target and pool their information. Each node you add contributes a term to the sum; the fused estimate (and its ellipse) updates to the optimal pooled belief. Toggle a node off and its term simply drops out of the sum — no re-derivation, the rest still fuse correctly. This is the decentralized recipe, live.
The cross is the true target. Click Add node to drop a sensor node on the rim; each contributes its information to the pool (the fused ellipse tightens). Click a node to toggle it off — its term leaves the sum and the pool re-fuses, no central coordination. The readout shows Ωpool as a literal sum of the active nodes.
Every idea from chapters 0–5 now lives in one interactive scene. Several sensor nodes surround a target, each with its own directional view — some see the target sharply left-right but vaguely in depth, others the reverse. You toggle each node on and off and watch the fused uncertainty ellipse tighten as their informations add, and a side-by-side readout confirms the decentralized sum equals what a central filter would compute. No new theory — just the whole lesson, breakable.
The cross is the true target. Each node has an oriented view (its faint ellipse shows what it can and can't pin down). Toggle nodes with the buttons (lit = contributing). The fused ellipse is the sum of all active informations — it tightens most where the active nodes are sharp. The panel shows decentralized sum vs central filter — they match exactly. Press Run to let the target drift (predict + update live).
Play with it and the lesson announces itself through behavior, not formulas:
Every lesson in this series ends with the same four chapters — Use Cases, Practical Application, Debugging, Connections — because a method you can't point at in a shipped system is a method you don't really own. The information form is the quiet workhorse behind a surprising amount of real robotics and tracking infrastructure, precisely because it makes the one operation those systems do constantly — fuse many sources — into addition.
Decentralized sensor networks — the original use. The information form was popularized for exactly this: a web of radar, sonar, or camera nodes tracking targets with no central computer. Each node fuses its own observations into local information, broadcasts the contribution, and every node adds the received contributions to hold the same global estimate — provably identical to a (non-existent) central filter. Durrant-Whyte and Rao's decentralized data-fusion work, the foundation of modern multi-sensor tracking, is built on this additive property.
Multi-robot fusion. A team of robots mapping a building, or a swarm of drones tracking a moving object, each see a slice of the world. The information form lets each robot ship a compact information contribution and the team fuse them by addition — robust to a robot dropping out (its term just leaves the sum) and easy to grow (a new robot adds a term). The same property that helps a 12-camera warehouse helps a 12-robot swarm.
SLAM in information form — the SEIF. Simultaneous Localization and Mapping (a later lesson in this series) carries a giant state: the robot's pose plus every landmark. In the covariance form the map's covariance matrix is dense and grows expensively. In information form, the Sparse Extended Information Filter (SEIF) exploits a gorgeous fact: the information matrix of a SLAM problem is nearly sparse — most landmark pairs share almost no information, so most off-diagonal entries are near zero and can be dropped. That sparsity makes information-form SLAM scale to huge maps where covariance-form SLAM chokes. Thrun, Burgard & Fox devote a chapter to it.
Track-to-track fusion in defense / aerospace. Multiple radars each produce a track (an estimate of a target's state). Fusing tracks from independent radars into one common air picture is naturally an information-form operation: convert each track to information, add, convert back. It's the standard tool when many independent trackers must agree on one truth.
Notice the recurring logic: the information form shows up exactly where a system must fuse many sources, often without a central authority, and where extensibility (adding/dropping sources) matters.
| System | What's fused | Why the information form |
|---|---|---|
| Decentralized radar/sonar net | target tracks across nodes | no central computer; each node adds its contribution, all agree |
| Multi-robot mapping / swarm | partial observations per robot | robust to a robot dropping out; trivial to add robots (one more term) |
| SEIF SLAM | robot pose + thousands of landmarks | information matrix is sparse → scales where dense covariance can't |
| Track-to-track fusion (aerospace) | independent radar tracks | combine many trackers into one air picture by addition |
| Smart-building / warehouse sensing | many ceiling cameras / lidars | each camera adds its directional information; easy to grow the grid |
SLAM is the masterclass, and it's worth seeing why the information form wins there specifically. In SLAM the robot accumulates measurements relating its pose to landmarks. Each measurement, in information form, is a sparse addition: a landmark observation adds information only to the entries linking the robot and that one landmark — it touches a tiny corner of the giant information matrix. So the matrix stays sparse: zeros everywhere two things were never co-observed. A sparse matrix is cheap to store and (with the right tricks) cheap to work with, which is why information-form SLAM scales to city-sized maps.
The covariance form does the opposite: every measurement makes the whole covariance matrix denser (correlations leak everywhere), so it bloats and slows as the map grows. The lesson's central trade reappears: the information form's expensive step is prediction (the robot moving), but in SLAM you make far more measurements than motions — you observe many landmarks per step — so paying for cheap, sparse, additive measurement updates is exactly the right bargain. The structure of the problem picks the form.
You now know both coordinate systems and the trade between them. The practical question every engineer faces is simply: for this problem, do I track covariance (Kalman) or information (this lesson)? This recurring "now build it" chapter turns the trade-off into a procedure you can run on any project, then walks two concrete designs end-to-end.
It all comes down to which step you do more of, because the two forms have mirror-image costs (Chapter 4):
Three questions pin down the choice:
One practical trick deserves its own spotlight because the Kalman form simply can't do it cleanly. To start a filter, you need an initial belief — a prior. But what if you genuinely have no prior, no idea where the target is? In the covariance form you'd need an "infinite covariance," which is singular and breaks the math (you can't invert it in the update). In the information form, no prior = zero information, and zero is a perfectly ordinary number:
You initialize the filter at zero and let the first measurement's information be the entire belief (0 + Ωz = Ωz). We did exactly this in Chapter 3's worked fuse — started from Ω = 0, ξ = 0, and the two sensors built the belief from scratch. This non-informative prior is clean, standard, and a real reason to pick the information form when you want to start a filter with honest ignorance rather than a made-up guess.
The task: track a forklift's [x, y] using 12 ceiling cameras, fused live, with the grid easy to expand. Walk the procedure. Q1: distributed? Yes — 12 cameras, ideally no single choke-point computer (and one camera dying shouldn't blind the system). Q2: many sources per step? Twelve measurements every frame, and the forklift moves slowly (one predict between batches of twelve updates) — measurement-heavy. Q3: sparse structure? Not especially, but Q1 and Q2 already point hard at the information form.
The design: each camera computes its directional information contribution (sharp along the direction it sees the forklift, vague in depth), ships the two numbers, and a tally (central or gossiped) adds them. Start the filter at Ω = 0 (no prior — we don't know where the forklift starts). Adding camera 13 next month = adding one term. A camera failing = its term drops out. The expensive predict happens only once per slow forklift step, so its cost is amortized over twelve cheap additive updates. Information form, decisively.
Now flip every answer. The task: estimate a racing drone's orientation at 1000 Hz from one IMU. Q1: distributed? No — one onboard computer, one sensor. Q2: many sources per step? No — one sensor, and the drone's attitude changes violently, so you predict the fast dynamics many times between the relatively sparse usable measurements. This is prediction-heavy: lots of fast motion steps, few independent sources. Q3: sparse structure? No.
The design: every question points to the Kalman/covariance form, because its cheap step (predict) is the one you do constantly, and its expensive step (update) happens rarely. Wrapping this in the information form would mean paying the double-inversion predict cost a thousand times a second to save on an update you barely do — backwards. Kalman form, decisively. Same estimation problem, opposite form, chosen purely by the predict-vs-update count.
The information form's elegance hides three specific failure modes, and the deadliest of them — double-counting — is a direct consequence of the very property that makes the form so attractive (addition). This recurring chapter catalogs the traps, the symptom each shows in the field, and the test that exposes it before it ships.
This is the big one, the failure that the addition trick is one mistake away from. Recall the fatal assumption from Chapter 5: you may only add independent information. When two sources secretly share a piece of evidence and you add both, that shared piece gets counted twice, and the pool becomes falsely overconfident — the information matrix claims you know far more than you actually do, and the uncertainty ellipse shrinks to a tiny lie.
How does it happen? The classic route: node A sends its estimate to node B; B fuses it with its own and sends the result back to A; A, not realizing B's message already contains A's own contribution, adds it again. Now A's own information is double-counted. This is the data-incest (or rumor-propagation) problem — information echoing around a network gets re-summed as if it were fresh evidence.
Make the failure numerical. Suppose a true single piece of evidence is Ω = 0.25 (variance 4). If two correlated sources each carry that same evidence and you naively add them, you get Ω = 0.5 (variance 2) — you now believe you're twice as certain as the evidence warrants. Your ellipse is half the area it should be; your filter will confidently reject correct measurements that disagree with its phantom precision. Overconfidence is more dangerous than under-confidence — a too-tight filter stops listening and diverges.
The information form's predict step (Chapter 4) needs two matrix inversions per step. Two things go wrong. First, cost: in a high-dimensional state, paying n³ twice per prediction can dominate your runtime — if your profiler shows the filter pinned in the predict step, you may have picked the wrong form for a prediction-heavy problem (re-read Chapter 8). Second, numerical conditioning: the matrices you invert during predict can become ill-conditioned (nearly singular), so the inverse amplifies floating-point error and the filter's numbers drift or blow up.
Chapter 8 sold you the zero-prior superpower: start at Ω = 0 (total ignorance). True — but it has a sharp edge. A zero information matrix (or any singular / non-invertible one) means you cannot yet convert back to a mean, because reading the mean requires μ = Ω−1ξ and you can't invert a singular matrix. Worse, a single directional sensor (like Chapter 2's edge-on camera) contributes information in one direction only, so its information matrix is rank-deficient — if that's all you've fused, Ω is still singular and there is no mean to read in the unobserved direction.
| Check | How | Catches |
|---|---|---|
| Independence audit | Could any two added contributions share evidence (echoed estimates, shared priors)? | Trap 1 (double-counting) |
| Consistency / NEES test | Are actual errors much bigger than the claimed ellipse? | Trap 1 (overconfidence from double-counting) |
| Condition number monitor | Watch the eigenvalue ratio of matrices inverted in predict | Trap 2 (ill-conditioned predict) |
| Profile predict vs update | Which step dominates runtime? | Trap 2 (wrong form for the problem) |
| Rank check before invert | Is Ω full-rank before reading the mean? | Trap 3 (singular init / rank-deficient) |
You can now flip the Kalman filter inside out at will, and you know exactly when that flip pays off. Before pointing at what's next, here's everything in one place — the cheat-sheet to screenshot, the motto, the real sources, and the cross-links.
Screenshot this. It is the whole lesson collapsed into a single lookup:
| Step | Kalman (moment) form | Information form | Cost in info form |
|---|---|---|---|
| State carried | μ, Σ | ξ = Ωμ, Ω = Σ−1 | — |
| Update (fuse sensor) | Kalman gain + inversion | Ω += Ωz, ξ += ξz (ADD) | CHEAP — addition |
| Predict (motion) | FΣFT + Q (direct) | (FΩ−1FT + Q)−1 | EXPENSIVE — 2 inversions |
| Total ignorance | Σ → ∞ (singular) | Ω = 0 (clean) | trivial |
| Fuse N sources | entangled batch inversion | sum of N terms | N additions |
This was Lesson 9 of 22, in the spatial-fusion arc. The information form you just learned is the bridge between the classical filters (Lessons 4–8) and the distributed, large-scale fusion the rest of the series tackles — it's the form that makes SLAM (later), sensor networks, and robust multi-source fusion tractable.
| Lessons | Arc | How the information form connects |
|---|---|---|
| 1–3 | Foundations | the fusion problem; architectures (centralized/decentralized/distributed — the information form is decentralization's native language); the inverse-variance fuse the information add reproduces |
| 4–8 | Classical estimation | Bayes filter; the Kalman filter (this lesson is its inverse-covariance twin); EKF/UKF; particle filters |
| 9 | Information form (here) | the Kalman filter flipped — fusion becomes addition, enabling distributed fusion and sparse SLAM |
| 10–13 | Spatial fusion | occupancy grids; SLAM (the SEIF uses exactly this form); visual-inertial odometry; multi-target tracking & data association |
| 11, 14–18 | Robust & distributed | covariance intersection (fuses correlated info safely — the fix for this lesson's double-counting trap); fault detection; consensus over networks |
Next up, Lesson 10: Occupancy Grids — where the additive-information idea reappears in a delightfully concrete form. An occupancy grid maps the world as cells that are "occupied" or "free," and each sensor sweep adds evidence (in log-odds, which behave just like information — independent evidence adds) to each cell. You'll see the same "fusion = addition" principle from this lesson, applied to building a map instead of tracking a point.