Understand the Kalman
Filter

Chapter 1: Gaussians — The Language of Uncertainty

Everything the Kalman filter does is expressed in the language of Gaussian distributions (bell curves). A Gaussian is defined by just two numbers: the mean (your best guess) and the variance (how uncertain you are). A small variance means you're confident. A large variance means you're shrugging your shoulders.

Why Gaussians? Because they have a magical property: the combination of two Gaussians is another Gaussian. This means the Kalman filter can fuse predictions and measurements using simple arithmetic — no expensive computations, no approximations. It's exact.

Chapter 2: The State — What Are We Tracking?

Before we can filter anything, we need to decide: what are we tracking? For a ball rolling on a table, we care about its position and its velocity. Together, these form the state vector x. The state is not what we observe — it's what we believe about the world.

We also keep track of our uncertainty about the state: the covariance matrix P. If P is small, we're confident. If P is large, we're unsure. The Kalman filter updates both x and P at every step.

Concrete Shapes: 2D Constant-Velocity Tracker

To make this real, let's track a ball in 2D. We track its x-position, x-velocity, y-position, and y-velocity. Here are the exact matrix shapes — every single matrix in the Kalman filter, with dimensions:

Chapter 3: The Predict Step

At each timestep, before we look at any measurement, we use physics to predict where the ball should be. If the ball was at position 5 with velocity 2, then after one timestep it should be at position 7. This is the predict step.

The state transition matrix F encodes our motion model. For constant velocity: new_position = old_position + velocity × dt. But our model isn't perfect, so we add process noise Q to account for things we can't model (wind, bumps, etc.). Uncertainty always grows during prediction.

Q and R in Practice: Where Do They Come From?

R comes from datasheets. Your GPS says ±3 meters accuracy? That means one standard deviation is 3m, so R = 9 m² (variance = σ²). A laser rangefinder with ±0.01m gives R = 0.0001. R is a property of the sensor hardware — you look it up.

Q is tuned by the engineer. Q represents how much you trust your physics model. Big Q says "my model is approximate — expect surprises." Small Q says "I trust the model — the world is predictable." There is no datasheet for Q because it captures everything your model doesn't account for: wind gusts, bumps, unmodeled friction.

Chapter 4: The Measurement

Now the camera takes a picture and says "I see the ball at position 7.3." But we know the camera is noisy — it might be off by a bit. The measurement z has its own uncertainty, captured by the measurement noise covariance R.

The observation matrix H maps our state to what the sensor actually sees. If we're tracking [position, velocity] but the camera only measures position, then H = [1, 0] — it picks out just the position part.

Chapter 5: The Kalman Gain

Here's the million-dollar question: how much should we trust the measurement vs our prediction? The Kalman gain K answers this. It's a number between 0 and 1 (roughly speaking) that acts as a trust slider.

When your sensor is very accurate (R is small), K is large — you trust the measurement. When your prediction is very confident (P is small), K is small — you trust the prediction. The formula automatically balances these:

Chapter 6: The Update Step

Now we fuse the prediction and measurement. The innovation (or residual) is the difference between what we measured and what we expected: y = z − H x̂¯. The Kalman gain K scales this innovation and adds it to our prediction:

The updated covariance P is always smaller than the predicted covariance P¯. This is the magic: every measurement, no matter how noisy, makes our estimate more certain. Information only accumulates; it never destroys.

Chapter 7: The Full Algorithm

That's it. The entire Kalman filter is just two steps repeated forever: Predict (use physics to guess forward, uncertainty grows) then Update (incorporate a measurement, uncertainty shrinks). The predict-update cycle is the heartbeat of the filter.

The Complete KF in Python

Python
import numpy as np

def kalman_filter(x, P, F, H, Q, R, measurements):
    """Run a Kalman filter over a sequence of measurements."""
    estimates = []
    for z in measurements:
        # ── Predict ──
        x_pred = F @ x                    # state prediction
        P_pred = F @ P @ F.T + Q          # covariance prediction

        # ── Update ──
        y = z - H @ x_pred                # innovation (residual)
        S = H @ P_pred @ H.T + R          # innovation covariance
        K = P_pred @ H.T @ np.linalg.inv(S)  # Kalman gain
        x = x_pred + K @ y                # updated state
        P = (np.eye(len(x)) - K @ H) @ P_pred  # updated covariance

        estimates.append(x.copy())
    return estimates

Plugging In Real Numbers

Let's wire up the 2D constant-velocity tracker from Chapter 2. State x = [p_x, v_x, p_y, v_y], and the camera measures position only:

Python
import numpy as np

dt = 0.1   # 10 Hz update

# State transition: constant velocity
F = np.array([[1, dt, 0,  0],   # px' = px + vx*dt
              [0,  1, 0,  0],   # vx' = vx
              [0,  0, 1, dt],   # py' = py + vy*dt
              [0,  0, 0,  1]])  # vy' = vy

# Observation: camera sees position only
H = np.array([[1, 0, 0, 0],    # z_px = px
              [0, 0, 1, 0]])   # z_py = py

# Measurement noise: GPS ±3m → variance = 9
R = np.diag([9.0, 9.0])    # [2,2]

# Process noise: unknown acceleration q=0.5 m/s²
q = 0.5
G = np.array([[dt**2/2, 0],
              [dt,      0],
              [0, dt**2/2],
              [0,      dt]])
Q = q * G @ G.T              # [4,4]

Variable Reference

python
def kalman_1d(measurements, x0=0.0, P0=100.0, F=1.0, H=1.0, Q=0.1, R=1.0):
    x = x0
    P = P0
    estimates = []
    for z in measurements:
        # ── Predict ──
        x_pred = F * x           # state prediction
        P_pred = F * F * P + Q   # covariance prediction

        # ── Update ──
        K = P_pred * H / (H * P_pred * H + R)  # Kalman gain
        x = x_pred + K * (z - H * x_pred)      # state update
        P = (1 - K * H) * P_pred               # covariance update

        estimates.append(x)
    return estimates

Chapter 8: Track a Moving Object

Time to see the Kalman filter in action. Below, a ball bounces around the canvas. The gray circle is the true ball. The red dots are noisy camera measurements. The green circle is the Kalman filter's estimate, and the green ellipse shows the uncertainty.

Play with the sliders to see what happens when the sensor is noisy or the model is uncertain. Watch how the filter smoothly tracks the ball even when individual measurements are wildly off.

Chapter 9: Sensor Fusion

Real systems rarely have just one sensor. Your phone has GPS, an accelerometer, a gyroscope, a barometer, and Wi-Fi signal strength — all measuring different things with different noise characteristics. The Kalman filter handles this naturally: each sensor provides a measurement z with its own H and R, and the filter fuses them all.

Below, two sensors track a 1D target. Sensor A (like GPS) has low update rate but decent accuracy. Sensor B (like an accelerometer) updates fast but drifts. The Kalman filter combines both to outperform either alone.

Chapter 10: Limitations & Beyond

The Kalman filter is optimal — but only under two assumptions: the system is linear and the noise is Gaussian. What if your system is nonlinear (a rocket re-entering the atmosphere, a car turning)? What if the noise distribution has heavy tails or multiple modes?

Several extensions address these limitations. Each trades off accuracy against computational cost. The original KF remains the gold standard when its assumptions hold — which is surprisingly often in practice.

What Breaks (and How to Tell)

The Kalman filter can fail silently — it keeps producing estimates even when those estimates are garbage. Here are the three most common failure modes practitioners hit: