The Trip Analyst

M1 · Load & Look (Reproduce)

Before any estimator: load every stream, put them in one frame, and measure what you actually have. The pose ground truth arrives as ECEF coordinates — meters from the center of the Earth (the first sample is [-2,712,087.5, -4,261,670.1, 3,881,014.5]). Useless for plotting a 1 km drive. You will convert everything to a local ENU (East-North-Up) frame anchored at the first GNSS fix.

The conversion is two pieces of honest geometry. First, geodetic→ECEF for the reference point on the WGS-84 ellipsoid (a = 6,378,137 m, e² = 6.694×10^-3):

then a fixed rotation of the difference vector into the local tangent plane. Every position in the expedition — truth, fixes, filter outputs — lives in this one frame from here on. Frame mismatches are the #1 silent killer in fusion work; you are eliminating the possibility on day one.

Work it by hand once

Take the reference fix the data actually gives you: φ = 37.72°N, λ = −122.50°, h = 33.4 m. The prime-vertical radius: sin(37.72°) = 0.6118, so N = 6,378,137 / √(1 − 0.006694 × 0.6118²) = 6,386,143 m — 8 km longer than the equatorial radius, because the ellipsoid flattens toward the poles and the surface normal has farther to travel to reach the axis. Then (N+h)cosφ = 5,051,530 m is the distance from Earth’s spin axis, and multiplying by cosλ and sinλ splits it into ECEF x and y. Do this once with a calculator and the mysterious [-2.71e6, -4.26e6, 3.88e6] in the data stops being mysterious — it’s just where San Bruno is, measured from the center of the planet.

The ENU rotation rows are readable too: the first row (−sinλ, cosλ, 0) is the unit vector pointing locally East (tangent to the latitude circle), the second points North along the meridian, the third is the outward normal. A rotation matrix is just three unit vectors stacked — check each row has length 1 and the geometry writes itself.

● EXECUTED Write load(), geo2ecef(), ecef2enu(); re-zero all clocks to the first pose timestamp; print the stats artifact. Expected golden output:

M2 · The Kalman Filter Earns 52 cm (Build + Benchmark)

Constant-velocity KF, state x = [e, n, v_e, v_n], measurements = the 57 challenge fixes. Process noise enters through the white-acceleration model Q = q·GG^T with G = [dt²/2, dt²/2, dt, dt]^T — one scalar q to tune, in units of acceleration variance. That q is the entire personality of the filter.

● EXECUTED Sweep q and record RMSE against ground truth at fix times. The golden sweep:

Read the shape, not just the minimum. At q=0.05 the filter believes the car cannot accelerate more than ±0.2 m/s² — but this car pulls 1–2 m/s² merging onto 280, so the estimate lags into every speed change and does 35% worse than not filtering at all (5.09 vs 3.78). At q=2 the model is loose enough to follow maneuvers and tight enough to average noise: 3.25 m, a 14% win over the raw feed. The optimum is interior and flat — which is itself a lesson: tuning buys you half a meter here, and no tuning can buy more, because a CV model simply cannot dead-reckon through 1-second gaps. That requires the odometry you’ll wire in next milestone.

Also compute NIS (normalized innovation squared, y^TS^-1y, χ² with 2 dof, 95% bound 5.991). Golden: 100% of steps inside the band — too clean. A perfectly consistent filter sits at ~95%; 100% means S over-predicts the innovations, i.e. the filter is conservative because q=2 also has to cover model bias, not just noise. NIS reading skills come back in M3 with teeth.

Where the gain comes from — then one real update by hand

Don’t take K on faith. In one dimension, the predict step leaves a Gaussian belief N(x̂, σₚ²) and the fix hands you N(z, σᵣ²). Multiply them (that is all Bayes with Gaussians is) and complete the square — the posterior mean lands at

K is literally my doubt divided by total doubt — a trust dial Bayes sets for you. The matrix form P H^TS⁻¹ is the same fraction wearing linear algebra.

Now run the first real update of this dataset by hand. Fix 0 lands at (0.91, −3.12); initialize there with position variance 9 and velocity variance 100 (we admit we don’t know the speed). One second later fix 1 arrives at (2.61, 11.69) — the car moved ~14.8 m north. Predict across dt = 1.00 s with q = 2:

The velocity variance flooded into position (the 100·dt² term), so the filter trusts the new fix almost completely — K = 0.92 — and, through the position–velocity covariance the predict just created, the innovation y = (1.70, 14.81) writes velocity too: v⁺ = 0.852 × (1.70, 14.81) = (1.45, 12.62) m/s. The filter measured speed with a position sensor, one update in. Position doubt collapses 109.5 → 8.32. Every later update is this same arithmetic with smaller numbers — which is why the gain settles near 0.4 within a few fixes and the path stops jumping.

M3 · Dead-Reckoning EKF (Build + Break & Debug)

The car carries better sensors than GPS: wheel speed at 83 Hz and a gyro at 104 Hz. Between two 1-second fixes those can dead-reckon to centimeter-class accuracy — if you carry a heading. Enter the unicycle EKF, state [e, n, θ, v]:

The nonlinearity lives in cosθ/sinθ, so you linearize. Derive the two Jacobian entries yourself — one partial derivative each: ∂(e + v cosθ dt)/∂θ = −v sinθ dt and ∂(n + v sinθ dt)/∂θ = v cosθ dt. Read them physically: at v = 19 m/s, a heading error of 1° (0.0175 rad) displaces the predicted position by 19 × 0.0175 ≈ 0.33 m per second, perpendicular to travel. That coupling runs both directions — it is how position fixes can correct a heading the filter never measures (heading RMSE 2.34° with zero heading sensors), and equally how position noise can corrupt heading if P_θ is allowed to grow. Hold that thought for the tuning surprise below. Predict at ~10 Hz with CAN+gyro; correct with a fix once a second. ● EXECUTED Golden result: RMSE 2.83 m vs the KF’s 3.25 m (a 13% win), heading RMSE 2.34° without ever measuring heading — it is observable only through the F-coupling. NEES sits at 98.8% inside the 95% band: honest.

Break it on purpose: the drifting gyro

● EXECUTED Add 0.01 rad/s to every gyro sample — a realistic uncompensated MEMS bias. Do the damage arithmetic before running it: 0.01 rad/s is 0.57°/s, so between two fixes the heading walks 0.57° off — only ~0.1 m of lateral error per gap, seemingly harmless. But the fixes observe heading only weakly, so it accumulates: ten un-corrected seconds is 5.7°, and at 19 m/s that steers the dead-reckoning sideways at 1.9 m/s — faster than 3-m-noise corrections can pull back. A bias is not noise: it does not average out, it compounds. Golden damage: RMSE 2.83→4.27 m, heading 2.34→3.95°, and — the point of the exercise — NEES inside-rate collapses from 98.8% to 61%. The filter’s covariance says “I should be within ~2 m” while its error says otherwise, and NEES is exactly the statistic that detects the contradiction. A filter cannot fix a bias it doesn’t model, but it can tell you one exists. (The production fix — augmenting the state with a bias estimate — is the error-state KF you can chase in the ESKF lesson.)

M4 · Regimes: the Hidden Markov Layer (Build)

“Position at t=30.2s” is a Kalman question. “Was the driver braking?” is not — braking is a discrete hidden state that persists for seconds and switches abruptly. Wrong tool, wrong uncertainty. The right tool is a 4-state HMM over features the car already gave you: longitudinal acceleration a (from smoothed CAN speed) and yaw rate |ω| (smoothed gyro).

Hand-set the emission model first — you know the physics: cruise centers at a=0, accel at +0.8 m/s², brake at −0.9 m/s², turn at |ω|=0.055 rad/s, each a diagonal Gaussian. The transition matrix is one number: stay-probability p_stay=0.95, rest spread evenly. At 10 Hz that encodes “regimes last ~2 s, not ~0.1 s”. Decode with Viterbi — dynamic programming over log-probabilities, the single most reusable 15 lines in this expedition.

Derive the recurrence, then price a switch

The most probable path maximizes Σₖ [log p(xₖ|sₖ) + log A(sₖ₋₁, sₖ)]. Because a path’s score splits into (score up to k−1) + (one transition) + (one emission), the best path ending in state j at frame k only needs the best paths at k−1:

That max-with-memory is the whole algorithm; backpointers record which i won. Now price a switch with our numbers: p_stay = 0.95 spreads 0.05 across 3 alternatives, so each switch costs log(0.95/0.0167) ≈ 4.04 nats. A one-frame blip must switch in and out — 8.1 nats — and since a Gaussian log-likelihood buys nats at ½z², the blip needs to be a 4σ emission event to survive. Cruise has σₐ = 0.25 m/s², so a single frame of |a| < 1 m/s² cannot break out of cruise — but a sustained brake pays the toll once and keeps the winnings every frame after. That is exactly which 36 of argmax’s 64 segments died: every blip too weak to pay 8 nats.

● EXECUTED Golden decode: 28 segments in 60 s, occupancy cruise 61.0% / accel 20.7% / brake 16.9% / turn 1.4% (it’s a highway — turning is rare and brief), 93.3% agreement with instantaneous threshold rules. The interesting number is what the HMM removed: per-frame argmax over the same emissions — no transition model — produces 64 segments, flickering at every noise blip. The Markov prior deleted 36 phantom maneuvers without touching the emissions. That is what “temporal structure is information” means, executed.

M5 · Learned Emissions: the ML Layer (Build + Modify)

Hand-set Gaussians worked because a and ω map to regimes by obvious physics. Real products face features where no one can write the emission density down. So swap the layer: keep the HMM skeleton, learn the emissions. Features: 0.5-s windows (±2 frames) of [mean(a), std(a), mean(ω), std(ω)], standardized. Labels: the threshold rules. Model: softmax regression, 20 parameters, trained by 600 steps of batch gradient descent you write yourself.

Derive the gradient once and it stops being a library incantation. Cross-entropy loss L = −log pₛ for the true class; softmax gives pᵢ = eʻᵢ/Σₖeʻₖ. Differentiate through the log and the normalizer and everything collapses to

Prediction minus truth, times input — over-believe a class and its weights get pushed away from this feature vector; under-believe and they get pulled toward it. Sanity-check on a real window from the braking run near t = 51 s: an untrained model outputs p = 0.25 everywhere, so the brake row’s gradient is (0.25 − 1)x = −0.75x while each wrong class gets +0.25x — the true class is corrected three times harder, which is why accuracy climbs fastest in the first fifty steps. Watch exactly that in the trainer below.

● EXECUTED Golden: train 88.1% / test 86.2% on a seeded 70/30 split — a 2-point gap, no meaningful overfit at 20 params for 579 frames. Feed log p(class|window) into the same Viterbi as pseudo-log-likelihoods (this hybrid is exactly how classic speech recognition wired neural nets into HMMs). Result: 11 segments vs the hand-set 28, at 91.4% frame agreement — the windowed features already average out flickers, so the learned layer plus the Markov prior yields the cleanest timeline yet.

M6 · The Convergence: HMM × KF = IMM (Build + Benchmark)

Now the punchline the whole expedition has been arranging. M2 left you with a dilemma dressed as a tuning table: q=0.1 is superb during cruise and lags in maneuvers (4.41 m overall); q=5 follows maneuvers and chases noise in cruise (3.31 m). M4 gave you machinery for exactly this shape of problem: discrete hidden regimes with sticky transitions. Put a Markov chain over which filter is right and you have the Interacting Multiple Model estimator — run both KFs, mix their states by regime probability before each predict, update the regime belief from each filter’s measurement likelihood. An HMM whose hidden states are Kalman filters.

Derive the mix, then do one by hand

Why mix before predicting? Total probability: the prior that the mode is j now, before the new fix, is c̄ᵣ = Σᵢ PIᵢᵣ μᵢ — the M4 transition matrix acting on last cycle’s posteriors. And the state filter j should restart from is the mode-conditioned mixture of both filters’ states, weighted wᵢᵣ = PIᵢᵣ μᵢ/c̄ᵣ. One hand pass with μ = (0.8, 0.2), p_stay = 0.95:

Read the asymmetry: the quiet filter restarts almost purely from itself, but the maneuver filter — whose own posterior weight was only 0.2 — restarts 17% seeded from the quiet filter’s state. That seeding is the entire trick. In plain MMAE (no mixing), a filter that has been wrong for ten seconds is lost in its own fiction and takes seconds to recover when its regime finally arrives; IMM keeps every filter perpetually one mix away from the consensus, so the handoff at a cruise→brake boundary is instant. The covariance mix adds the spread term Σ wᵢ(Pᵢ + (xᵢ−x̄)(xᵢ−x̄)^T) — disagreement between the filters is itself uncertainty, and honest bookkeeping records it.

● EXECUTED Golden, on the same 57 fixes: IMM 3.106 m < best single 3.312 m < quiet single 4.411 m. And the mode belief behaves like a regime detector for free: mean maneuver-model probability 0.365 during real maneuvers vs 0.280 in cruise. Modest margins — honestly reported — because 60 s of highway has limited regime drama; the structure of the win is what scales.

M7 · The Analyst: an LLM With Tools, Not Opinions (Build + Benchmark)

The top layer answers language. “Was I turning at t=30?” must become: call the regime decoder, read the timeline, answer — never estimate from vibes. The architecture rule: the LLM narrates over estimates; it never estimates. Filters run at 100 Hz for cents of CPU; the LLM runs at query time for cents per query. Inverting that — asking the model to eyeball a trajectory — buys hallucinated positions at premium prices.

Expose your pipeline as four tools and drive the standard manual loop (Python SDK, model claude-opus-4-8, adaptive thinking). The grounding contract lives in the system prompt: answer only from tool results; if a tool can’t answer, say so.

● EXECUTED (harness + deterministic stub)  ■ DOCS+PINNED (live API call — current SDK surface, verified against the official docs 2026-07-28)

import anthropic, json
client = anthropic.Anthropic()                    # reads ANTHROPIC_API_KEY

TOOLS = [
  {"name": "get_regimes", "strict": True,
   "description": "Regime timeline from the hybrid HMM: [{regime,t0,t1}]",
   "input_schema": {"type":"object","properties":{},"required":[],"additionalProperties":False}},
  {"name": "get_state_at", "strict": True,
   "description": "EKF state {e,n,heading_deg,speed_ms} at time t seconds",
   "input_schema": {"type":"object","properties":{"t":{"type":"number"}},
                    "required":["t"],"additionalProperties":False}},
  # + get_stats, get_imm_modes — same shape
]

msgs = [{"role":"user", "content": question}]
while True:
    r = client.messages.create(
        model="claude-opus-4-8", max_tokens=16000,
        thinking={"type":"adaptive"},
        system="You are a trip analyst. Answer ONLY from tool results. "
               "Report numbers exactly as tools return them.",
        tools=TOOLS, messages=msgs)
    if r.stop_reason != "tool_use": break
    msgs.append({"role":"assistant", "content": r.content})
    results = [{"type":"tool_result", "tool_use_id": b.id,
                "content": json.dumps(run_tool(b.name, b.input))}
               for b in r.content if b.type == "tool_use"]
    msgs.append({"role":"user", "content": results})   # ALL results in ONE message

The milestone is the eval harness, not the loop: 6 questions whose answers are computed deterministically from your own pipeline (top speed 19.8 m/s at t=9.8 s; 4 braking segments; 66.8% cruise; not turning at t=30; 1011.3 m; IMM maneuver belief did exceed 0.5). Score the model’s answers against them programmatically. The golden run validates the harness with a deterministic stub (100%); your live run must score ≥5/6. Log tokens in/out — at current Opus pricing the whole eval costs a few cents; put the number in your artifact and know your unit economics from day one.

M8 · Publish the Benchmark (Publish)

The expedition ends with an artifact a stranger can run: a public repo where python trip_analyst.py downloads the 1.6 MB of signals, reproduces every number in this page, and regenerates the results table. Reproducible beats impressive — a README with a table of ATE numbers and the command that recreates them is worth more to a hiring engineer than any screenshot.

Required README sections — empty section = incomplete: Setup (the two commands), Hardware (“any laptop, $0”), Method (four layers, one paragraph each), Results (the table below, from your artifacts), Reproduce (exact commands + seed), Limitations (60 s, one segment, pseudo-labels from rules, IMM margin modest — honesty is a section, not a vibe).

Then write the three-sentence “what broke” note — the gyro-bias NEES collapse, the humble-Q surprise, the already-filtered ublox discovery. What broke and how you knew is the part that reads as competence.

Bricks Earned & Where They Recur

Each capability you just verified is a brick — it recurs in other builds, and having earned it here means the next expedition starts warmer:

run-ekf-on-real-logs decode-regimes-hmm train-emissions-model fuse-imm llm-tool-loop build-eval-harness build-a-benchmark publish-a-card

llm-tool-loop and build-eval-harness reappear verbatim in agent-engineering builds; run-ekf-on-real-logs and fuse-imm are the entry ticket to visual-inertial odometry; build-a-benchmark recurs in every Forge, because publishing is the last rung of all of them.

Theory bridges

Everything used here has a from-zero lesson on this site: Kalman Filter · EKF · HMM · IMM · NEES/NIS consistency debugging · error-state KF (the gyro-bias production fix) · harness engineering for the M7 layer. The Fusion & Estimation path walks the full theory ladder this expedition practices.

What’s next

Scale the same skeleton: a full comma2k19 chunk (10 GB, regimes get statistically interesting), EuRoC MAV for 6-DoF with a real ESKF, or the GPU expeditions when they land — the vLLM serving Forge reuses your eval-harness brick on day one.