How a Camera Turns the World into Pixels — and Back

Chapter 1: The Pinhole Model — Similar Triangles

Long before lenses, people noticed that light passing through a tiny hole into a dark box forms an image on the far wall — upside down. This is the camera obscura, and it is the simplest possible model of a camera: the pinhole camera. There is no lens, no focus, no aperture. Just a single point that all light rays must pass through.

Set up a coordinate frame fixed to the camera. The origin is the pinhole — call it the camera center O_C. The camera looks down its optical axis, which we take to be the Z axis. The X axis points right, the Y axis points down (this is the standard computer-vision convention, and it is why images are indexed from the top-left).

Now place the image plane at distance f in front of the pinhole, where f is the focal length. (Physically the image forms behind the pinhole and is inverted; we use the equivalent virtual image plane in front, which is upright and saves us a pair of minus signs. The geometry is identical.)

Where does the ray pierce the plane? Use similar triangles. Look at the X–Z plane (a top-down slice). There is a big triangle from the pinhole out to the point at depth Z_C with height X_C, and a small similar triangle from the pinhole out to the image plane at depth f with height x. Similar triangles have equal ratios:

The Y axis works identically with its own slice:

These two lines are the pinhole camera equations. Let us name every symbol so nothing is mysterious: (X_C, Y_C, Z_C) are the point's coordinates in the camera frame (metres); Z_C is its depth (how far in front of the camera); f is the focal length (the plane's distance from the pinhole); and (x, y) are the resulting image-plane coordinates, in the same physical units as f (we fix this in Chapter 2 by switching to pixels).

Why divide by Z? The signature of perspective

The division by depth Z_C is the most important feature of the whole equation. It is what makes far things small and near things large — perspective. Double the depth and the image coordinate halves. This nonlinearity (a ratio of coordinates) is exactly why projection is not a simple matrix multiply yet, and exactly why Chapter 3 introduces homogeneous coordinates to tame it.

Worked example with real numbers

Take a focal length f = 500 (we will see in Chapter 2 these units are pixels). A point sits at P_C = (2, 1, 5) — two metres to the right, one metre down, five metres ahead. Then:

So the point lands at image coordinates (200, 100) measured from the optical axis. Now push the same point twice as far away, to (4, 2, 10) — same direction, double the distance:

Identical pixel! That is the depth ambiguity of Chapter 0, now in arithmetic: the projection depends only on the ratios X/Z and Y/Z, so any two points on the same ray collapse together.

From scratch in Python

python
def project_pinhole(P_c, f):
    """Project a 3D camera-frame point P_c=(Xc,Yc,Zc) to image coords (x,y)."""
    Xc, Yc, Zc = P_c
    if Zc <= 0:
        return None            # behind the camera: not imaged
    x = f * Xc / Zc            # similar triangles, X-Z slice
    y = f * Yc / Zc            # similar triangles, Y-Z slice
    return (x, y)

print(project_pinhole((2, 1, 5), f=500))   # (200.0, 100.0)
print(project_pinhole((4, 2, 10), f=500))  # (200.0, 100.0) -- same ray!

The Zc <= 0 guard matters: a point behind the camera has negative depth and the equations would happily produce a phantom pixel. Real cameras can't see behind themselves, so we reject it.

Now play with the model. Drag the 3D point in the side view; watch the ray swing through the pinhole and the projection slide along the image plane. Move the point deeper (increase Z) and watch its projection shrink toward the center — perspective, live.

Chapter 2: From Metric to Pixels — The Intrinsic Matrix K

The pinhole equations give image coordinates (x, y) measured in physical units (millimetres, say) from the optical axis. But a camera does not report millimetres — it reports pixel coordinates (u, v), counted from the top-left corner of the image. Two adjustments bridge the gap.

Adjustment 1: shift the origin (principal point)

The optical axis pierces the image plane at a special pixel called the principal point (c_x, c_y) — often written (u₀, v₀). It is roughly, but not exactly, the center of the image. Because pixels are counted from the corner, we shift the metric coordinate by the principal point:

Adjustment 2: scale to pixels (focal length in pixels)

The image sensor has some number of pixels per millimetre, and it can differ between the horizontal and vertical directions (non-square pixels). Call those densities k_x and k_y (pixels per unit length). Folding the pixel density into the focal length gives the focal lengths in pixels:

(Stanford writes these as α = k_xf and β = k_yf; same thing.) Now we have the full pixel map directly from the camera-frame point:

Adjustment 3 (rarely needed): skew

If the sensor's pixel axes are not perfectly perpendicular, a tiny skew term γ couples u to Y. For almost all real cameras γ ≈ 0 and you can ignore it — but it is part of the general model, so we carry it along once.

Packaging it: the intrinsic matrix K

All five numbers — f_x, f_y, c_x, c_y, γ — live in one 3×3 upper-triangular matrix, the intrinsic matrix (or camera matrix) K:

These are called intrinsic parameters because they describe the camera itself — its lens and sensor — independent of where the camera is or what it is looking at. A camera carries its K everywhere; that is exactly why we can measure K once (calibration, Chapter 7) and reuse it.

Worked example: project (2, 1, 5) all the way to a pixel

Reuse the point P_C = (2, 1, 5). Take a realistic VGA-ish camera: f_x = f_y = 500, principal point (c_x, c_y) = (320, 240) (the middle of a 640×480 image), and skew γ = 0. Step by step:

The point lands at pixel (520, 340). Notice it is right-and-down of center (320, 240), which matches the point being to the right (X>0) and below (Y>0) the optical axis. Sanity-checking the direction of the shift catches sign-convention bugs instantly.

From scratch, then the matrix form

python
import numpy as np

def project_pixels_scalar(P_c, fx, fy, cx, cy, gamma=0.0):
    Xc, Yc, Zc = P_c
    u = fx * Xc / Zc + gamma * Yc / Zc + cx
    v = fy * Yc / Zc + cy
    return (u, v)

print(project_pixels_scalar((2, 1, 5), 500, 500, 320, 240))  # (520.0, 340.0)

# Same thing with the matrix K (homogeneous; see Ch.3 for why this works)
K = np.array([[500, 0, 320],
              [0, 500, 240],
              [0, 0,   1]])
P_c = np.array([2, 1, 5])
uvw = K @ P_c              # [520*?, ...]; this is (Zc*u, Zc*v, Zc)
u, v = uvw[0]/uvw[2], uvw[1]/uvw[2]   # divide by the last entry
print(u, v)              # 520.0 340.0

The matrix form does the same arithmetic but hides the divide-by-Z inside "divide by the last entry of K @ P_c." That trick — treat the depth as a scale factor and divide it out at the end — is the homogeneous-coordinate magic we unpack next.

Play with K below. Drag the principal point and watch the whole projected image shift; change f_x and f_y independently to stretch the image. The little grid of world points re-projects live.

Chapter 3: Homogeneous Coordinates — Making Projection Linear

Here is a problem. The projection u = f_xX/Z + c_x has a division in it. Matrix multiplication only adds and multiplies — it cannot divide one coordinate by another. So projection, as written, is not a linear map, and we cannot fold a whole projection-plus-pose pipeline into a single matrix. That is a real obstacle, because we want to chain transformations cleanly. The fix is a change of coordinates so old it predates cameras: homogeneous coordinates.

The trick: add a coordinate, allow free scaling

To make a 2D point homogeneous, append a 1: (x, y) → (x, y, 1). A 3D point gains a 1 too: (X, Y, Z) → (X, Y, Z, 1). The defining rule: a homogeneous vector represents the same point after multiplying by any nonzero scalar α:

To read a homogeneous vector back to ordinary (inhomogeneous) coordinates, divide by the last entry:

And there is the division we needed — pushed to the very last step. The idea: do all the linear work with matrices, carry a scale factor along in the extra coordinate, and divide it out only at the end.

Projection becomes one clean matrix multiply

Watch how the pinhole-plus-K map becomes purely linear. Multiply the camera matrix by the homogeneous 3D point (with the bottom-right "1" of K supplying the depth):

Call the result (a, b, w). Divide by the last entry w = Z_C:

Exactly the pixel equations from Chapter 2. The matrix multiply did the linear part; the divide-by-last-entry did the perspective part. The depth Z_C appears as the homogeneous scale factor and divides itself out — which is precisely why the camera can't recover depth: that scale factor is exactly what homogeneous coordinates throw away.

See the equivalence directly below: one pixel, the whole ray of 3D points behind it, all stamped from the same homogeneous direction.

A subtle but crucial benefit: in homogeneous coordinates, rigid motions (rotation + translation) are also just matrix multiplies (Chapter 4). So the full chain — move the world into the camera frame, then project — collapses into one matrix product. That is the payoff we cash in Chapter 5.

Chapter 4: Extrinsics — Where the Camera Stands [R | t]

So far the point was given in the camera frame — coordinates relative to the camera itself. But the world doesn't revolve around the camera. A robot's map, its waypoints, the checkerboard on the wall — all of these live in a fixed world frame. Before we can project a world point, we must first express it in the camera frame. That requires knowing where the camera is: its pose.

Recall from the kinematics lessons that a rigid-body pose in 3D — an element of SE(3) — is exactly a rotation plus a translation. For the camera we need the transform that takes a point's world coordinates P_W and returns its camera coordinates P_C.

Deriving the world→camera transform

Look at the geometry. The camera-frame coordinate of the point is the vector from the camera center to the point, written in the camera's axes. By chasing the triangle (Stanford's Figure 8.3) you get:

Define every symbol: R is a 3×3 rotation matrix that re-expresses world directions in camera axes (its rows are the world axes seen from the camera, its columns the camera axes seen from the world). t is a 3-vector, the position of the world origin expressed in camera coordinates — it slides the rotated point so the arithmetic comes out in the camera frame. Together [R | t] are the camera's extrinsic parameters: where it is and which way it points.

Homogeneous form: one 4×4 matrix

Just like before, the affine map RP_W + t — a multiply plus an add — becomes a single matrix multiply in homogeneous coordinates. Append a 1 to the world point and stack R and t:

The bottom row [0 0 0 1] just copies the 1 through, keeping the point homogeneous. For projection we only need the top three rows, the 3×4 block [R | t].

Worked example: move a world point into the camera frame

Suppose the camera is rotated 90° about the world Y axis (it has turned to look along the world's +X direction) and sits 4 units back along its own view. A clean 90°-about-Y rotation matrix is:

and take t = (0, 0, 4). Take the world point P_W = (3, 1, 0). Compute RP_W row by row:

So RP_W = (0, 1, −3). Add t = (0, 0, 4):

The point now sits at depth Z_C = 1 in front of the camera, dead center horizontally (X_C=0), one unit down (Y_C=1). Positive depth means it is visible. The rotation turned the world's +X direction into the camera's view direction, exactly as intended.

python
import numpy as np

R = np.array([[ 0, 0, 1],
              [ 0, 1, 0],
              [-1, 0, 0]])
t = np.array([0, 0, 4])
P_w = np.array([3, 1, 0])

P_c = R @ P_w + t          # world -> camera frame
print(P_c)                 # [0 1 1]  -> depth Zc=1, in front of camera

# homogeneous 4x4 form does the same in one multiply
T = np.eye(4); T[:3, :3] = R; T[:3, 3] = t
P_w_h = np.array([3, 1, 0, 1])
print((T @ P_w_h)[:3])     # [0 1 1]

Chapter 5: The Full Projection — P = K[R | t]

We now have both halves. Chapter 4 maps a world point into the camera frame; Chapters 2–3 map a camera-frame point to a pixel. In homogeneous coordinates each half is a matrix, so the whole pipeline is one matrix product. Chain them:

where P_W^h = (X_W, Y_W, Z_W, 1) is the world point in homogeneous coordinates and p_h = (a, b, w) is the homogeneous pixel, recovered as (u, v) = (a/w, b/w). The product

is the camera projection matrix (also called the homography in the calibration literature, written M). It maps any homogeneous world point straight to a homogeneous pixel. Read left to right: K (intrinsics) times [R|t] (extrinsics). Intrinsics on the left, extrinsics on the right — memorize the order; swapping them is meaningless.

End-to-end worked example: world point → pixel

Use the camera from Chapter 4: R = 90° about Y, t = (0, 0, 4), and intrinsics f_x=f_y=500, (c_x, c_y)=(320, 240). Project the world point P_W = (6, 1, 0).

Step 1 — world to camera. Compute RP_W: rows of R dotted with (6,1,0) give (0·6+0·1+1·0, 0·6+1·1+0·0, −1·6+0+0) = (0, 1, −6). Add t: P_C = (0, 1, −6) + (0, 0, 4) = (0, 1, −2).

Negative depth — the point is behind the camera! It would not be imaged. This is a real, useful failure: the camera is looking along world +X, and a point at world X=6 in front of a camera backed off to X-view by only 4 units ends up behind it. Let's pick a visible point instead.

Take P_W = (−1, 1, 0). Then RP_W = (0, 1, 1), add t: P_C = (0, 1, 5). Depth Z_C = 5 > 0, visible.

Step 2 — camera to pixel. This is exactly the Chapter 2 worked example with P_C=(0,1,5):

The world point (−1, 1, 0) images at pixel (320, 340) — horizontally centered, below middle. The full P = K[R|t] did this in one multiply; we just unrolled it to see every step.

python
import numpy as np

K  = np.array([[500, 0, 320], [0, 500, 240], [0, 0, 1]])
R  = np.array([[0, 0, 1], [0, 1, 0], [-1, 0, 0]])
t  = np.array([[0], [0], [4]])
Rt = np.hstack([R, t])                  # 3x4 extrinsic block [R | t]
P  = K @ Rt                             # 3x4 projection matrix

Pw_h = np.array([-1, 1, 0, 1])         # homogeneous world point
ph   = P @ Pw_h                         # homogeneous pixel (a, b, w)
u, v = ph[0]/ph[2], ph[1]/ph[2]
print(round(u,1), round(v,1))      # 320.0 340.0

Now drive the whole thing. Below, a small 3D scene (a wireframe cube) is projected to the image. Orbit the camera around it, slide the focal length, and watch the picture re-render — the camera P updating live, every cube vertex pushed through P = K[R|t].

Chapter 6: Lens Distortion — When Straight Lines Bend

The pinhole model assumes light travels in perfectly straight lines through a single point. Real cameras use lenses, and lenses bend light — especially near the edges. The result is radial distortion: straight lines in the world image as curves, more strongly the farther they are from the image center.

Two flavors. In barrel distortion, the image magnification decreases with distance from center, so a square bulges outward like the side of a barrel — classic in wide-angle and fisheye lenses. In pincushion distortion, magnification increases with distance, so the square's edges pinch inward — common in telephoto lenses. Both are radial: they push pixels toward or away from the distortion center along the radius.

The distortion model

Model distortion as a radial scaling about the distortion center (u_cd, v_cd), which is usually the principal point. Let r² = (u − u_cd)² + (v − v_cd)² be the squared distance of an ideal pixel from that center. Stanford's one-parameter model maps the ideal pixel (u, v) to the distorted pixel (u_d, v_d) the camera actually records:

Here k is the radial distortion factor (sometimes split into k₁, k₂ for more accuracy with higher even powers of r). Read it: take the offset from center (u − u_cd), scale it by (1 + k r²), add the center back. When k > 0 the scale grows with radius → pixels pushed outward → pincushion. When k < 0 → pixels pulled inward → barrel. At the center r=0 nothing moves; distortion is purely an edge effect.

Worked example with numbers

Distortion center (320, 240), factor k = 1.0×10⁻⁶ per pixel² (a small barrel-ish positive value for illustration). An ideal pixel sits near a corner at (620, 440). Offsets from center: Δu = 300, Δv = 200. Then:

The corner pixel moves from (620, 440) to (659, 466) — pushed outward by about 47 pixels. Now check the center: a pixel at (330, 250) has offsets (10, 10), r²=200, scale 1+0.0002 — it barely moves (by ~0.002 px). Distortion is dramatic at the edges, invisible at the center. That radial signature is exactly what calibration detects.

Undistortion

Undistortion is the inverse: given a measured (distorted) pixel, recover the ideal pinhole pixel so the clean model applies. Because the forward map is a polynomial in r, the inverse has no simple closed form — it is solved numerically (iterate, or use a precomputed lookup). Once you know k (from calibration), every incoming frame can be undistorted to a clean pinhole image, and all the linear machinery of Chapters 1–5 holds exactly.

python
import numpy as np

def distort(u, v, k, ucd=320, vcd=240):
    """Map an ideal pinhole pixel to the distorted pixel the lens records."""
    du, dv = u - ucd, v - vcd
    r2 = du*du + dv*dv
    s  = 1.0 + k * r2          # radial scale; grows with distance from center
    return s*du + ucd, s*dv + vcd

print(distort(620, 440, k=1e-6))   # (659.0, 466.0)  -- corner pushed out
print(distort(330, 250, k=1e-6))   # (330.002, 250.002) -- center barely moves

See it on a grid. A regular grid of straight lines (a checkerboard) is the world; slide k and watch it bow into a barrel or pinch into a pincushion. This is exactly the pattern calibration software fits to recover k.

Chapter 7: Camera Calibration — Recovering K, R, t

Every projection so far assumed we knew K, R, t. But a fresh camera comes with no reliable numbers (manufacturer specs are approximate, and they say nothing about the principal point or distortion). Camera calibration measures these parameters from data: a set of known 3D world points whose pixel positions we can observe. The classic tool is a checkerboard — its corners have known world coordinates (the squares are 20.5 mm in Stanford's HW3) and OpenCV finds their pixels automatically.

The strategy: known correspondences → solve for P

Given n correspondences p_i ↔ P_W,i (pixel ↔ world point), each obeys p_i^h = M P_W,i^h with the unknown 3×4 projection matrix M = K[R|t]. Write M as three rows m₁, m₂, m₃. The pixel equations are:

Clear the denominators to make them linear in the unknown entries of M (this is the Direct Linear Transformation, DLT):

Each correspondence gives 2 equations; M has 12 entries but only 11 independent ones (overall scale is free, just like a homogeneous vector). So we need at least 2n ≥ 11, i.e. n ≥ 6 correspondences. Stack all the equations into one big matrix P̃ (size 2n×12) acting on the stacked entries m:

The catch, and the fix: constrained least squares

With noise, no exact solution exists, and the trivial answer m = 0 always satisfies P̃m=0 but is useless. So we constrain the scale and minimize the residual — a constrained least-squares problem:

The solution is a gem of linear algebra: m is the eigenvector of P̃^⊤P̃ with the smallest eigenvalue — equivalently, the last column of V in the SVD P̃ = UΣV^⊤ (the right-singular vector for the smallest singular value). The smallest-singular-vector direction is the one P̃ stretches least, so it comes closest to landing on zero.

Worked mini least-squares (with numbers)

The full DLT needs a computer, but the heart — "smallest-singular-vector solves min ‖Ax‖ s.t. ‖x‖=1" — fits on one screen. Take a tiny A:

We want the unit vector x that makes ‖Ax‖ as small as possible. Form A^⊤A:

This is diagonal, so its eigenvalues are simply 9 and 2, with eigenvectors (1,0) and (0,1). The smallest eigenvalue is 2, eigenvector x = (0, 1). Check it: Ax = (0, 1, 1), so ‖Ax‖² = 0+1+1 = 2. Try the other unit vector x=(1,0): Ax=(3,0,0), ‖Ax‖²=9. Indeed the smallest-eigenvalue eigenvector gives the smaller residual (2 vs 9). That is the exact mechanism the DLT uses on the giant 2n×12 matrix — same idea, bigger matrix.

python
import numpy as np

A = np.array([[3., 0.], [0., 1.], [0., 1.]])
# constrained least squares: min ||A x|| s.t. ||x||=1  ->  smallest right-singular vector
U, S, Vt = np.linalg.svd(A)
x = Vt[-1]                         # last row of V^T = smallest-singular-value direction
print(np.round(x, 3))            # [0. 1.]  (sign may flip)
print(round(np.linalg.norm(A @ x)**2, 3))   # 2.0  (the minimum residual)

Step 2: factor M back into K, R, t

Once m (hence M) is found, recover the parameters. The left 3×3 block of M equals KR, the product of an upper-triangular K and an orthonormal R — exactly an RQ factorization, which splits any matrix into (upper-triangular)×(orthogonal). That hands you K and R directly; then t = K⁻¹c₄ from M's fourth column.

A sanity check Stanford asks you to verify: the recovered skew γ should be tiny (|γ| « α) and the principal point (u₀, v₀) should land near the image center. If they don't, you have a correspondence-ordering bug — the single most common calibration mistake.

Below, run a toy calibration. The "true" camera projects a checkerboard's corners to pixels (with a little noise); your DLT-style solver recovers a projection matrix from those correspondences and re-projects. Watch the recovered crosses converge onto the measured dots as you add correspondences.

Chapter 8: Back-Projection — From Pixel to Ray

Calibration done, we know K, R, t exactly. Surely now we can take a pixel and recover the 3D point that made it? No — and understanding precisely why is the whole point of this chapter. The projection matrix M is 3×4: it maps a 4D homogeneous world point to a 3D homogeneous pixel. It is not square, so it has no inverse. The information lost in the perspective divide (Chapter 3) cannot be conjured back from a single pixel.

What we can recover is the ray: the line through the camera center along which the 3D point must lie. Start from a pixel (u, v), lift it to homogeneous (u, v, 1), and undo the intrinsics with K⁻¹:

d_C is a direction in the camera frame — the ray's heading. Every visible 3D camera-frame point on that ray has the form P_C = λ d_C for some unknown positive depth scale λ > 0. The pixel fixes the direction; λ — the depth — is exactly the homogeneous scale that projection threw away.

Worked example: back-project a pixel

Use K with f_x=f_y=500, (c_x, c_y)=(320, 240). Its inverse for a no-skew K is clean:

Back-project the pixel (520, 340) — the very pixel our Chapter 2 point landed on. Apply K⁻¹(520, 340, 1) row by row:

So the ray direction is d_C = (0.40, 0.20, 1). Now sweep the depth: at λ=5, P_C = (2, 1, 5) — exactly the original point. At λ=10, (4, 2, 10); at λ=2.5, (1, 0.5, 2.5). All on the same ray, all project back to (520, 340). The pixel told us the direction perfectly and the depth not at all.

python
import numpy as np

K = np.array([[500, 0, 320], [0, 500, 240], [0, 0, 1]])
Kinv = np.linalg.inv(K)

px = np.array([520, 340, 1])
d  = Kinv @ px                 # ray direction in camera frame
print(np.round(d, 3))        # [0.4 0.2 1. ]

for lam in [2.5, 5.0, 10.0]:    # sweep unknown depth lambda
    print(lam, np.round(lam * d, 2))   # all project back to (520,340)

How do you get the depth back? A second look.

Since one camera fundamentally can't, you add information. Stanford lists the options:

• Stereo vision. Two calibrated cameras a known distance apart see the same point; intersect the two rays (triangulation) to get the 3D point. Two eyes — literally why you have two.
• Structure from motion (SfM). One moving camera takes images from different, initially unknown positions; matching features across frames recovers both the structure and the motion. This is Lesson 9.
• Depth from focus. Refocus until the point is sharp; the thin-lens equation turns the known focus distance into depth.
• Known landmarks / priors. If you know a wall is vertical or an object's true size, one image plus that assumption fixes the scale.

Below, see depth ambiguity resolved by a second camera. Move the world point; with one camera you only know its ray; turn on the second camera and the two rays intersect to pin the point in 3D.

Chapter 9: Showcase — The Interactive Pinhole Camera

Everything we built — pinhole projection, K, extrinsics, the full P=K[R|t], and distortion — lives in one sim. A 3D scene (a wireframe cube on a ground plane, plus a tracked point) sits in the world. A movable camera images it. Orbit the camera, slide the focal length, drag the principal point, and toggle lens distortion; the rendered image updates live, and a readout shows the full (X, Y, Z) → (u, v) computation for the highlighted point.

Things to try. Increase f: the scene zooms in and fewer vertices fit — a narrower field of view, exactly as Chapter 1 promised. Drag the principal point: the whole image slides, with no change to the 3D scene — intrinsics are independent of the world. Orbit the camera behind the cube: vertices cross to negative depth and get clipped (a point behind the camera is not imaged). Turn on distortion: the straight cube edges bow, more at the image corners.

Read the panel as you go. For the tracked yellow vertex it prints its world coordinates, its camera-frame coordinates after [R|t], the metric ratios X/Z and Y/Z, and the final pixel after K (and distortion). When a vertex's camera-frame Z goes negative, the panel says "behind camera — clipped." That is the entire forward model, narrated.

Chapter 10: Connections & Cheat-Sheet

You now own the geometry that turns a 3D world into pixels and reasons back to rays. Here it is, compressed.

Cheat-sheet

Where this sits in the series

The camera model is the geometry beneath the Perception module. It feeds directly into feature detection and structure-from-motion next door:

• Lesson 7 — Sensors: LiDAR, IMU & ICP: the other perception sensors; LiDAR does measure depth directly, the complement to a camera's depth blindness.
• Lesson 9 — SfM, Features & RANSAC: the direct sequel — match features across images and use the very back-projection rays from Chapter 8 to triangulate 3D structure.
• Multi-View Geometry: epipolar geometry, the fundamental/essential matrices, and stereo, all built on this projection model.
• Visual Navigation — Image Formation: the same pinhole/intrinsics/distortion story from the visual-inertial navigation angle.

Related Engineermaxxing lessons

• Lesson 1 — Perception, Action & Maps: the autonomy stack this perception layer plugs into.
• NeRF & 3D Gaussian Splatting: modern 3D reconstruction that assumes exactly this camera model (it calibrates and back-projects rays at massive scale).
• Classical Visual-Inertial Odometry: fuses these camera rays with an IMU to estimate pose over time.