Introduction to Robot Learning · Lecture 3 of 25 · CMU 16-831

ML / DL Refresher, Part 1

The whole machinery of supervised learning in one sitting: fit a function from examples so it works on data it has never seen. Train/test, loss, gradient descent, the overfitting U-curve, and why a robot's data is the part that breaks.

Prerequisites: basic vectors + a little Python. No calculus beyond "slope of a curve." Lectures 1–2 help but aren't required.
10
Chapters
5
Simulations
3
Code Labs

Chapter 0: The Model That Memorized the Test

Suppose you are teaching a robot car to steer. You record an expert driving one parking lot a thousand times and you train a network until it reproduces every one of those thousand frames perfectly. On the recorded lot, it is flawless — zero error. You are thrilled. Then you drive it to a lot it has never seen, and it plows into a curb.

What went wrong? The network did not learn to steer. It learned to memorize the answers for the exact frames you showed it. A student who memorizes the answer key aces that one exam and fails the next. The whole goal of machine learning is to avoid being that student.

The one word the whole lecture turns on: generalization. Generalization means doing well on data you did not train on. We never actually care about performance on the training examples — we already know their answers. We care about the test data: the future, the new parking lot, the road we will actually drive. Every idea in this lesson exists to make a model that generalizes.

So the experiment that matters is never "how well does it fit what it has seen?" It is "how well does it do on data it has never seen?" To even measure that, we split our data in two: a training set the model gets to learn from, and a test set we lock in a drawer and only peek at to judge it. A model can drive its training error to zero and still have enormous test error. The gap between the two is the entire drama.

Memorize vs. learn — slide the model complexity

The teal dots are training data; the faint purple dots are unseen test data drawn from the same underlying curve. Slide model complexity right. A flexible-enough curve threads every training dot exactly — train error hits zero — yet it wiggles wildly between them and misses the test dots. That is memorizing, not learning.

Model complexity (degree) 1
Ready.

Watch the readout: as the curve gets more flexible, the training error keeps falling toward zero, but the test error first drops and then shoots back up. The best model is not the one that fits the training data best. It is the one that does best on the data it has never seen.

Why this lesson exists. This is the machine-learning refresher for a robot-learning course, and the punchline appears right at the end of the lecture: in robotics your label can be the world's dynamics (for control) or an expert's action (for imitation), and how you generate that data and how you use it matters even more than which model you pick. But before any of that, you need the bedrock: how do we fit a function from examples so it generalizes? That bedrock is everything below.

Your steering network reaches exactly zero error on its recorded training frames but crashes on a new lot. What is the right diagnosis?

Chapter 1: The Supervised-Learning Setup

Let's make the goal precise. Supervised learning is: learn a function that maps an input to an output, using a set of labeled examples — inputs already paired with their correct outputs (their labels). "Supervised" because a teacher hands you the right answer for each training input.

Input x
What we observe. An image (pixels), a sensor reading, a robot's state vector. Lives in the input space.
↓ feed into the model…
Model f(x | θ)
A function with tunable knobs θ (theta). Same shape of input in, prediction out. θ is what we fit.
↓ produces…
Prediction ŷ
The model's guess for the target. Lives in the output space.
↓ compare to the true label y from the dataset; the mismatch trains θ

The instructor lays the whole field on one slide as a set of ingredients — and the punchline is that these are the same no matter what model you use, from a straight line to a billion-parameter transformer:

IngredientWhat it isLinear-regression example
Training dataLabeled examples (input, target) pairsPoints (x, y)
Model / architectureThe function family, with knobs θA line: prediction = slope·x + bias
LossHow wrong one prediction isSquared error (y − prediction)²
Learning objectivePick θ to minimize loss over the training setLeast squares
OptimizationHow we actually find that θClosed form, or gradient descent
GeneralizationDoes the loss stay small on the test set?Small error on held-out points

Two flavors of output

Supervised problems split by what the output space looks like. In regression the target is a real number (or a vector of them): "what steering angle?", "how far is that obstacle?". In classification the target is one label from a fixed finite set: "dog or cat?", "is this grasp safe or unsafe?". The ingredients are identical; only the loss differs, as we will see. We will use regression as our running example because it has the cleanest math, then meet classification in Chapter 8.

Why is this even hard? The instructor's framing: what a human sees in an image and what a computer sees (a raw grid of pixel brightnesses) are worlds apart. Learning exists precisely to extract the pattern from complex, raw data — to find the function hiding inside the examples. If a human could write that function by hand, we wouldn't need to learn it.

A robot predicts the distance to an obstacle as a real number in meters. Which kind of supervised problem is this, and which ingredient distinguishes it from "is the path blocked: yes/no"?

Chapter 2: Loss and Empirical Risk Minimization

We have inputs, a model, and predictions. We need one more thing before we can fit anything: a way to score how wrong a prediction is. That score is the loss.

For regression the standard choice is the squared loss: take the prediction, subtract the true target, and square the difference. On one example with true target y and prediction ŷ, the loss is:

loss(y, ŷ) = (y − ŷ)²

Why square it? Two reasons. Squaring makes every error positive, so over- and under-shooting both count as bad. And it punishes a big miss far more than a small one — an error of 2 costs 4, but an error of 4 costs 16. The model is pushed hardest to fix its worst predictions.

One example's loss tells us little. We want the model to be good on average over the whole training set. So we average the per-example losses. That average has a name — the empirical risk — and minimizing it is the learning objective, called empirical risk minimization (ERM):

empirical risk = average over training examples of loss(y, model prediction)

Empirical because it is measured on the data we happen to have, not the true unknown distribution of all possible data. Risk is just statistics-speak for expected loss. ERM says: find the knobs θ that make this average training loss as small as possible. That single sentence is the engine under every supervised learner in the course.

The quiet leap of faith. ERM minimizes loss on the training set, but we actually care about the test set (Chapter 0). The whole reason machine learning works at all is the bet that a low training risk usually buys a low test risk — as long as the model isn't flexible enough to memorize. Everything in Chapters 5–7 is about keeping that bet honest.

A worked number

Say a model predicts steering angles and on three held-out turns it predicts (0.5, −0.2, 0.9) while the truths were (0.4, 0.0, 1.0). The per-example squared losses are (0.5−0.4)² = 0.01, (−0.2−0.0)² = 0.04, and (0.9−1.0)² = 0.01. The empirical risk is their average: (0.01 + 0.04 + 0.01) / 3 = 0.06 / 3 = 0.02. That single number, the mean squared error, is what we drive downward.

Two models miss one target. Model A is off by 1.0; Model B is off by 2.0. Under squared loss, how much worse is B's loss on that example than A's?

Chapter 3: Linear Regression and the Normal Equations

Now the simplest real model: a line. We predict the target as a weighted sum of the input features. In the one-feature case that is just prediction = slope·x + bias. The instructor folds the bias into the weights with a clean trick: glue a constant dummy feature of 1 onto every input. Then "slope·x + bias" becomes a single dot product of a weight vector with the augmented input — one tidy expression that handles any number of features.

The beautiful thing about a line under squared loss is that you do not need to search for the best weights iteratively. There is a closed-form solution — an exact formula. Stack all the (augmented) inputs into a matrix we'll call X, one row per example, and stack the targets into a vector y. The least-squares weights are:

θ = (Xᵀ X)⁻¹ Xᵀ y

These are the normal equations. Read them as: XᵀX measures how the features correlate with each other, Xᵀy measures how each feature correlates with the target, and the inverse balances them out into the best linear fit. You will solve them by hand in a moment — it is just a 2×2 system.

Solve a 2×2 normal-equations system by hand

Take three points: (1, 2), (2, 2), (3, 4). With a dummy 1 column, each augmented input is [x, 1]. Then XᵀX collects the sums. The top-left entry is the sum of x² = 1 + 4 + 9 = 14. The off-diagonal is the sum of x = 1 + 2 + 3 = 6. The bottom-right is the count = 3. So:

XᵀX = [ 14  6 ; 6  3 ]    Xᵀy = [ sum of x·y ; sum of y ] = [ 18 ; 8 ]

where sum of x·y = 1·2 + 2·2 + 3·4 = 2 + 4 + 12 = 18 and sum of y = 2 + 2 + 4 = 8. Now solve the two equations 14·slope + 6·bias = 18 and 6·slope + 3·bias = 8. From the second, bias = (8 − 6·slope)/3. Substitute into the first: 14·slope + 2·(8 − 6·slope) = 18, so 14·slope + 16 − 12·slope = 18, giving 2·slope = 2, so slope = 1 and bias = (8 − 6)/3 = 0.667. The best-fit line is prediction = x + 0.667.

Sanity check. At x = 1, 2, 3 the line predicts 1.67, 2.67, 3.67 against truths 2, 2, 4 — residuals of +0.33, −0.67, +0.33. They sum to zero, which is exactly what least squares guarantees when a bias term is present: the line passes through the "center of mass" of the data. That zero-sum is your free correctness check.

Drag the line, watch the squared loss

The teal dots are data. Slide the line's slope and bias. The shaded squares are each point's squared error — literally the area being minimized. Press Best fit to snap to the normal-equations solution and see those areas shrink to their minimum total.

Slope0.30
Bias0.50
Mean squared error: …

Let's compute that solution in code, on a slightly bigger dataset, and — crucially — report the error on a held-out test set, not just the training set. That is the first time you'll measure generalization with your own hands.

For the points (1,2), (2,2), (3,4) we found slope = 1, bias = 0.667 by solving the normal equations. What does the term XᵀX capture?

Chapter 4: Gradient Descent and the Learning Rate

A line has a closed-form answer. Almost nothing else does — a neural network has millions of weights and no formula for the best ones. For everything beyond a line we need a general way to walk toward the minimum. That walk is gradient descent.

The picture is a ball on a hilly landscape, where height is the loss and your position is the parameters. The gradient is the direction of steepest uphill. So to go down, step in the opposite direction. Repeat. That is the entire algorithm:

θ ← θ − (learning rate) · gradient of the loss at θ

The learning rate (the instructor's symbol is the Greek letter eta) is the size of each step — how far you move downhill before re-checking the slope. It is the single most important knob in all of deep learning, and getting it wrong is the most common way to fail.

One gradient-descent step by hand

Take the toy loss = (parameter)², a simple bowl with its minimum at zero. Calculus gives its slope at any point as 2×parameter. Start the parameter at 3 with a learning rate of 0.1. The gradient is 2×3 = 6. The update is: new parameter = 3 − 0.1×6 = 3 − 0.6 = 2.4. We moved from 3 toward 0 — downhill, as promised. Step again: gradient = 2×2.4 = 4.8, new = 2.4 − 0.48 = 1.92. Each step shrinks the parameter by a factor of (1 − 0.2) = 0.8, marching smoothly toward the minimum.

The learning-rate cliff. Now repeat with a learning rate of 1.1 instead of 0.1. From 3: gradient 6, new = 3 − 1.1×6 = 3 − 6.6 = −3.6. We overshot the minimum and landed farther away, on the other side. Next step jumps to +4.32, then −5.18 — the parameter oscillates and diverges, the loss exploding instead of shrinking. Too-big a step turns the descent into a launch.

The instructor's rules of thumb: normalize the loss by the number of examples, start with a large step size, and whenever the validation error stops improving, halve the learning rate. Stop when it makes no further progress — that is early stopping. And because computing the gradient over the entire training set is prohibitively expensive when the dataset is huge, we use stochastic gradient descent (SGD): estimate the gradient from a small random mini-batch at each step. It is noisier but vastly cheaper, streams one batch at a time, and rides fast vectorized GPU math — the workhorse of all modern training.

The ball on the loss curve — find the learning-rate sweet spot

A warm ball rolls down the loss bowl by gradient descent. Slide the learning rate. Too small: it crawls and never arrives in time. Just right: it glides to the bottom. Too large: it overshoots, the bounces grow, and it diverges off the curve. Press Run after each change.

Learning rate0.10
Ready.

Now build the loop yourself: gradient descent on the line-fitting loss from Chapter 3. You'll watch the loss fall step by step — and if you nudge the learning rate too high, watch it blow up exactly as the hand calculation predicted.

On loss = parameter² (gradient 2×parameter), you start at 3 with learning rate 1.1. The first step lands at −3.6, the next at +4.32. What is happening, and why?

Chapter 5: Overfitting, Underfitting, and the Validation Set

Back to the villain from Chapter 0, now named precisely. As we make a model more flexible — more features, a higher-degree polynomial, more layers — two failure modes sit at the two ends.

Underfitting is when the model is too simple to capture the pattern. Its training error and test error are both high, and they sit close together. The instructor's example: fitting a straight line to data that genuinely curves. The model just can't bend.

Overfitting is the opposite. The model is so flexible it fits the training data's noise, not just its signal. The tell-tale sign: training error is tiny but test error is far larger. The model has memorized the training set, exactly as in Chapter 0.

Train errorTest errorDiagnosis
Underfithighhigh (close to train)Model too simple — add capacity
Just rightlowlowGeneralizes — keep it
Overfit~0high (≫ train)Too flexible — regularize or get more data
The generalization U-curve — sweep the model complexity

Slide complexity (polynomial degree). The teal curve is training error — it only ever falls. The purple curve is test error — it falls, bottoms out, then climbs. The minimum of the purple curve is the model you actually want. Left of it: underfit. Right of it: overfit.

Complexity (degree)1

But we never see the test set — so how do we choose?

Here is the catch. To pick the right complexity (or tune any knob the optimizer doesn't set itself — a hyperparameter like the learning rate or polynomial degree), we'd love to read off the test error. But we promised to lock the test set in a drawer; peeking at it to make choices would contaminate our final, honest estimate.

The fix: a validation set. Carve a third slice out of the training data — the validation set. Train on what's left, then judge each candidate model on the validation set to estimate its test error. Pick the winner, and only then unlock the real test set once for the final score. The instructor states the governing principle bluntly, quoting Vinyals: "test and train conditions must match." When data is scarce, rotate the slice with k-fold cross-validation: split into k parts, validate on each in turn, average the scores.

A model has 0.001 training error but 0.8 test error. What is the diagnosis and a valid fix?

Chapter 6: Why It Over/Underfits — Bias and Variance

The U-curve is a fact; this chapter is the explanation for it. Imagine training your model not once but many times, each on a fresh training set drawn from the same world. Your learned parameters depend on which dataset you happened to get, so your predictions wobble from run to run. The instructor decomposes the expected test error (under squared loss) into two parts that add up:

expected test error = bias² + variance  (+ irreducible noise)

Bias is how far the model's average prediction (averaged over all those retrainings) sits from the truth. It's the error baked in by the model being too rigid to represent the real pattern. Variance is how much the prediction jumps around as the training set changes — the error from the model being so flexible it chases each dataset's particular noise.

The two failure modes, re-explained. Underfitting is high bias: a too-simple model is wrong on average no matter how much data you give it — the extreme case, a constant prediction, has zero variance but huge bias. Overfitting is high variance: a too-flexible model fits each training set's noise differently, so its predictions swing wildly. The U-curve is bias falling and variance rising as complexity grows; their sum bottoms out in the middle.

Two more facts the instructor stresses, both intuitive once you have the picture. Variance shrinks with more training data — more examples wash out the noise, so the model wobbles less between datasets. That's why "get more data" cures overfitting. And variance grows with model complexity — more knobs means more ways to chase noise.

Bias vs. variance — retrain on many datasets, watch the fits scatter

Each faint line is the model fit to a different random training set; the warm line is their average and the dashed teal is the truth. Slide complexity: a simple model gives tight lines far from the truth (low variance, high bias); a complex one gives a wild spray that averages near the truth (high variance, low bias).

Complexity (degree)1
A modern twist worth knowing. The instructor flags that this clean tradeoff can break for very large, over-parameterized models — in deep learning, networks with far more parameters than data points often generalize better, not worse. For decades over-parameterization was considered a mistake; in practice it works, and there are now partial explanations (the implicit regularization of SGD). The bias–variance picture is your foundation, not the final word.

A model that always predicts the constant 0, no matter the input, has which bias/variance profile?

Chapter 7: Regularization — Taming a Flexible Model

Suppose you need a flexible model but you're drowning in variance. One cure is more data (Chapter 6). The other, when data is fixed, is regularization: gently discourage the model from using its full flexibility, by adding a penalty on the weights to the loss.

objective = empirical risk + λ · penalty(θ)

The penalty is an inductive bias — a preference baked into training for "simpler" weights before any data is even seen. The knob λ (lambda) sets how hard we push: λ = 0 is plain ERM; large λ flattens the model toward a constant. The instructor names the two classic penalties:

Why this matters for robots. The instructor explicitly calls regularization "useful for robot learning," pointing to smooth dynamics learning and smooth imitation learning: a penalty can encode that a robot's motion should be smooth, not jerky. Regularization is how you inject physical prior knowledge — "I expect the answer to be gentle" — directly into the objective.

Ridge has a clean closed form: it just adds λ to the diagonal of XᵀX before inverting. That tiny addition is also what rescues the normal equations when XᵀX is not invertible (too many features, too few examples) — the diagonal bump makes it solvable. Let's see ridge crush an overfit polynomial.

You add a ridge penalty (sum of squared weights) with a large λ. What happens to the fitted model?

Chapter 8: The Probabilistic View — MLE and Logistic Regression

So far the loss looked like an arbitrary choice. The instructor reveals it isn't: there is a deeper story where the loss falls out of a probability model. This is the probabilistic approach to supervised learning, and it both justifies our losses and unlocks classification.

The idea: instead of predicting a bare number, the model predicts a probability distribution over outputs given the input — "given these pixels, here's how likely each label is." Training then asks: which weights make the actually-observed training labels most probable? That principle is maximum likelihood estimation (MLE): choose θ to maximize the probability the model assigns to the training labels (equivalently, to maximize the log of that probability, which is easier and turns products into sums).

The reveal. Squared loss is not arbitrary — it is exactly what MLE gives you if you assume the targets are the true line plus Gaussian (bell-curve) noise. Maximizing the likelihood of Gaussian-corrupted data is the same problem as minimizing squared error. Every loss in this course is a disguised log-likelihood. (Adding a prior belief on the weights turns MLE into the Bayesian view — and a Gaussian prior is exactly the ridge penalty from Chapter 7.)

Logistic regression: classification, done right

Now classification. We want a yes/no answer, but a line outputs any real number. We squash that raw score through the sigmoid (logistic) function, which maps any number to a probability between 0 and 1: large positive scores go near 1, large negative near 0, and 0 maps to exactly one-half.

sigmoid(s) = 1 / (1 + e−s)

So logistic regression predicts the probability of the positive class as sigmoid(line score). To train it, MLE again: maximize the log-likelihood of the observed labels. That objective has a name — the logistic loss, also called binary cross-entropy or log loss. Worked feel for the sigmoid: a raw score of 0 gives probability exactly 0.5 (maximum uncertainty); a score of +2 gives 1/(1 + e−2) = 1/(1 + 0.135) ≈ 0.88 (confident yes); a score of −2 gives ≈ 0.12 (confident no). For more than two classes, the sigmoid generalizes to the softmax, and the loss becomes the multi-class cross-entropy.

The sigmoid — turning a raw score into a probability

The curve is the sigmoid. Slide the raw score from the linear part; the dot shows the output probability. Notice it is steepest near 0 (decisions are most sensitive there) and saturates flat at the extremes (very confident, barely changing) — the flat tails are exactly the "vanishing gradient" trap Part 2 will revisit.

Raw score s0.0
Tie back to robots. The lecture closes the supervised-learning half with the one slide that makes it a robot-learning lecture: the target you regress or classify can be the world's dynamics (predict the next state — for model-based control and RL) or an expert's action (predict what to do — imitation learning). Same pipeline, robotic labels. As the instructor warns: how you generate and use that data matters more than the model. That is the bridge from this refresher to the rest of 16-831.

Why pass a linear model's raw score through a sigmoid for binary classification, and what training objective do we then use?

Chapter 9: Connections & Cheat Sheet

You now own the entire supervised-learning pipeline — the foundation every method in 16-831 stands on, whether the label is a class, a dynamics step, or an expert action. Here is the whole flow on one diagram, then the vocabulary to carry forward.

Data & ingredients
Labeled (input, target) pairs; choose a model family, a loss, an objective.
↓ minimize average loss (ERM)
Fit θ
Closed form (normal equations) for a line; gradient descent / SGD for everything else.
↓ but don't memorize — check on held-out data
Generalize
Validation set picks complexity; regularization tames variance; watch the train/test gap.
↻ in robotics: the target is dynamics or actions → control, imitation, RL
Cheat sheet — the vocabulary you now own.
  • Supervised learning = fit a function from labeled examples that generalizes to unseen (test) data.
  • Ingredients = data, model, loss, learning objective, optimization, generalization — the same for a line or a transformer.
  • Squared loss = (truth − prediction)²; average it = empirical risk; minimize it = ERM.
  • Linear regression = fit a line; exact answer via the normal equations θ = (XᵀX)⁻¹Xᵀy.
  • Gradient descent = step θ downhill by the learning rate × gradient; too-big a rate diverges; SGD uses mini-batches.
  • Overfit = low train, high test (memorized noise); underfit = both high (too simple). A validation set picks the sweet spot.
  • Bias–variance = test error = bias² + variance; underfit is high bias, overfit is high variance; data cuts variance.
  • Regularization = penalty on weights; ridge shrinks, lasso sparsifies; encodes priors like smoothness.
  • Probabilistic view = loss = negative log-likelihood; MLE; logistic regression = sigmoid + cross-entropy for classification.

This is Part 1 of the refresher — the classical core that holds for any model. Part 2 takes the exact same pipeline (data, model, loss, objective, optimization) and swaps in deep models: the artificial neuron, fully-connected networks, why nonlinearities matter, backpropagation, vanishing gradients, and the tricks that make deep nets trainable.

Where to go next on this site

Continue the series: Lecture 4 — ML / DL Refresher, Part 2 (deep networks & backprop), or step back to Lecture 2 — Robot Learning: An Overview. To go deeper on the threads in this lesson:

The test of this lesson. Open the original Lecture 3 slides (the supervised-learning half). You should now be able to explain, from memory: what supervised learning and generalization are; the six ingredients; squared loss and ERM; the normal equations (and solve a 2×2 by hand); gradient descent and the learning-rate cliff; over/underfitting, validation, and model selection; the bias–variance decomposition; ridge vs. lasso regularization; and how MLE makes logistic regression. If you can, you're ready for Part 2's deep networks.

"Don't get confused by fancy names — ask yourself what exactly makes this method work." — Guanya Shi, Lecture 3