The whole machinery of supervised learning in one sitting: fit a function from examples so it works on data it has never seen. Train/test, loss, gradient descent, the overfitting U-curve, and why a robot's data is the part that breaks.
Suppose you are teaching a robot car to steer. You record an expert driving one parking lot a thousand times and you train a network until it reproduces every one of those thousand frames perfectly. On the recorded lot, it is flawless — zero error. You are thrilled. Then you drive it to a lot it has never seen, and it plows into a curb.
What went wrong? The network did not learn to steer. It learned to memorize the answers for the exact frames you showed it. A student who memorizes the answer key aces that one exam and fails the next. The whole goal of machine learning is to avoid being that student.
So the experiment that matters is never "how well does it fit what it has seen?" It is "how well does it do on data it has never seen?" To even measure that, we split our data in two: a training set the model gets to learn from, and a test set we lock in a drawer and only peek at to judge it. A model can drive its training error to zero and still have enormous test error. The gap between the two is the entire drama.
The teal dots are training data; the faint purple dots are unseen test data drawn from the same underlying curve. Slide model complexity right. A flexible-enough curve threads every training dot exactly — train error hits zero — yet it wiggles wildly between them and misses the test dots. That is memorizing, not learning.
Watch the readout: as the curve gets more flexible, the training error keeps falling toward zero, but the test error first drops and then shoots back up. The best model is not the one that fits the training data best. It is the one that does best on the data it has never seen.
Let's make the goal precise. Supervised learning is: learn a function that maps an input to an output, using a set of labeled examples — inputs already paired with their correct outputs (their labels). "Supervised" because a teacher hands you the right answer for each training input.
The instructor lays the whole field on one slide as a set of ingredients — and the punchline is that these are the same no matter what model you use, from a straight line to a billion-parameter transformer:
| Ingredient | What it is | Linear-regression example |
|---|---|---|
| Training data | Labeled examples (input, target) pairs | Points (x, y) |
| Model / architecture | The function family, with knobs θ | A line: prediction = slope·x + bias |
| Loss | How wrong one prediction is | Squared error (y − prediction)² |
| Learning objective | Pick θ to minimize loss over the training set | Least squares |
| Optimization | How we actually find that θ | Closed form, or gradient descent |
| Generalization | Does the loss stay small on the test set? | Small error on held-out points |
Supervised problems split by what the output space looks like. In regression the target is a real number (or a vector of them): "what steering angle?", "how far is that obstacle?". In classification the target is one label from a fixed finite set: "dog or cat?", "is this grasp safe or unsafe?". The ingredients are identical; only the loss differs, as we will see. We will use regression as our running example because it has the cleanest math, then meet classification in Chapter 8.
We have inputs, a model, and predictions. We need one more thing before we can fit anything: a way to score how wrong a prediction is. That score is the loss.
For regression the standard choice is the squared loss: take the prediction, subtract the true target, and square the difference. On one example with true target y and prediction ŷ, the loss is:
Why square it? Two reasons. Squaring makes every error positive, so over- and under-shooting both count as bad. And it punishes a big miss far more than a small one — an error of 2 costs 4, but an error of 4 costs 16. The model is pushed hardest to fix its worst predictions.
One example's loss tells us little. We want the model to be good on average over the whole training set. So we average the per-example losses. That average has a name — the empirical risk — and minimizing it is the learning objective, called empirical risk minimization (ERM):
Empirical because it is measured on the data we happen to have, not the true unknown distribution of all possible data. Risk is just statistics-speak for expected loss. ERM says: find the knobs θ that make this average training loss as small as possible. That single sentence is the engine under every supervised learner in the course.
Say a model predicts steering angles and on three held-out turns it predicts (0.5, −0.2, 0.9) while the truths were (0.4, 0.0, 1.0). The per-example squared losses are (0.5−0.4)² = 0.01, (−0.2−0.0)² = 0.04, and (0.9−1.0)² = 0.01. The empirical risk is their average: (0.01 + 0.04 + 0.01) / 3 = 0.06 / 3 = 0.02. That single number, the mean squared error, is what we drive downward.
Now the simplest real model: a line. We predict the target as a weighted sum of the input features. In the one-feature case that is just prediction = slope·x + bias. The instructor folds the bias into the weights with a clean trick: glue a constant dummy feature of 1 onto every input. Then "slope·x + bias" becomes a single dot product of a weight vector with the augmented input — one tidy expression that handles any number of features.
The beautiful thing about a line under squared loss is that you do not need to search for the best weights iteratively. There is a closed-form solution — an exact formula. Stack all the (augmented) inputs into a matrix we'll call X, one row per example, and stack the targets into a vector y. The least-squares weights are:
These are the normal equations. Read them as: XᵀX measures how the features correlate with each other, Xᵀy measures how each feature correlates with the target, and the inverse balances them out into the best linear fit. You will solve them by hand in a moment — it is just a 2×2 system.
Take three points: (1, 2), (2, 2), (3, 4). With a dummy 1 column, each augmented input is [x, 1]. Then XᵀX collects the sums. The top-left entry is the sum of x² = 1 + 4 + 9 = 14. The off-diagonal is the sum of x = 1 + 2 + 3 = 6. The bottom-right is the count = 3. So:
where sum of x·y = 1·2 + 2·2 + 3·4 = 2 + 4 + 12 = 18 and sum of y = 2 + 2 + 4 = 8. Now solve the two equations 14·slope + 6·bias = 18 and 6·slope + 3·bias = 8. From the second, bias = (8 − 6·slope)/3. Substitute into the first: 14·slope + 2·(8 − 6·slope) = 18, so 14·slope + 16 − 12·slope = 18, giving 2·slope = 2, so slope = 1 and bias = (8 − 6)/3 = 0.667. The best-fit line is prediction = x + 0.667.
The teal dots are data. Slide the line's slope and bias. The shaded squares are each point's squared error — literally the area being minimized. Press Best fit to snap to the normal-equations solution and see those areas shrink to their minimum total.
Let's compute that solution in code, on a slightly bigger dataset, and — crucially — report the error on a held-out test set, not just the training set. That is the first time you'll measure generalization with your own hands.
A line has a closed-form answer. Almost nothing else does — a neural network has millions of weights and no formula for the best ones. For everything beyond a line we need a general way to walk toward the minimum. That walk is gradient descent.
The picture is a ball on a hilly landscape, where height is the loss and your position is the parameters. The gradient is the direction of steepest uphill. So to go down, step in the opposite direction. Repeat. That is the entire algorithm:
The learning rate (the instructor's symbol is the Greek letter eta) is the size of each step — how far you move downhill before re-checking the slope. It is the single most important knob in all of deep learning, and getting it wrong is the most common way to fail.
Take the toy loss = (parameter)², a simple bowl with its minimum at zero. Calculus gives its slope at any point as 2×parameter. Start the parameter at 3 with a learning rate of 0.1. The gradient is 2×3 = 6. The update is: new parameter = 3 − 0.1×6 = 3 − 0.6 = 2.4. We moved from 3 toward 0 — downhill, as promised. Step again: gradient = 2×2.4 = 4.8, new = 2.4 − 0.48 = 1.92. Each step shrinks the parameter by a factor of (1 − 0.2) = 0.8, marching smoothly toward the minimum.
The instructor's rules of thumb: normalize the loss by the number of examples, start with a large step size, and whenever the validation error stops improving, halve the learning rate. Stop when it makes no further progress — that is early stopping. And because computing the gradient over the entire training set is prohibitively expensive when the dataset is huge, we use stochastic gradient descent (SGD): estimate the gradient from a small random mini-batch at each step. It is noisier but vastly cheaper, streams one batch at a time, and rides fast vectorized GPU math — the workhorse of all modern training.
A warm ball rolls down the loss bowl by gradient descent. Slide the learning rate. Too small: it crawls and never arrives in time. Just right: it glides to the bottom. Too large: it overshoots, the bounces grow, and it diverges off the curve. Press Run after each change.
Now build the loop yourself: gradient descent on the line-fitting loss from Chapter 3. You'll watch the loss fall step by step — and if you nudge the learning rate too high, watch it blow up exactly as the hand calculation predicted.
Back to the villain from Chapter 0, now named precisely. As we make a model more flexible — more features, a higher-degree polynomial, more layers — two failure modes sit at the two ends.
Underfitting is when the model is too simple to capture the pattern. Its training error and test error are both high, and they sit close together. The instructor's example: fitting a straight line to data that genuinely curves. The model just can't bend.
Overfitting is the opposite. The model is so flexible it fits the training data's noise, not just its signal. The tell-tale sign: training error is tiny but test error is far larger. The model has memorized the training set, exactly as in Chapter 0.
| Train error | Test error | Diagnosis | |
|---|---|---|---|
| Underfit | high | high (close to train) | Model too simple — add capacity |
| Just right | low | low | Generalizes — keep it |
| Overfit | ~0 | high (≫ train) | Too flexible — regularize or get more data |
Slide complexity (polynomial degree). The teal curve is training error — it only ever falls. The purple curve is test error — it falls, bottoms out, then climbs. The minimum of the purple curve is the model you actually want. Left of it: underfit. Right of it: overfit.
Here is the catch. To pick the right complexity (or tune any knob the optimizer doesn't set itself — a hyperparameter like the learning rate or polynomial degree), we'd love to read off the test error. But we promised to lock the test set in a drawer; peeking at it to make choices would contaminate our final, honest estimate.
The U-curve is a fact; this chapter is the explanation for it. Imagine training your model not once but many times, each on a fresh training set drawn from the same world. Your learned parameters depend on which dataset you happened to get, so your predictions wobble from run to run. The instructor decomposes the expected test error (under squared loss) into two parts that add up:
Bias is how far the model's average prediction (averaged over all those retrainings) sits from the truth. It's the error baked in by the model being too rigid to represent the real pattern. Variance is how much the prediction jumps around as the training set changes — the error from the model being so flexible it chases each dataset's particular noise.
Two more facts the instructor stresses, both intuitive once you have the picture. Variance shrinks with more training data — more examples wash out the noise, so the model wobbles less between datasets. That's why "get more data" cures overfitting. And variance grows with model complexity — more knobs means more ways to chase noise.
Each faint line is the model fit to a different random training set; the warm line is their average and the dashed teal is the truth. Slide complexity: a simple model gives tight lines far from the truth (low variance, high bias); a complex one gives a wild spray that averages near the truth (high variance, low bias).
Suppose you need a flexible model but you're drowning in variance. One cure is more data (Chapter 6). The other, when data is fixed, is regularization: gently discourage the model from using its full flexibility, by adding a penalty on the weights to the loss.
The penalty is an inductive bias — a preference baked into training for "simpler" weights before any data is even seen. The knob λ (lambda) sets how hard we push: λ = 0 is plain ERM; large λ flattens the model toward a constant. The instructor names the two classic penalties:
Ridge has a clean closed form: it just adds λ to the diagonal of XᵀX before inverting. That tiny addition is also what rescues the normal equations when XᵀX is not invertible (too many features, too few examples) — the diagonal bump makes it solvable. Let's see ridge crush an overfit polynomial.
So far the loss looked like an arbitrary choice. The instructor reveals it isn't: there is a deeper story where the loss falls out of a probability model. This is the probabilistic approach to supervised learning, and it both justifies our losses and unlocks classification.
The idea: instead of predicting a bare number, the model predicts a probability distribution over outputs given the input — "given these pixels, here's how likely each label is." Training then asks: which weights make the actually-observed training labels most probable? That principle is maximum likelihood estimation (MLE): choose θ to maximize the probability the model assigns to the training labels (equivalently, to maximize the log of that probability, which is easier and turns products into sums).
Now classification. We want a yes/no answer, but a line outputs any real number. We squash that raw score through the sigmoid (logistic) function, which maps any number to a probability between 0 and 1: large positive scores go near 1, large negative near 0, and 0 maps to exactly one-half.
So logistic regression predicts the probability of the positive class as sigmoid(line score). To train it, MLE again: maximize the log-likelihood of the observed labels. That objective has a name — the logistic loss, also called binary cross-entropy or log loss. Worked feel for the sigmoid: a raw score of 0 gives probability exactly 0.5 (maximum uncertainty); a score of +2 gives 1/(1 + e−2) = 1/(1 + 0.135) ≈ 0.88 (confident yes); a score of −2 gives ≈ 0.12 (confident no). For more than two classes, the sigmoid generalizes to the softmax, and the loss becomes the multi-class cross-entropy.
The curve is the sigmoid. Slide the raw score from the linear part; the dot shows the output probability. Notice it is steepest near 0 (decisions are most sensitive there) and saturates flat at the extremes (very confident, barely changing) — the flat tails are exactly the "vanishing gradient" trap Part 2 will revisit.
You now own the entire supervised-learning pipeline — the foundation every method in 16-831 stands on, whether the label is a class, a dynamics step, or an expert action. Here is the whole flow on one diagram, then the vocabulary to carry forward.
This is Part 1 of the refresher — the classical core that holds for any model. Part 2 takes the exact same pipeline (data, model, loss, objective, optimization) and swaps in deep models: the artificial neuron, fully-connected networks, why nonlinearities matter, backpropagation, vanishing gradients, and the tricks that make deep nets trainable.
Continue the series: Lecture 4 — ML / DL Refresher, Part 2 (deep networks & backprop), or step back to Lecture 2 — Robot Learning: An Overview. To go deeper on the threads in this lesson:
"Don't get confused by fancy names — ask yourself what exactly makes this method work." — Guanya Shi, Lecture 3