Introduction to Robot Learning · Lecture 19 of 25 · CMU 16-831

Bandits & Preference-Based Learning

The cleanest possible version of the explore–exploit problem: no states, no transitions, just actions. From regret — the right way to keep score — to the three great strategies (ε-greedy, UCB, Thompson), then to learning from which option a human preferred rather than a number, the foundation under RLHF.

Prerequisites: comfort with an average and a little probability (a mean, a variance, a coin flip). A little Python. No reinforcement learning assumed — bandits are where RL begins.
11
Chapters
5
Simulations
3
Code Labs

Chapter 0: A Row of Slot Machines

You walk into a casino and find a row of slot machines — "one-armed bandits," in the old slang. They look identical, but they are not: each one pays out at a different secret rate. One is generous, one is stingy, the rest are somewhere in between. You have a fixed evening and a fixed stack of coins. Which machine should you keep pulling?

Here is the cruel part. The only way to learn a machine's payout rate is to pull it and see what falls out. You never get to peek inside. And every coin you spend probing a machine that turns out to be bad is a coin you did not spend on the one that turns out to be good. The machines you don't pull tell you nothing at all.

This is the whole game, and it has a precise name: the multi-armed bandit problem. It is the simplest reinforcement-learning problem there is — and, as the lecture says, still has many open questions. Strip RL of everything fancy — no states, no transitions, no dynamics — and the irreducible core that remains is this single dilemma: spend your next pull on what already looks best, or on something you haven't tried enough to be sure about?

The one-sentence problem. You have several actions; each pays an unknown average reward; you only ever observe the reward of the action you actually choose; and you want as much total reward as possible over a finite number of tries. The catch that makes it hard is counterfactual: you never learn what the arms you skipped would have paid. So you must spend some pulls just to find out — that spending is the price of information.

It is not a toy. Drop the casino story and the same structure appears everywhere. Which ad do I show this visitor? — arms are ads, reward is a click. Which news topic should the feed recommend? — arms are topics, reward is a "like." Which of two treatments do I give the next patient in a trial? — arms are treatments, reward is recovery, and a wasted pull is a patient who got the worse drug. Each is the same loop: choose an action, observe only its outcome, update, repeat.

The slot machines — pull one and see what the others hide

Five machines with secret payout rates. Pull a machine and you see only that machine's reward (a coin or nothing) and its running average. The other machines stay grey question-marks — you learn nothing about an arm you don't pull. Try a few pulls. How sure are you which is best? Reveal shows the hidden truth.

Ready. All five payout rates are hidden.

Notice the discomfort already. After a handful of pulls you have a guess about each machine, but each guess rests on a tiny, noisy sample. The machine that looks best so far might just have been lucky. If you commit to it now you might be locking onto a fluke; if you keep probing you are burning coins. That tension — nothing more — is the engine of this entire lecture.

Where this is going. First we name the problem precisely and choose the right scoreboard, regret (Ch 1–2). Then three strategies climb a ladder of cleverness: ε-greedy (explore at random, Ch 3), UCB (explore where you're most uncertain — "be optimistic," Ch 4–5), and Thompson sampling (explore by gambling on your own beliefs, Ch 6). We race all three (Ch 7), generalize to contexts (Ch 8), and finally replace the reward number with a human preference — "A is better than B" — which is exactly the machinery underneath RLHF (Ch 9).

What single feature makes the bandit problem fundamentally hard — the thing that forces you to "waste" some pulls?

Chapter 1: The Multi-Armed Bandit, Made Precise

Let us turn the casino story into something we can reason about. The setup is deliberately minimal — this is the simplest reinforcement-learning problem, and we keep it that way.

The ingredients

What is not here. There is no state and there are no transitions. In a full MDP your action changes where you are, and the consequences ripple into the future. In a bandit there is no "where" — every round looks identical, every arm always has the same hidden mean, and your action affects only the reward you collect this round, never the next round's situation. Pulling arm 3 does not move you anywhere. That is exactly why the bandit isolates the explore–exploit tension in its purest form, with nothing else mixed in.

The feedback constraint — this is the crux

The defining limitation, repeated from Chapter 0 because it controls everything: the algorithm only receives feedback on the chosen arm. Pull arm 3, you observe a reward sampled around μ3. You learn nothing about μ1, μ2, μ4, μ5. This is called bandit feedback (as opposed to "full feedback," where you'd see what every arm would have paid). It is why bandit problems demand counterfactual reasoning: to act well you must reason about outcomes you will never directly observe.

How do we estimate an arm's mean? Just average.

Since μ(a) is the expected reward of arm a, the natural estimate is the sample average of the rewards we've actually gotten from a. Call the estimate Q(a) and let N(a) be the number of times we've pulled a so far:

Q(a) = (sum of rewards received from a so far) / N(a)

This is just a running mean. By the law of large numbers, as N(a) grows, Q(a) converges to the true μ(a). Two facts that will matter for everything that follows: an arm pulled many times has a Q(a) we can trust (large N, tight estimate); an arm pulled rarely has a Q(a) that might be wildly off (small N, loose estimate). The number of pulls N(a) is not bookkeeping — it is our confidence.

The running average converges — but slowly, and noisily at first

One arm with a true mean of 0.60 (dashed line). Each pull draws a noisy reward; the warm curve is the running sample-average Q(a). Slide the noise up and watch how many pulls it takes before Q(a) settles near the truth — that delay is exactly why you can't trust an arm you've barely tried.

Reward noise 0.25
Ready.
In a multi-armed bandit, how does an arm's number of pulls N(a) relate to how much we should trust its estimated value Q(a)?

Chapter 2: Regret — The Right Way to Keep Score

How do we judge a bandit algorithm? "Total reward earned" is tempting but unfair — it depends on how generous the machines happen to be. The lecture uses a cleaner scoreboard that subtracts out the luck of the draw: regret.

The idea: compare against the genie

Imagine a genie who knew every arm's true mean from the start. The genie would pull the best arm, the one with mean μ1, every single round — collecting an expected T × μ1 over T rounds. That is the unbeatable ideal, the OPT (optimal-in-hindsight) reward. Your algorithm, ignorant at first, scatters its pulls across arms while it learns and earns less. The shortfall is the regret:

Regret(T) = T · μ1 − ( μa(1) + μa(2) + … + μa(T) )

Read it term by term. The first piece, T·μ1, is what the genie earns — the best arm pulled T times. The bracket sums the expected reward of whatever arms you actually chose on rounds 1 through T. The difference is the opportunity cost of not knowing the answer from the start. Every time you pull a sub-optimal arm a, you add a little chunk μ1 − μ(a) — the per-pull gap — to your regret.

Why regret, not reward. Regret is reward measured against the best you could have done, so it cancels the difficulty of the instance. It can only go up — it never decreases — and it grows by the size of the gap each time you pick a loser. Minimizing regret is therefore exactly the same goal as maximizing reward, but stated in a way that lets us compare algorithms fairly and prove guarantees.

The magic word: sublinear

Here is the subtle, beautiful part. A learner that always pulled some fixed bad arm would rack up regret growing like c·T — linear in T, a constant loss every round forever. We want to do dramatically better. We call an algorithm no-regret (or sublinear-regret) if its average regret per round vanishes as T grows:

Regret(T) / T → 0   as  T → ∞

That happens whenever the total regret grows slower than T — for example like the square root of T, or like the logarithm of T. The lecture's examples are exactly these: Regret(T) on the order of the square-root-of-T, or the even better log-T. If your total regret is only log T after a million rounds, then your per-round mistake has shrunk to almost nothing — asymptotically the mistakes are negligible. You have, in effect, found the best arm and are mostly playing it.

The bar every good algorithm must clear. ε-greedy with a fixed ε has linear regret (it keeps exploring forever at a constant rate — we'll see why). UCB achieves regret on the order of the square-root-of-(T·log T) — sublinear, provably no-regret. Thompson sampling matches it and often wins in practice. "Sublinear regret" is the dividing line between an algorithm that eventually learns and one that wastes a fixed fraction of its pulls forever.

Linear vs sublinear regret — the shape that separates winners from losers

Cumulative regret over T rounds. The red line is a fixed-exploration learner: regret grows straight up, linearly — a constant loss every round. The teal curve grows like the square root of T, and the warm curve like log T — both bend over and flatten. Drag the slider to extend the horizon and watch the gap explode.

Horizon T 1500
Ready.
An algorithm's total regret after T rounds grows like log T. Is it "no-regret" (sublinear), and what does that mean practically?

Chapter 3: ε-Greedy — and Why Pure Greed Fails

The most obvious strategy is greedy: estimate each arm's mean by its sample average Q(a), then always pull the arm with the highest Q. Pure exploitation. It sounds reasonable. It is a trap.

Why greedy gets stuck

Suppose early on, by bad luck, the truly-best arm pays out poorly on its first pull while a mediocre arm pays well. Greedy now believes the mediocre arm is best, and — because it only ever pulls the current best — it never pulls the good arm again to discover its mistake. It has locked into a local optimum and stays there forever. On the 10-armed testbed in the lecture, the pure-greedy curve flattens early at a clearly sub-optimal reward. Greedy is not optimal, because it never gathers the information that would correct it.

The misconception to kill first. "Just pick whatever looks best so far." That is greedy, and it can fail catastrophically — one unlucky early sample and it commits to the wrong arm for all time. The lesson the bandit teaches, before anything else: you cannot do exploitation and exploration at the same time, and you must do both. Some pulls have to be spent learning, even though they look wasteful in the moment.

The fix: explore a little, on purpose

ε-greedy (epsilon-greedy) bolts a pinch of randomness onto greedy. With probability 1 − ε it exploits (pulls the current-best arm); with probability ε it explores (pulls a uniformly random arm):

pull  argmaxa Q(a)  with prob. 1 − ε;   pull a random arm  with prob. ε

That small ε guarantees every arm keeps getting tried, so Q(a) eventually converges for all arms and greedy's "lock-in" disaster can't happen. On the testbed, ε = 0.1 and ε = 0.01 both beat pure greedy comfortably — the price of a little exploration buys you out of the local-optimum trap.

The two things wrong with ε-greedy

It works, but it is crude, in two ways the next chapters will fix:

The shape of a fix. Both flaws point the same way: exploration should be strategic, concentrated on the arms we're genuinely uncertain about, and it should fade as uncertainty fades. That is precisely what UCB and Thompson sampling deliver — they make exploration adaptive instead of a blind coin flip.

ε-greedy on a 5-armed bandit — slide the explore rate

Arm 2 (teal) is best, but the agent must find it. Left (ε→0): nearly pure greed — sometimes it locks onto a worse arm and never recovers. Right (ε→1): pure exploration — it finds the winner but keeps throwing pulls away on losers. Grey = true payoff, colored = how often each arm got pulled over 600 rounds.

ε (explore rate) 0.10
Ready.
Why does a pure greedy bandit strategy (always pull the highest-Q arm, never explore) often fail?

Chapter 4: UCB — Optimism in the Face of Uncertainty

ε-greedy's sin was exploring blindly. Upper Confidence Bound (UCB) fixes it with one of the most elegant ideas in all of learning: be optimistic about the arms you haven't tried much. Don't explore at random — explore wherever you have the most to gain from being wrong in your favor.

Building the confidence interval

Start from Chapter 1's insight: an arm pulled N(a) times has an estimate Q(a) whose uncertainty shrinks as N(a) grows. We can make that precise. Suppose each arm's rewards come from a distribution with some spread σ. A concentration inequality — the lecture names Hoeffding's inequality — says: with high probability, the true mean μ(a) lies within a window around the estimate Q(a), and that window has half-width

bonus(a) = c · √( ( ln t ) / N(a) )

where t is the total number of pulls so far and c is a tunable constant controlling how much you value exploration. Don't memorize the exact form — read its shape, because the shape is the whole idea:

The decision rule: pick the highest upper bound

Now the magic. Instead of acting on the estimate Q(a), UCB acts on the optimistic top of the window — the upper confidence bound — and pulls whichever arm has the largest one:

UCB(a) = Q(a) + bonus(a),    pull  argmaxa UCB(a)

This is the principle of optimism in the face of uncertainty (OFU): when unsure about an arm, assume the best plausible truth about it. An arm gets pulled for one of two good reasons — either its estimate Q(a) is high (it really is good: exploitation), or its bonus is high (we're very unsure and it could be good: exploration). UCB fuses explore and exploit into a single number. There is no separate coin flip.

Why optimism cannot get permanently stuck. Suppose UCB keeps pulling a genuinely bad arm. Each pull shrinks that arm's bonus (N goes up), pulling its UCB down toward its true, low mean. Meanwhile the neglected good arm's bonus slowly grows (ln t rises while its N stays put), pushing its UCB up. Inevitably the good arm's optimistic bound overtakes the bad arm's, and UCB switches. Optimism is self-correcting: being wrong about an arm is the very thing that fixes the error. This is why UCB does not fall into greedy's local-optimum trap — and why it provably achieves sublinear regret, on the order of the square-root-of-(T·log T).

UCB picks the highest upper bound, not the highest estimate

Three arms. Each dot is the estimate Q(a); the bracket above it is the confidence window (its upper tip is the UCB). Pull the arm UCB chooses and watch its bracket shrink (more data). Notice UCB often picks an arm whose estimate is lower but whose window is wider — that is optimism choosing exploration. Over time every bracket narrows onto the truth (dashed).

Ready. UCB chooses the highest upper tip.
UCB pulls the arm with the highest Q(a) + bonus(a), where the bonus grows when N(a) is small. What does the bonus term accomplish?

Chapter 5: UCB by Hand

The bonus formula stays abstract until you push real counts and means through it. Let's compute UCB scores for three arms with actual data and watch optimism pick the surprising one.

The setup

We are at total time t = 100 pulls. Use the exploration constant c = 2. Three arms, with these running statistics:

ArmEstimate Q(a)Pulls N(a)
Arm A0.6080
Arm B0.5510
Arm C0.4010

Arm A looks best by raw estimate (0.60), and we've pulled it 80 times — we trust that number. Arms B and C have been tried only 10 times each, so their estimates are shaky. Which does UCB choose? Compute the bonus c · √((ln t)/N(a)) for each. Note ln(100) ≈ 4.605, so √(ln 100) ≈ 2.146.

The three bonuses

bonus(A) = 2 · √( 4.605 / 80 ) = 2 · √(0.0576) = 2 × 0.240 = 0.480
bonus(B) = 2 · √( 4.605 / 10 ) = 2 · √(0.4605) = 2 × 0.679 = 1.357
bonus(C) = 2 · √( 4.605 / 10 ) = 2 × 0.679 = 1.357

The three upper bounds

UCB(A) = 0.60 + 0.480 = 1.080
UCB(B) = 0.55 + 1.357 = 1.907
UCB(C) = 0.40 + 1.357 = 1.757
Optimism picks Arm B, not the best-looking Arm A. Even though A has the highest estimate (0.60 vs 0.55), B's tiny pull-count gives it a far bigger bonus (1.357 vs 0.480), so B's upper bound wins — 1.907. UCB is saying: "A is probably good, and I know it well; but B might be even better and I've barely checked — worth a look." Notice B beats C too (1.907 > 1.757) because they have the same bonus but B's estimate is higher. The bonus is what breaks the tie between under-explored arms in favor of the more promising one.

Now you write the rule that is UCB — the bonus and the argmax — and confirm it picks Arm B on exactly this data.

With Q = [0.60, 0.55, 0.40], N = [80, 10, 10], t = 100, c = 2, why does UCB pull Arm B rather than the higher-estimate Arm A?

Chapter 6: Thompson Sampling — Gamble on Your Own Beliefs

UCB explores by being optimistic: take the top of each confidence window. Thompson sampling (also called posterior sampling) explores a completely different, almost playful way: maintain a full belief about each arm, then act as if a single random draw from those beliefs were the truth. It is an old idea — from 1933 — that for decades had no theory, yet works extremely well in practice (the proper analysis only arrived around 2012).

A range vs a whole distribution

Here is the clean contrast the lecture draws. UCB keeps, for each arm, a range the mean probably lies in. Thompson keeps something richer: a full posterior distribution over each arm's mean — a probability curve saying "I think this arm's mean is most likely around here, but it could plausibly be anywhere under this curve." More data makes the curve narrower and sharper, just like the shrinking UCB window, but it carries the whole shape of our uncertainty, not just its width.

The procedure (it's three steps)

1. Keep a posterior per arm
A belief distribution over each arm's mean. For 0/1 rewards (click / no-click) the natural one is a Beta distribution, updated by counts of successes and failures.
↓ each round
2. Sample one number from each arm's posterior
Draw a random plausible mean for every arm — a "what-if the truth were this" guess, weighted by how strongly we believe it.
3. Pull the arm whose sample is largest
Act greedily on the sampled values, then observe the reward and update that arm's posterior.
↻ beliefs sharpen, samples cluster on the true best arm

Why does sampling explore automatically? Because the width of a posterior is the chance of a surprising draw. An arm we've barely tried has a wide, flat posterior, so its random sample is often large — it gets picked, gathering data. An arm we've pulled a thousand times has a needle-thin posterior, so its sample is essentially its true mean every time. Uncertainty buys the arm extra chances exactly in proportion to how unsure we are — no explicit bonus, no coin flip, just the geometry of the beliefs.

The Beta posterior, concretely. Suppose each pull of an arm is a success (reward 1) or failure (reward 0). Start each arm with a flat belief, written Beta(1, 1) — "could be any rate." After seeing S successes and F failures, the belief becomes Beta(1 + S, 1 + F): a bump centered near S/(S+F) that gets taller and narrower as S + F grows. To "sample the posterior" is to draw one random rate from that bump. Few pulls → wide bump → wild draws → exploration. Many pulls → sharp bump → steady draws → exploitation. The exploration tunes itself.

One sample, by hand

Two arms after some play. Arm X: 8 successes, 2 failures, so its posterior is Beta(9, 3) — a fairly confident bump near 8/10 = 0.80. Arm Y: 1 success, 1 failure, posterior Beta(2, 2) — a wide, flat hump centered at 0.50 but spreading from near 0 to near 1. On this round we draw one sample from each: say X draws 0.78 (close to its mean — it's confident) and Y draws 0.83 (a high draw from its wide belief — entirely plausible). We pull Arm Y, because its sample 0.83 beat X's 0.78. Y got chosen not because we think it's better — we don't — but because we're unsure enough about it that an optimistic draw was possible. That is exploration, emerging for free from the randomness of the draw.

Posterior beliefs sharpen with data — and that's what controls exploration

Two arms' Beta posteriors over their success-rate. Feed a reward to an arm and watch its curve grow taller and narrower — belief concentrating. The warm arm's true rate is 0.7, the teal arm's is 0.4. Early curves are wide (a random sample could pick either); after enough data they're needles, and the sampled values almost always favor the true winner.

Ready. Both start as flat Beta(1,1) beliefs.

Now implement Thompson sampling on a 3-arm Bernoulli bandit and confirm it homes in on the best arm — the proof is in the pull counts.

In Thompson sampling, why does an arm we've barely pulled get explored automatically — with no explicit bonus term?

Chapter 7: The Race — Watch Regret Accumulate

Three strategies, one bandit, side by side. This is the payoff: let ε-greedy, UCB, and Thompson sampling all attack the same 6-armed bandit, and watch their cumulative regret diverge as the rounds tick by. Regret is the scoreboard from Chapter 2 — lower is better, and the shape of each curve tells you everything.

What to look for as it runs:

The lesson in one picture. Curves that flatten (UCB, Thompson) are sublinear / no-regret — their per-round mistake vanishes, they've essentially found the best arm. The curve that stays straight (fixed ε-greedy) is linear — it pays a constant tax every round forever. This is the whole point of the chapter on regret made visible: adaptive exploration (UCB's optimism, Thompson's sampling) beats blind exploration (a fixed coin flip).

Cumulative regret: ε-greedy vs UCB vs Thompson on one 6-armed bandit

Press Race. Six arms, one clearly best. Three agents run the same horizon; each line is its cumulative regret (lower = better). Watch ε-greedy (red) keep climbing while UCB (teal) and Thompson (warm) bend over and flatten. Slide the horizon to stretch the race; the gap widens with time.

Horizon T 1200
Press Race.

Re-run it a few times. Because rewards are random, the exact ordering of UCB vs Thompson can swap from run to run — both are no-regret, both excellent. But the qualitative story is rock-solid: the two adaptive methods flatten, fixed ε-greedy does not. That gap is the difference between sublinear and linear regret, and it is why nobody ships a fixed-ε bandit in production when UCB or Thompson is a few lines away.

In the race, UCB and Thompson curves bend over and flatten while fixed ε-greedy keeps climbing in a straight line. What is this difference called?

Chapter 8: Contextual Bandits — One Step Toward RL

So far every round looked identical: the same arms, the same hidden means, every time. But most real problems hand you a clue before you choose. The visitor on your site has an age and a country; the patient has a chart; the user has a watch history. The best arm depends on that clue. This is the contextual bandit.

The one change

In a plain bandit, the reward of arm a is just r(a), and we estimate a single number Q(a) per arm. In a contextual bandit, the reward depends on a context s that you observe at the start of each round: the reward is r(s, a) and we estimate Q(s, a). Same explore–exploit machinery — ε-greedy, UCB, Thompson all carry over — but now the value is a function of both the context and the action.

The bridge to reinforcement learning. Notice how close Q(s, a) is to the action-value from RL. The context s plays the role of a statebut with one giant simplification: there are no transitions and no dynamics. Your action affects only this round's reward; it does not change what context you see next. Add transitions — let the action move you to a new state whose future rewards you must also reason about — and a contextual bandit becomes a full Markov Decision Process. Bandit → contextual bandit → MDP is the ladder of generality, and you have just climbed the first two rungs.

Practically, contextual bandits are the workhorse of real recommendation and ad systems precisely because they capture "personalize the choice to who's in front of me" without paying the full cost of planning over a future. They are also the natural home of the personalization example from lecture: each user-topic pair is a context-action, and a "like" is the reward.

What distinguishes a contextual bandit from a full Markov Decision Process (MDP)?

Chapter 9: Preference-Based Learning — the Root of RLHF

Every algorithm so far assumed a reward number falls out of each pull — a click, a payout, a 0/1 success. But the lecture's last move asks a deeper question: what if there is no reliable number? In many real applications rewards aren't directly measurable, or an absolute score is meaningless.

Why numbers fail and comparisons work

How good, on a scale of 0 to 100, was that meal? That paragraph? That robot trajectory? People are terrible at absolute scores — everyone's scale drifts, a "7" today is an "8" tomorrow. But ask "did you prefer A or B?" and people are fast, consistent, and reliable. The lecture's example: a user clicks search result #2. One reading is absolute ("#2 is good"); the far more trustworthy reading is relative ("#2 is better than #1"). Preference-based feedback is easier to obtain and often more reliable than a number.

This reshapes the bandit. Instead of pulling one arm and reading its reward, each round you present a pair of arms and learn only which one won the comparison — the dueling bandit problem (Yue, Broder, Kleinberg & Joachims). The feedback is one bit: "A beat B." No payoff scale anywhere. As the dueling-bandits paper puts it, this fits applications where absolute rewards "have no natural scale or are difficult to measure (user-perceived quality, taste of food, product attractiveness), but where pairwise comparisons are easy to make."

The regret changes meaning too. With no reward scale, "how much worse was my choice" is undefined. So dueling-bandit regret is defined by the probability the best arm would have won the comparison you actually showed — equivalently, the fraction of users who'd have preferred the best option over what you presented. Same spirit as before (cost of not knowing the best), expressed purely in comparisons. The good algorithms still achieve sublinear (logarithmic-in-T) regret.

From comparisons to scores: the Bradley-Terry model

How do scattered "A beat B" bits become a usable ranking — or, crucially, a reward number we can optimize? We posit that each item i has a hidden score si, and that the probability i beats j rises smoothly with the score gap. The standard choice is the Bradley-Terry model (a logistic in the score difference):

P( i beats j ) = σ( si − sj ) = 1 / ( 1 + e−(si − sj) )

Read it: if two items have equal scores, the gap is 0 and each wins half the time (a coin flip, σ(0) = 0.5). The bigger i's score advantage, the closer the win probability to 1. Given a pile of observed comparisons, we fit the scores si by maximum likelihood — nudge each score up or down until the model's predicted win-rates best match the comparisons we actually saw (logistic regression on score differences). Out comes a single number per item, recovered from nothing but "this beat that."

This is exactly RLHF. Train a large language model: sample two responses A and B to a prompt, ask a human which they prefer (no absolute score — humans can't reliably grade an essay 0–100, but they can pick the better of two). Collect thousands of such pairwise preferences. Fit a reward model — a network whose scalar output plays the role of the Bradley-Terry score — by maximizing the likelihood of the observed preferences. Then optimize the language model's policy to maximize that learned reward (with PPO or similar). Pairwise preferences → a Bradley-Terry reward model → a policy. The dueling-bandit idea on this slide is the conceptual seed of how every modern aligned LLM is trained.

Concept + realization: what flows where

Trace the data so you could build it. In: a set of comparisons, each a triple (winner i, loser j). Model: one learnable score si per item; predicted win-prob σ(si − sj). Loss: negative log-likelihood — for each comparison, −log σ(swinner − sloser) — summed over all comparisons. Fit: gradient descent on the scores. Out: a score vector that orders the items (and, scaled, serves as a reward). The scores are only identified up to an additive constant (shifting everyone by +5 changes no comparison), so we pin one down — exactly as RLHF reward models are defined only up to a baseline.

Now fit a Bradley-Terry model yourself: turn a table of pairwise wins into a score per item, and confirm the recovered scores order the items correctly.

RLHF trains a reward model from human pairwise preferences ("response A is better than B") using a Bradley-Terry model. Why use comparisons instead of asking humans for an absolute quality score?

Chapter 10: Cheat Sheet & Connections

You arrived knowing how to average. You leave able to balance exploration against exploitation optimally, prove an algorithm is "no-regret," and learn a reward from nothing but human preferences — the seed of RLHF. Here is the whole lecture on one page.

The strategy ladder

MethodHow it exploresRegretTuning knob
Greedynot at alllinear (can lock in)none
ε-greedyrandom arm w.p. εlinear (fixed ε)ε (decay it)
UCBoptimism: Q + bonussublinear, ~√(T log T)constant c
Thompsonsample the posteriorsublinear, great in practicethe prior
Contextual banditany of the above, on Q(s,a)sublinear
Dueling / preferencecompare pairs, fit scoressublinear (log T)

The six things to remember

The two mechanisms behind everything. The lecture frames bandits as teaching the fundamental ideas behind all of RL: the explore–exploit trade-off, and the optimism vs pessimism mechanisms. UCB is optimism (assume the best plausible truth, to encourage exploration). Its mirror — pessimism (assume the worst plausible truth) — is what makes offline RL safe, where you must not trust under-explored actions. Same confidence-interval idea, opposite sign, chosen to fit whether you can gather more data or are stuck with a fixed dataset.

Where to go next

The throughline of the course. Lecture 1 framed robot learning as sequential decision-making in the physical world. The bandit is that idea boiled to its irreducible core — one decision, repeated, under uncertainty. Master the trade-off here, with no transitions to cloud it, and every harder problem in the course (Q-learning's ε-greedy, offline RL's pessimism, RLHF's preference reward) is the same two mechanisms — optimism and posterior sampling — wearing heavier clothes.

"You can do both exploitation and exploration — but not at the same time." — from the lecture. Every algorithm you just learned is a different, beautiful answer to how to split the difference.