Your brain makes a small bump when the letter you want flashes. Add up enough flashes and the bump beats the noise, so a computer can read the one letter you chose.
Set the rounds to 2 and press Spell it. Watch the evidence bars grow. Then set 10 rounds and spell again. Then we build, piece by piece, why adding flashes works and where it breaks.
You need the idea of a voltage (a push that moves charge) and the idea of an average. We build the rest from zero.
Part of the series Brain-Computer Interfaces, from the spike to the foundation model. Grounded in Farwell and Donchin, “Talking off the top of your head” (1988), and the sources listed in the Field Guide.
Toy model, not a recording. Each flash gives the classifier a score: a mean of 1 if the flash holds your letter, 0 if not, with a spread of 1.2. One round flashes all 6 rows and 6 columns once. The animation runs faster than real time; the read-out counts 175 ms per flash. The accuracy is measured live over 4,000 simulated letters.
Chapter 0
See why a cap of electrodes can spell a word only if it flashes the screen again and again
Picture a person who is awake and thinks clearly but cannot move or speak. A friend holds up a grid of letters. The friend points at one row after another and watches for any sign. There is no sign. The person has no hand to raise and no voice to say yes.
Now put a cap of small sensors on that person’s head. A computer, not a friend, runs the grid. It flashes a row, then another, then a column. The person only looks at the letter they want and waits for it to light. The cap records a faint electrical change each time. The computer reads those changes and types the letter.
This is the P300 speller. Lawrence Farwell and Emanuel Donchin described it in 1988, in a paper titled “Talking off the top of your head”. They built a system for people who cannot use any muscle to communicate. Their abstract reports that, under their conditions, volunteers could send 12.0 bits, or 2.3 characters, per minute.
That is slow. A typist at 40 words a minute sends about 200 characters a minute, roughly ninety times more. But it is a real channel out of a body that offers no other. It uses no surgery, no implant and no training in a special mental trick. The person only has to pay attention.
First, what does the cap actually see? The sensors do not read thoughts. They measure a voltage on the scalp, a few millionths of a volt, and most of it is background chatter unrelated to the letter. Chapter 1 shows how small the useful part is.
Second, why does a flash of the right letter leave a mark? The brain makes a particular slow wave when something rare and important happens. It is called the P300, and Chapter 2 builds it from the experiment that found it.
Third, how does a mark too small to see become a letter on the screen? The answer is the idea that runs through the whole lesson: average many tries. Noise points in random directions and cancels. The mark points the same way each time and stays. Chapter 3 proves how fast that works.
The rest of the lesson follows the signal from the scalp to the screen. Chapter 4 cleans it. Chapter 5 shows how rows and columns find a letter. Chapter 6 builds the classifier that scores each flash. Chapter 7 counts the speed in bits per minute. Chapter 8 shows what breaks. Chapter 9 gathers the numbers, the code and the sources.
It helps to see the whole machine as a function before we open it. Two lists go in. The first is the EEG stream: for each of the C electrodes, one voltage every few milliseconds. The second is the event log: a list of flashes, each with a time and a code that says which row or column lit up. One number comes out: the index of the chosen letter, from 0 to 35.
Here is a tiny, illustrative picture of the gap between the two. Suppose one round holds 12 flashes and the speller runs 10 rounds. That is 120 flashes, and 120 short windows of EEG to judge. Only 20 of those windows follow a flash of your letter. The machine must find the 20 from the 120, using only a bump that is smaller than the noise. Each later chapter removes one obstacle on that road.
The road has one hard rule. Nothing may peek at the answer. The computer knows when each flash happened, but not which letter you chose. It must work that out from the voltage alone. That is why the design is a sorting job, and why averaging is the heart of it.
Go back to the grid at the top of the page. The teal ring marks the letter you “want”. Each flash gives the classifier a score. A flash that holds your letter scores higher on average, but each score is noisy. The purple bars add the scores up for each row and each column. With one round the bars are a mess. With ten rounds one row and one column rise above the rest. That rise is the whole story in miniature.
Chapter 1
Measure what a scalp electrode sees, and find out why a single flash hides its own answer
You sit in a quiet room with a cap on. A screen shows a line that wiggles up and down. You blink, and the line jumps. You clench your jaw, and it jumps again. You do nothing, and it still wiggles, in a smooth wave about ten times a second. That wiggle is your own brain, and it is the raw material of this lesson.
Brain cells talk with tiny electric currents. One cell makes a current far too small to feel through the skull. But thousands of cells in one patch of cortex (the folded outer layer of the brain) often work together. Their currents add up. The sum leaks through bone and skin to the scalp.
An electrode is a small metal disc, often held on the skin with a conductive gel. It does not measure a current. It measures a voltage: how hard the charge is pushed, compared with a second point called the reference. Think of water pressure. One gauge alone says nothing; you need the difference between two pipes.
The record of that voltage over time is the electroencephalogram, or EEG. The scale is the microvolt (µV), one millionth of a volt. A small battery makes 1.5 volts, so the brain’s scalp signal is about a million times weaker. An amplifier must boost it before a computer can read it.
Electrode positions follow a standard map called the 10-20 system. Each site has a letter for the brain region and a number for the side. Pz sits on the midline, over the parietal lobe at the back-top of the head. The P3b flavour of the P300 is typically largest over parietal sites such as Pz (as summarised in the Wikipedia article on the P300, which cites Polich, 2007). That is why speller caps always include a few electrodes near there.
Now the key problem. Suppose a flash of your letter makes the brain add a bump of about 6 µV. That number is illustrative, a round figure that keeps the arithmetic easy. The background EEG at the same moment wanders by 10 to 20 µV. The bump is smaller than the noise around it.
An event-related potential (ERP) is a voltage change that is locked in time to an event, such as a flash. The background is not locked to anything. We call the locked part the signal and everything else the noise. Noise here does not mean error. It means brain activity that has nothing to do with our flash: the ten-per-second rhythm, thoughts about lunch, a tense neck.
We measure how bad it is with the signal-to-noise ratio (SNR). In this lesson SNR is the size of the signal divided by the typical size of the noise, both in microvolts. A ratio of 1 means they match. Below 1, the noise wins.
SNR = signal peak / noise size = 6 µV / 15 µV = 0.40
Drag the noise below. With a quiet background the bump is plain. With a typical background it vanishes. Press Show the hidden bump to see where it really is.
The dashed line is the moment of the flash. The blue trace is what the electrode records after a flash of your letter. Raise the noise and try to spot the bump near 300 ms.
Illustrative: a 6 µV bump peaking at 320 ms on top of a small visual response that every flash causes, plus noise made of a smooth wobble and a 10 Hz rhythm. Positive is up. These are not recordings.
The amplifier measures the voltage many times a second. Each measurement is one number. How fast depends on the system; this lesson draws 100 samples a second to keep plots light. A converter turns each voltage into an integer (the quantisation step, as in the quantization lesson). So one trace is just a list of numbers.
For one electrode and a window from 100 ms before a flash to 800 ms after, the list holds 90 numbers. For a cap with C electrodes it is a table of shape (C, 90). We call that window an epoch. The cap makes one epoch per flash.
Three things add noise that has nothing to do with the brain. First, mains hum: wires in the walls add a 50 or 60 Hz wave. Second, drift: the gel dries and the baseline slides slowly. Third, muscle and eye activity: a blink or a clenched jaw is far bigger than any brain wave.
The symptom is a trace that looks wild. The measurement is simple: look at the size and rhythm of the wobble. A fast, even buzz at 50 or 60 Hz is hum. A slow slide is drift. A tall spike once every few seconds is a blink. The fixes are a filter for hum and drift, and a rule to throw away blink epochs. Chapter 4 builds both. The DFT lesson shows how to read a rhythm by its frequency.
The bump itself was found by averaging, not by looking at single traces. The next chapter shows what the bump is. The chapter after shows why averaging finds it.
The converter that turns voltage into integers has a fixed range and a fixed number of steps. Take an illustrative converter that spans 10 V in 16 bits. It has 65,536 steps, so one step is 10 / 65,536 = 153 µV. A 6 µV bump is far smaller than one step. The converter would not see it.
So an amplifier sits in front of the converter. With a gain of 1,000, one converter step corresponds to 153 / 1,000 = 0.15 µV at the scalp. Now the 6 µV bump spans about 40 steps, and the converter can follow its shape. The same amplifier boosts the noise by the same factor, so the SNR does not change. Gain makes the signal measurable. It does not make it cleaner.
The reference matters too. Every number the electrode reports is a difference from the reference. If the reference picks up a blink, every channel shows the blink. Labs choose a reference site that is quiet, often on the ear or the bone behind it. A sloppy reference adds noise to all channels at once.
One last point about plots. This lesson draws positive voltage upward. Some labs draw negative upward, so a P300 points down in their figures. Always check the axis label before you compare a figure with this lesson.
Chapter 2
Run the experiment that found the P300, and see why a rare, wanted event makes the bump
You wear headphones. A low beep sounds every second or so, over and over. Now and then a high beep slips in. Your job is silent: count the high beeps. The low ones mean nothing. When a high beep arrives, something in your head says there it is.
That small inner “there it is” has an electrical shadow. It shows up about a third of a second after the sound, over the back of the head. It is the P300. This setup is the oddball paradigm: many common stimuli, a few rare ones, and a task that makes the rare ones matter. The common ones are standards. The rare ones are targets.
Sutton, Braren, Zubin and John published the first report in Science in 1965. They recorded scalp potentials to clicks and flashes while the subject was unsure which one would come. The average wave was larger when the subject was uncertain about what was coming. The PubMed abstract describes differences in the average evoked-potential waveforms as a function of the subject’s uncertainty about the stimulus.
The name is old shorthand. P means the wave points positive (up, in the usual plot). 300 means it peaks about 300 ms after the stimulus. In real people the peak can fall anywhere from about 250 to 500 ms, depending on the task and the person (Polich, 2007, as summarised on the Wikipedia P300 page). We use 320 ms in our toy.
Two facts matter for a speller. First, the bump follows what the stimulus means, not what it looks like. The same flash on the same letter makes a big bump if it is the letter you want, and a small one if you ignore it. A wave that follows meaning is exactly what a mind-reading machine needs. The machine does not need to know where you look in your head. It needs to know which flashes you cared about.
Second, rarer targets make a bigger bump. Sutton and colleagues saw a larger late positive wave for the less likely stimulus in one of their conditions. Later work by Donchin and others described the same pattern for overall and local probability (as summarised on the Wikipedia page). Surprise matters. A target that shows up half the time is hardly surprising.
Try it. The stream below holds 40 stimuli. Slide the target share and watch the expected bump change. Then switch off your attention and see the bump shrink.
Top: 40 stimuli in the order they arrive. Warm squares are targets. Bottom: the bump the model expects after a target. Drag the target share. Then toggle attention.
Illustrative shape, not data: bump size is set to 12 µV times (1 minus the share) squared, and cut to 30% when you ignore the targets. The real finding is the direction (rarer and wanted gives a bigger bump). The 17% marks the speller, where 2 of 12 flashes hold your letter.
A speller has no beeps. It has flashes. Out of the 12 flashes in a round (6 rows and 6 columns), only 2 hold the letter you want: one row and one column. So 2 in 12 flashes, or about 17%, are targets. You are the counter. You silently count each time your letter lights up, just as the subject counted high beeps.
The computer cannot see your count. It sees the clock. It knows the exact time of every flash and which row or column flashed. When it cuts the EEG into epochs (windows after each flash), the epochs that follow your letter’s row and column carry the bump. All the others do not. The task becomes a sorting job: which epochs have it?
The standards and the targets both cause a response in the first 250 ms. That is the brain seeing light. It is the same for every flash. In our toy this is the small wiggle at 100 to 250 ms. The difference between target and standard epochs is the P300. Later, we will subtract or classify to isolate it.
Researchers have split the late positive wave into two parts. The P3a peaks earlier (about 250 to 280 ms) and is strongest over frontal and central sites. It tracks novelty. The P3b is the classic wave near 300 ms over parietal sites. It tracks rare, task-relevant events (Squires and colleagues, 1975, via the Wikipedia summary). Speller work uses the P3b, and it often also gains from the earlier visual waves.
The symptom: a user spells poorly even with a clean recording. One measurement to run is to check whether the target epochs differ from the standards at all, for example by comparing their averages. A bump that is missing means the user did not attend, or the flashes were too fast, or the targets were too common. Fixes: give a clearer task (“count the flashes”), slow the flashes, and keep the target share low.
How do you check that your recording holds a P300 at all? Average the target epochs. Average the standard epochs. Subtract the second from the first. The result is the difference wave. Anything both kinds of flash share, such as the visual response, cancels in the subtraction. What is left is the part that depends on the flash being wanted.
In our toy, the response to a standard flash at 320 ms is 0.3 µV, left over from the visual wiggle. The response to a target flash at 320 ms is 6.3 µV. The difference is 6.0 µV, the full bump. Real labs then look for the peak inside a window, say 250 to 500 ms, and record its height and its latency.
The difference wave is also the first debugging tool. If it is flat, nothing downstream can work. If it has a bump at the right time and place, the recording is sound. Do this check before you train any classifier. It takes one line of code and saves hours.
Chapter 3
Add up trials, prove that noise shrinks with the square root of their number, and count the cost
Ten friends each guess the weight of a jar of coins. Each guess is off by some random amount, one too high, the next too low. You take the average of the ten. It lands closer to the truth than most single guesses. The errors point in different directions, so they partly cancel. The true weight is the same in every guess, so it stays.
Averaging EEG epochs works the same way. Every epoch after a target flash holds the same bump plus new random noise. Add the epochs and divide by their number. The bump keeps its size. The noise shrinks. The rest of this chapter says how fast.
Write one epoch as signal plus noise. At a given moment, the signal is s, the same every trial. The noise nk is a fresh random value in trial k. The noise has a typical size σ, its standard deviation, which says how far a value usually strays from zero.
The step that matters is “independent variances add”. The variance is the square of the standard deviation. If two values are unrelated, the variance of their sum is the sum of their variances. With N trials the sum has variance Nσ², so a size of σ√N. Dividing by N leaves σ/√N.
Take the numbers from Chapter 1: signal 6 µV, noise 15 µV, one trial. SNR = 6 / 15 = 0.40. How many trials give an SNR of 2, so the bump stands twice as tall as the noise?
So 25 target trials turn a hidden bump into a clear one. Now move the slider. The blue trace is the average. The warm trace is the truth. The lower plot shows the measured noise in the average against the theory curve.
Top: the average of the first N target trials (blue) against the true response (warm). Bottom: the noise left in the average, measured, against the curve 15 divided by the square root of N.
Illustrative: 100 simulated target trials with a 6 µV bump and 15 µV noise, the same every visit. The measured noise is the root mean square of the average minus the true response, over 90 samples; it wobbles around the theory curve because it is one sample of a random thing.
Here is the bad news hidden in √N. Four trials halve the noise. Sixteen trials cut it to a quarter. Each doubling of quality costs four times the trials. At N = 25 the SNR is 2. At N = 100 it is 4. The last 75 trials bought only two more points of ratio.
In the speller, each trial is a flash, and each flash takes time. So the square root is a speed limit. Chapter 7 turns it into bits per minute.
The proof needs the noise to be independent from trial to trial. Most background EEG is. The rhythm at 10 Hz has a random phase relative to each flash, so it cancels. But some things are not independent. A slow drift that goes up through the whole session is the same in every trial. A blink that you always make after a bright flash is locked to the flash. Those do not shrink with N.
The symptom: the average stops improving, or it keeps a shape that is not the bump. The measurement: plot the leftover noise against N. If it follows σ/√N, the assumption holds. If it flattens, something is locked to the flash. The fixes are a high-pass filter for drift, and removal of blink epochs. Chapter 4 does both.
In code, the epochs for one class form an array of shape (F, C, 90). The average is X.mean(axis=0), shape (C, 90). You do not have to store every epoch. A running average updates one epoch at a time: new average = old average + (new epoch − old average) / N.
Check it on three numbers at one sample: 4, 10 and 1. After the first, the average is 4. After the second, it is 4 + (10 − 4) / 2 = 7. After the third, it is 7 + (1 − 7) / 3 = 5. The plain mean of 4, 10 and 1 is also 5. The two agree, and the running form uses constant memory.
One more point. A speller does not need the averages themselves. It needs a score for each flash, and it adds the scores (Chapter 5). Adding scores is a close cousin of averaging the epochs first: both sum evidence over flashes. That is why the square-root law carries over to the speller, and why the hero’s bars grow more reliable with each round.
Chapter 4
Slice the endless recording into epochs, level each one, and throw away the blinks before you average
The amplifier never stops. It sends a stream of numbers, say 100 or 250 every second, with no marks in it. Nothing in the stream says “a flash was here”. If you averaged the whole stream you would get a flat line near zero, because the bumps are rare and the rest is wobble.
So the system keeps a second list: the event log. Each entry holds a sample number and a label, such as “row 3 flashed”. The computer uses the log to cut the stream. Think of cutting a film reel at every clap of the clapperboard. Each cut piece is one epoch.
A common pipeline runs in this order. Real labs vary the numbers. The numbers below are typical, not rules.
Baseline correction needs the pre-flash window, so you epoch first. Rejection runs after baseline, so a large constant offset does not cause a good epoch to be dropped. Filtering runs on the long stream, before cutting, because a filter needs time to settle at its start. Cut a short epoch first and filter it, and the edges are bent.
The device below makes 20 target epochs. Each has a random offset and a slow slope, as drift would make. Three have a blink. Turn the steps on one at a time and watch two numbers: how many epochs survive, and how far the average sits from the truth.
Thin blue lines: the 20 epochs. Bold warm line: the average of the epochs kept. Dashed teal line: the rejection limit. Toggle the two steps, then move the limit.
Illustrative: each epoch is the bump plus 10 µV noise, a random offset up to ±25 µV, a slope up to ±12 µV across the window, and 3 of the 20 add a blink of 100 to 150 µV near 250 ms. The error is the root mean square gap between the average and the true response.
Try the order. With both steps off, the average is far off the truth. Baseline alone fixes the offset but keeps the three blinks. Rejection alone drops the blinks but leaves the offset. Together they work best. But a limit that is too low also throws out good epochs. Too high keeps the blinks.
Each rejected epoch is a lost trial. By Chapter 3, fewer trials mean less SNR. A limit of 30 µV on a noisy recording can discard half the data. There is a better option for blinks than dropping them: model the blink and subtract it. An independent component analysis (ICA) can find a blink as one source in a mixture and remove it, which keeps the epoch. PCA gives a related view.
The failure mode, as symptom, measurement and fix. Symptom: the average has a big, wide hump around 200 to 400 ms that looks like a P300 but is also present for standards. Measurement: average the standard epochs too, and look for the same hump; check how many epochs have values above 100 µV. Fix: reject or remove those epochs, then recompute.
Look at the numbers. Flashes arrive every 175 ms in our toy. An epoch lasts 900 ms. So each epoch holds about 900 / 175 = 5 flashes. The response to one flash is still unfolding when the next flash comes. Epochs overlap, and each epoch contains the tails of its neighbours’ responses.
This is not a mistake. The classifier learns from epochs with the overlap included, and it sees the same overlap when it scores new epochs. But it explains two facts. First, faster flashing does not give more independent evidence per second, because the responses stack. Second, the order of flashes matters a little: a target next to another target changes what the epoch looks like. Randomising the order, as the hero does, spreads this effect evenly.
There is a second timing issue in live use. A filter run on a stream can be causal (it uses only past samples) or zero-phase (it also looks ahead and so cancels the delay). Offline, you may use zero-phase. Online you cannot look ahead, so a causal filter delays the wave a little. Train and run with the same kind of filter, or the bump will sit at a different time in each.
Chapter 5
Turn flash scores into a letter by adding them up along rows and columns
You want to find one student in a hall of 36 desks, set in 6 rows of 6. You cannot see the student, but a helper can tell you “yes” or “no” if you point at a whole row. Pointing at each desk takes up to 36 questions. Pointing at rows and then columns takes 12. The row answer and the column answer cross at one desk.
That is the whole idea of the row and column speller that Farwell and Donchin used. Their abstract says the letters of the alphabet, plus a few other symbols and commands, appear on a screen as a matrix, and rows and columns flash repeatedly. Wikipedia describes the grid as 6 by 6, which gives 36 cells. We use A to Z, then 1 to 9 and a space.
You could flash the 36 letters one at a time. Then only 1 flash in 36 is a target, about 3%. By Chapter 2, a rarer target gives a bigger bump, so each flash would be easier to read. But one round would take 36 flashes, three times longer than the 12 flashes of rows and columns.
With rows and columns, a target flash happens 2 times in 12, about 17%. The bump is somewhat smaller per flash. But every letter gets two chances per round for the price of one flash each. It is a trade: fewer flashes against a less surprising target. Many systems chose the row and column design for its speed.
A classifier (Chapter 6) takes one epoch and gives a number, its score. A high score means “this looks like it has the bump”. The scores are noisy. A target flash scores high on average but not every time.
Keep a running total for each of the 6 rows and each of the 6 columns. Each time a row flashes, add that epoch’s score to the row total. Do the same for columns. After a few rounds, the row that holds your letter has the highest total. So does its column. The letter is where they cross.
letter score (row r, column c) = row totalr + column totalc
Because the sum splits into a row part and a column part, the best letter sits at the best row and the best column. The computer needs no search over 36 cells. It takes the biggest of 6 row totals and the biggest of 6 column totals.
Suppose after two rounds the row totals are 0.4, 2.1, −0.3, 0.9, 1.2 and −0.8 for rows 1 to 6. Row 2 wins with 2.1. The column totals are 0.1, 0.7, 2.6, 0.3, 1.0 and −0.2. Column 3 wins with 2.6. The speller picks row 2, column 3, which is the letter in the second row and third column: the letter I in our grid. The sum for that cell is 2.1 + 2.6 = 4.7, far above the next best, a cell like row 2 in column 5 at 2.1 + 1.0 = 3.1.
Slide the flashes below. The sequence is fixed: three rounds, 36 flashes, with a letter P as the target. At zero the board is blank. As flashes arrive, the row and column totals build and the grid brightens at the crossing.
Numbers on the left are row totals, numbers on top are column totals. A cell is as bright as its row total plus its column total. The teal ring marks the true target.
Illustrative scores from the toy model of the hero: a flash that holds the target scores 1 on average, any other 0, spread 1.2. One fixed random sequence of three rounds, the same on every visit. The real classifier is Chapter 6.
Notice what happens after one round (12 flashes). The leader is often wrong, because each total comes from only one score. After three rounds the target row and column usually stand out. Drag back and forth and watch the leader change. Each flip is a decision the speller would have got wrong if it had stopped there.
The inputs to this step are two lists. One holds the flash codes, integers from 0 to 11 (0 to 5 rows, 6 to 11 columns). The other holds the scores, one real number per flash. Their shapes are (F,) and (F,), where F is the number of flashes. The output is one integer, 0 to 35. In code this is six lines (see the Field Guide).
The failure mode. Symptom: the speller keeps choosing the right row but the wrong column, or the reverse. Measurement: log the row and column margins separately, the gap between the best and second-best total. A column margin that stays small means the column flashes are not producing a clear bump. Fix: check the display timing and the screen layout for that column, and give more rounds.
We can predict how the speller behaves. Say a target flash scores 1 on average and a non-target scores 0, each with a spread of 1.2 (the hero’s numbers). After R rounds, the target row total is about R. A wrong row total is about 0. Each total wanders by 1.2 × √R, since independent spreads add in quadrature.
Take R = 9. The target row sits near 9. The others sit near 0, wandering by 1.2 × 3 = 3.6. The gap of 9 is 2.5 wanderings wide. That is usually enough to beat all five wrong rows. At R = 1 the gap is 1 and the wandering is 1.2, so the gap is less than one wandering, and the wrong row wins often. The ratio R / (1.2 × √R) = 0.83 × √R grows with the square root of R, the same law as Chapter 3.
One small design detail. If every round shuffles the 12 flashes, the same row can come last in one round and first in the next. Two target flashes in a row then land close together, and their responses overlap. Designers can forbid such repeats. The hero does not, to keep the model simple.
Chapter 6
Build linear discriminant analysis by hand, and see why it needs shrinkage when data is short
Chapter 5 took the flash scores as given. Now we make them. A classifier is a rule that turns one epoch into one number. We want a high number when the epoch holds a P300 and a low number when it does not. The rule must learn from examples, because every head is a little different.
Before the user types freely, the system runs a short calibration. It asks the user to attend to a known list of letters. Because the system knows the target, it knows which flashes were targets and which were not. Each epoch gets a label: 1 (target) or 0 (non-target). That is a labelled training set.
One epoch from C electrodes and 90 time samples is a table of shape (C, 90). Chapters 4 and 5 left the epochs at 100 Hz. Systems often shrink the table further by averaging or skipping samples. Say 8 electrodes and 20 time points remain. Flatten the table into one row of D = 160 numbers. The classifier now sees a point in a 160-dimensional space.
Linear discriminant analysis (LDA) draws a flat boundary between two clouds of points. It assumes each cloud is a ball-shaped blob of the same shape, shifted. It finds the direction along which the two blobs are best separated, in units of the blob’s own spread. The score of an epoch is its position along that direction.
Take two features. The mean difference is Δ = (2, 0): feature 1 carries the signal, feature 2 carries none. The noise covariance is Σ = [[4, 3], [3, 4]]. The noise in the two features is strongly linked (correlation 0.75).
Look at w. Feature 2 holds no signal, yet it gets a weight of −0.86. Why? It measures the same noise that feature 1 picks up. Subtracting it removes noise from feature 1. This is the same trick as a reference electrode: use one sensor to cancel what another one shares. LDA finds such tricks on its own.
In real use we do not know Σ. We estimate it from the training epochs. With D = 160 features we must estimate 160 × 161 / 2 = 12,880 numbers. If we have only a few hundred epochs, the estimate is noisy. Worse, the matrix inverse amplifies the noisiest directions. The weights overfit: they fit the training epochs and fail on new ones.
Shrinkage pulls the estimate toward a simple target. Blend the estimated covariance with a plain diagonal matrix: Σ̃ = (1 − λ) Σ̂ + λ ν I. Here ν is the average variance of the features and I is the identity. At λ = 0 you trust the data fully. At λ = 1 you assume no features are linked.
Two studies back the practice. Krusienski and colleagues compared five classifiers on offline data from eight users. They found that stepwise LDA and Fisher’s LDA gave the best overall performance and implementation. Blankertz and colleagues reported that shrinkage LDA gave strong single-trial ERP results, clearly better than classical LDA.
Try it. The feature here is 20 time points of one channel. The noise is correlated across time. Move the training size down and see plain LDA collapse. Then raise the shrinkage.
Top: the learned weights across 0 to 600 ms (purple) against the ideal weights (dashed). Bottom: test accuracy against shrinkage. The teal line is your setting.
Illustrative: 20 features, a bump of 8 µV peaking at 320 ms, noise of 5.5 µV with correlation 0.8 between neighbours. Test accuracy is the balanced accuracy on 3,000 fresh epochs. Below 11 epochs per class the covariance has fewer samples than features and cannot be inverted well.
At 12 epochs per class and no shrinkage, the weights are jagged and the accuracy is low. A shrinkage of about 0.3 to 0.6 gives smooth weights and a better score. At 100 epochs or more, plain LDA is already near the ceiling, the gain from shrinkage is small, and full shrinkage (1.00) hurts. The best blend depends on how much data you have.
Only about one flash in six is a target, so the two classes are lopsided. LDA copes: the offset b moves, but the direction w does not. Only the ranking of scores matters for the row and column sums. A small deep network such as EEGNet (Lawhern and colleagues, 2018) is a modern alternative; it learns the filters and the classifier together. Chapter 8 returns to it.
The failure mode. Symptom: calibration accuracy is high but live typing is poor. Measurement: test on a held-out part of the calibration set, and watch the gap between training and test accuracy. Fix: more calibration epochs, stronger shrinkage, or fewer features.
Here is a worked example with illustrative numbers. Say the user spells 10 calibration letters with 10 rounds each. That gives 10 × 10 × 12 = 1,200 epochs. Two flashes in twelve hold the target, so 200 epochs are targets and 1,000 are not. Suppose each epoch is cut from 90 samples to every fifth sample, 18 per channel, over 8 channels: D = 144 features.
The covariance has D × (D + 1) / 2 = 10,440 free numbers. We estimate them from 1,200 epochs, after removing the class means. That is enough to make the matrix invertible, but not enough to make it exact. Shrinkage steadies what is left. If we had cut the data to only 3 calibration letters, we would have 360 epochs and the matrix would be far shakier.
A last warning on testing. Epochs from the same letter share the same background noise and the same slow drift. If you split flashes at random into training and test sets, the test flashes have near-twins in training, and the score looks too good. Split by letter, or better by session, so the test set is truly new.
Chapter 7
Count the bits per minute of a speller, and find how many rounds earn the most
A speller can be fast and wrong, or slow and right. One round per letter gives you words quickly, but most of them are garbage. Fifteen rounds give clean words, but you wait half a minute for each letter. Somewhere between lies the best rate. To find it we need one number that mixes speed and accuracy.
A bit is the answer to one fair yes-or-no question. Choosing one of 36 equal letters is worth log2 36 = 5.17 bits, because 25.17 = 36. If the speller picks the right letter every time, each letter delivers 5.17 bits.
When the speller can be wrong, a letter carries less. The standard fix is the information transfer formula of Wolpaw and colleagues. For N choices and accuracy P:
Work it for P = 0.9 and N = 36, step by step.
So a 90% letter is worth 4.19 bits, not 5.17. Accuracy of 50% would be worth far less. The formula punishes errors, and it says a letter that is right half the time still carries some information.
Now count the time. One round is 12 flashes. If each flash takes 175 ms (our illustrative figure), a round is 2.1 s. Add a pause of 2 s for the user to find the next letter. With 10 rounds, one letter takes 10 × 2.1 + 2 = 23 s. At 4.19 bits per letter that is 4.19 / 23 × 60 = 10.9 bits per minute.
Compare with the original. The Farwell and Donchin abstract reports that the channel can operate accurately at 0.20 bits per second and that subjects could send 12.0 bits, or 2.3 characters, per minute under their conditions. 0.20 bits per second is 12 bits per minute, so the two statements agree. If the matrix had 36 cells, then 2.3 characters at 5.17 bits would be 11.9 bits. That is our arithmetic, not a statement in the abstract.
Now the device. The purple line is bits per minute against the number of rounds. The dashed line is letter accuracy. Find the peak. Then turn on stopping early.
The speller model is the one from the hero. Slide the rounds per letter and read the bits per minute. Then switch on Stop early: the speller quits as soon as its lead is big enough, up to 15 rounds.
Toy model, not a measured speller: 175 ms per flash, a 2 s pause between letters, 36 letters, and the flash scores of the hero. Accuracy comes from simulation. The stopping rule is crude: stop when the best row and best column each lead the runner-up by the chosen number of standard deviations of the sum. Real systems tune this differently.
The best fixed setting is a middle one. Below it, errors waste bits. Above it, time wastes them. The peak is broad, so a few rounds either way cost little. In our toy, stopping early saves rounds, but the gain is modest: it helps most on letters where the evidence is clear, and it spends the cap on hard letters.
Bits per minute treats every choice as equally likely. Real language is not. A language model can guess that “Q” is followed by “U”. Spellers that add such priors can reach the same accuracy with fewer flashes. Bits per minute also ignores the cost of mistakes: a wrong letter needs a backspace, which is itself a selection.
The failure mode. Symptom: a user reports the speller is too slow, though accuracy is high. Measurement: time per letter, split into flash time and pause. Fix: fewer rounds with a better classifier, early stopping, a shorter pause, or a predictive text layer.
You can reason about the curve without the device. One letter at 4 rounds takes 4 × 2.1 + 2 = 10.4 s. At 8 rounds it takes 8 × 2.1 + 2 = 18.8 s. Going from 4 rounds to 8 takes 1.8 times as long. So the extra rounds pay off only if the bits per letter rise by more than a factor of 1.8.
Early on, they do. Going from 1 to 4 rounds takes a letter from nearly useless to mostly right, and the bits grow fast. Later, accuracy is already near 100% and the bits cannot rise above 5.17. The extra time buys almost nothing. That bend is why the curve has a peak.
Two more knobs shift the peak. The pause is dead time with no information, so a shorter pause helps short settings the most. And a faster flash rate cuts the time per round, but by Chapter 4 the responses overlap more, so the bump can shrink. Every lever trades against another, which is why spellers are tuned for each user.
Chapter 8
Smear the bump with timing jitter, then see where modern EEG models pick up the story
It is the end of a long session. The user is tired. The bump that arrived 320 ms after a flash at the start now arrives at 280 ms on one flash and 390 ms on the next. Each epoch still holds a clean bump. But the bumps no longer line up. The average puts them on top of each other at slightly different places, and the result is a low, wide hill.
This is latency jitter: the random spread in when the response peaks. Averaging was a good idea only if the signal was the same every time. When its timing drifts, averaging blurs it.
Treat the bump as a bell curve of width w (75 ms in our toy). Let each trial shift it by a random amount with spread j (the jitter). The average is the bell curve blurred by the shifts. Two bell curves blur into a wider one. The widths add in quadrature, like the sides of a right triangle:
blurred width = √(w² + j²), peak height = A × w / √(w² + j²)
The area of the bump is conserved. A wider bump with the same area is shorter. Work an example with j = 80 ms.
Slide the jitter below. The warm trace is the average of 40 trials with no noise, so you see only the effect of timing. The blue trace adds the usual 15 µV noise.
Thin warm lines: the bump in each of 40 trials, with its own latency. Bold warm line: their average. Blue: the average with noise added. Raise the jitter. Then try aligning the trials.
Illustrative: 40 trials, a 6 µV bump of width 75 ms centred at 320 ms, each trial shifted by a random latency with the spread you choose. “Align” shifts each trial back by its true latency, which a real system does not know; it must estimate it from noisy data. Noise: 15 µV.
With the alignment on, the bump is sharp again. A real system has to estimate each trial’s latency from a noisy epoch. That estimate is as hard as the original problem. So practical systems use another route: a classifier that looks at a window of samples and can find the bump at any place inside it.
Timing is not the only change. The Wikipedia page notes that the P300 peak can fall anywhere from about 250 to 500 ms, and varies with task and person. So the best classifier for you is not the best for the next user. And your own signal changes over a session: the electrodes warm, the gel dries, you get tired. A classifier trained at 9 o’clock may be worse at 11.
The symptom: accuracy falls over a session, or from day to day. The measurement: track the accuracy and the average bump height in blocks of a few letters. The fixes: calibrate again, adapt the classifier as you go from letters whose answer you can trust (the adaptive Kalman lesson shows the same idea for a filter), or train on many sessions so the model has seen drift.
The pipeline in this lesson is hand-built: a filter, a window, a linear classifier. Three newer ideas change parts of it.
Learned filters. EEGNet (Lawhern and colleagues, 2018) is a small convolutional network that learns the spatial and temporal filters itself. They tested it on four paradigms, including P300 responses. Their abstract says it generalised across paradigms better than the reference algorithms when training data was limited.
Pre-trained EEG models. LaBraM (Jiang and colleagues, ICLR 2024) cuts EEG into patches, turns each patch into a discrete code with a vector-quantised tokenizer, and trains a transformer to predict masked codes across many datasets. CBraMod (ICLR 2025) uses separate spatial and temporal attention. The hope is one pre-trained encoder that a short calibration can adapt to a new user. The transformer lesson covers the attention part.
Company claims. The start-up Sabi describes, on its site, a pipeline of new sensors, a large decoding dataset and a “Brain Foundation Model”. Those are the company’s own claims. We found no independent accuracy data for them.
None of this removes the need for the ideas above. Any decoder still faces a signal smaller than its noise, timing that wanders, and a user who changes. Averaging, careful cutting and honest tests on fresh data stay the base.
The jitter device shows a failure that a calibration score would not catch. The classifier is trained when the user is fresh. It is then used when the user is tired. The honest test is to train on one session and test on a later one. A model that holds up across days is worth more than one that scores high on a random split.
Suppose accuracy is 90% in the first block of ten letters and 72% in the fourth block. Then something drifted. Plot the average bump height per block. If the height fell, the user attends less or the electrodes have degraded. If the height held but the latency moved, the classifier’s window no longer fits. The two causes need different fixes, so measure before you act.
One fix is to adapt online. After the speller commits to a letter, and the user does not correct it, treat the letter as a label. The flashes of that row and column become new target examples. The classifier updates a little with each letter. Errors can slip in as wrong labels, so the update is kept small. A filter that tracks a drifting signal, like the one in the adaptive Kalman lesson, uses the same idea of leaning on recent data.
Chapter 9
Keep the numbers, the terms, the checklist and the code, and follow the links onward
The whole lesson in one paragraph. A flash of the letter you want, when you are counting it, makes a small slow positive wave about 300 ms later. One epoch hides it in noise, because its SNR is below 1. Averaging N epochs shrinks the noise by the square root of N and keeps the wave. A speller flashes 6 rows and 6 columns, scores each flash with a classifier such as shrinkage LDA, adds the scores per row and column, and reads the letter where the best row and best column cross. The price is time, counted in bits per minute. The danger is anything that breaks the “same signal every time” assumption, such as latency jitter.
| Quantity | Value | Where it comes from |
|---|---|---|
| Original speller rate | 12.0 bits, or 2.3 characters, per minute; 0.20 bits per second | Farwell and Donchin 1988, abstract |
| Grid | 6 by 6, rows and columns flash | Wikipedia P300 article, describing Farwell and Donchin |
| P300 latency | about 250 to 500 ms; P3b peaks near 300 ms | Polich 2007, via the Wikipedia article |
| Scalp site | strongest over parietal sites such as Pz | Polich 2007, via the Wikipedia article |
| Averaged SNR | SNRN = √N × single-trial SNR | Derived in Chapter 3 |
| Trials for 4 times the SNR | 16 | Arithmetic from the line above |
| Target share in the speller | 2 of 12 flashes, about 17% | Design of a 6 by 6 row and column speller |
| Bits per letter, 36 choices | 5.17 at 100%; 4.19 at 90% | Wolpaw formula, worked in Chapter 7 |
| Best classifiers, five compared | stepwise LDA and Fisher LDA, on offline data from eight users | Krusienski et al. 2006, abstract |
| Shrinkage LDA | clearly better than classical LDA for single-trial ERP | Blankertz et al. 2011, abstract |
| Jitter cost | peak ratio = w / √(w² + j²) | Derived in Chapter 8 |
Every other number in the devices (6 µV bump, 15 µV noise, 175 ms per flash, 2 s pause, score spread) is an illustration chosen to make the arithmetic clear. None is a measurement.
python import numpy as np FS, PRE, POST = 100, 0.1, 0.8 # Hz, seconds before and after a flash def epochs(eeg, flash_idx): # eeg: (C, T_total) flash_idx: (F,) a, b = int(PRE * FS), int(POST * FS) X = np.stack([eeg[:, i - a:i + b] for i in flash_idx]) # (F, C, a + b) X = X - X[:, :, :a].mean(axis=2, keepdims=True) # baseline return X def keep(X, thr=100.0): # drop blink epochs, thr in microvolts return np.abs(X).max(axis=(1, 2)) < thr def fit_lda(X, y, lam=0.1): # X: (F, D) flattened epochs, y in {0, 1} m1, m0 = X[y == 1].mean(0), X[y == 0].mean(0) Xc = np.where(y[:, None] == 1, X - m1, X - m0) S = Xc.T @ Xc / (len(X) - 2) # pooled covariance, (D, D) S = (1 - lam) * S + lam * np.trace(S) / S.shape[0] * np.eye(S.shape[0]) w = np.linalg.solve(S, m1 - m0) b = -w @ (m1 + m0) / 2 return w, b # score = X @ w + b def decode(scores, code): # code: 0..5 rows, 6..11 columns, (F,) tot = np.zeros(12) np.add.at(tot, code, scores) # running totals per line return 6 * tot[:6].argmax() + tot[6:].argmax() # letter index 0..35 def bits(p, n=36): # Wolpaw bits per selection return np.log2(n) + p * np.log2(p) + (1 - p) * np.log2((1 - p) / (n - 1))
Run epochs, then keep, flatten each kept epoch to a row of D numbers, fit with fit_lda on the calibration letters, and score live flashes with X @ w + b. Feed the scores and the flash codes to decode.
The P300 speller is the oldest non-invasive communication BCI and still the baseline. Its pieces recur everywhere. Averaging and epoching are the base of all ERP work. A linear classifier with shrinkage is the standard first model for EEG. Bits per minute is how BCI papers compare systems, including the invasive ones. Newer EEG models, from small networks to pre-trained transformers, replace the filter and classifier but face the same noise, timing and drift.
Press Present to explain it back in your own words, or Teach to draw it on the whiteboard. Or go back to the grid and see how many rounds your letter needs.