Spike counts from a few dozen neurons become a moving hand through one line of linear algebra. See when that line works, when it fails, and what ridge fixes.
Learn how a brain-computer interface (BCI, a device that reads brain signals and turns them into commands) decodes hand movement from spikes, using only counting, a weighted sum and a penalty.
Choose a decoder, then drag the number of neurons. Watch the orange line try to follow the blue hand on 15 seconds it never saw. Then we build it piece by piece: bins, weights, lags, ridge, and the curve that says how many neurons you need.
You need a feel for averages and what a straight line fit is. We build the matrix algebra, the lags and the ridge penalty from zero.
Grounded in Wessberg et al., Nature 2000, Serruya et al., Nature 2002, and Glaser et al., eNeuro 2020 (open access, with code). The recording in the devices is a simulation that we label as such.
Simulated, not a recording: 32 neurons with random tuning, one hand velocity, counts in 50 ms bins, 10 s of training data, 15 s of test data. The test seconds are never used to fit. R² is the share of the hand’s variance the decoder explains.
Chapter 0
See why a weighted sum of neuron counts is the first thing every brain-computer interface tries, and what questions it raises
A monkey sits in a chair and moves a handle across a table. A thin grid of electrodes rests on the surface of its motor cortex, the strip of brain that sends movement commands to the arm. Each electrode listens to a few nearby neurons.
A spike is the short electrical pulse a neuron fires. It is the brain’s smallest message, like one click of a Geiger counter. A neuron that fires fast is shouting. A neuron that fires slowly is almost silent. (Our lessons on the spike and on spike detection show where these pulses come from and how software finds them.)
Now ask a plain question. If you only saw the spikes, could you say where the hand is going? You would have no camera. You would have a list of numbers, one per neuron, updated many times a second. A decoder is the program that answers: it takes those numbers in and gives a movement out.
Think of a panel of judges scoring a diving contest. Each judge gives a score. The final mark is not the plain average, because some judges are known to be harsh and some generous. You give each judge a weight, a number that says how much to trust and how to tilt that judge’s score. The final mark is the sum of weight times score.
A linear decoder works the same way. The judges are neurons. Each neuron’s score is how many spikes it fired in a short window. The final mark is the predicted hand velocity. The decoder is only the list of weights, plus one extra number called the intercept that sets the output when every neuron is quiet.
This idea is old and it keeps winning. Wessberg and colleagues (Nature, 2000) recorded many neurons across three cortical areas in primates: premotor, primary motor and posterior parietal. They predicted one-dimensional and three-dimensional arm paths in real time, with both linear and nonlinear methods. They also drove robot arms with the result, locally and across the Internet. Serruya and colleagues (Nature, 2002) showed that activity from just 7 to 30 motor cortex neurons, decoded by a linear filter, let a monkey move a computer cursor at once, with no long training period. The monkey had a working device the first time it tried.
Look at what a decoder actually sees. Below, the top line is the velocity of a simulated hand over 12 seconds: up means moving right, down means moving left. Underneath, each row is one neuron and each column is a short slice of time. A bright cell means many spikes in that slice. Drag the playhead and compare the line with the rows.
Blue is the hand’s velocity. Each purple row is one neuron: brighter means more spikes in that 50 ms slice. Drag the time and look for rows that rise and fall with the blue line.
Simulated, not a recording: velocity is a smooth random signal, and each neuron fires a baseline count plus a tilt toward the hand’s velocity, with noise. The 12 neurons shown are the first 12 of the lesson’s 32.
You probably spotted that no single row follows the blue line cleanly. Some rows rise when the hand goes right. Others rise when it goes left. All of them are noisy. But a clever sum of the rows could cancel the noise and keep the shared signal. That is the whole bet of a linear decoder.
The bet raises four practical questions. Each one has a chapter.
What exactly do we sum? Spikes arrive at odd moments, not on a clock. We must first cut time into equal slices and count (Chapter 1). Then we find the best weights by least squares (Chapters 2 and 3).
When do the spikes matter? A neuron in motor cortex fires a little before the hand moves. So the decoder should look at the recent past, not only the present slice. That is the lagged design (Chapter 4).
How do we stop it fooling itself? A decoder with hundreds of weights and little data can memorise noise. We hold back data to test it, and we add a penalty called ridge (Chapter 5).
How many neurons, and how does it compare? The neuron-dropping curve answers the first (Chapter 6). Glaser and colleagues’ large comparison answers the second (Chapter 7). Chapter 8 asks what changes when a person closes the loop, and Chapter 9 gathers everything on one page.
Throughout, we build one small simulated recording and use it in every device. We say so each time. The papers supply the real numbers, and we cite where each one comes from.
Chapter 1
Turn a scatter of spike times into a regular table of counts that a weighted sum can use
Picture rain on a tin roof. You hear separate drops at odd moments. You cannot add up drops that fall at odd moments the way you add up numbers in a spreadsheet. But you can ask a simple question every half second: how many drops landed since I last asked? That count is a number you can use.
A neuron’s spikes are the raindrops. The electrode records the spike times, the exact moment of each pulse. A decoder wants one number per neuron per tick of a clock. The step that makes this is called binning: cut time into equal slices called bins, and write down how many spikes fall in each one.
Say one neuron fires at 12, 31, 47, 52 and 58 milliseconds (ms) after we start recording. Choose a bin width of 50 ms. Bin 0 covers 0 to 50 ms and catches the spikes at 12, 31 and 47. Bin 1 covers 50 to 100 ms and catches 52 and 58. The neuron becomes the list [3, 2].
Now use 10 ms bins instead. Spike 12 lands in bin 1, spike 31 in bin 3, spike 47 in bin 4, and spikes 52 and 58 both in bin 5. The same neuron becomes [0, 1, 0, 1, 1, 2] over the first 60 ms. With one 200 ms bin it is just [5].
The count divided by the width is the firing rate. Three spikes in a 50 ms bin is 3 / 0.05 s = 60 spikes per second (60 Hz). The decoder can use counts or rates. They differ only by the constant 1 / Δ, and the weights absorb that constant.
Try the bin width yourself. The raster at the top marks every spike as a tick. The grid below it shows what the decoder would receive.
Each tick is one spike from one of six neurons over 2 seconds. Choose a bin width and read the count table underneath: darker cells hold more spikes.
Simulated spike times: each neuron’s 50 ms counts come from the lesson’s toy recording, and the spikes are placed at random moments inside each bin. The 50 ms and 200 ms widths are the ones Glaser et al. 2020 used for their motor cortex and hippocampus data.
Narrow bins keep timing. They also hold mostly zeros and ones, and a zero tells you little. Wide bins give clean counts and smooth away fast changes. They also add delay, because a bin is only finished when its last millisecond has passed. A decoder that needs a full 200 ms bin cannot react faster than 200 ms.
So the bin width is a knob with a cost at both ends. Glaser and colleagues (eNeuro, 2020) used 50 ms bins for their monkey motor cortex and somatosensory cortex data, and 200 ms bins for rat hippocampus, where position changes more slowly. We use 50 ms for the rest of the lesson. That also gives the decoder 20 new rows of data per second.
After binning, a recording of N neurons over T bins is a table with T rows and N columns. Call it X. Row t holds the counts of all neurons in time slice t. The hand also gets one value per bin: the average velocity inside that slice, called y. So the learning problem is plain. We have T examples. Each example is a row of N numbers in, one number out.
For our toy recording, that means shapes you can write down. Ten seconds of training at 50 ms is 200 bins. With 16 neurons, X is 200 × 16 and y is 200 × 1. In the hero, the decoder learned from those 3,200 count numbers and then predicted 300 new velocities.
Real systems do the same. A Utah array (a bed of needles, 96 electrodes) streams spike times. A small program on the receiving computer counts them every 50 ms or so. Everything downstream, including the Kalman filter of a later lesson, starts from that table.
One more detail matters for honesty. The velocity in each bin is not recorded with the neurons. Cameras or a joystick log the hand, and software lines the two clocks up. If the clocks drift by 100 ms, every weight we learn will be quietly wrong. Check the timing before you check the maths.
Chapter 2
Fit the best straight line from one neuron’s spike count to hand velocity, and learn exactly what “best” means
You run a lemonade stand. On hot days you sell more. You have a notebook with 30 days: the temperature and the cups sold. A friend asks, “It will be 31 degrees tomorrow. How many cups?” You draw a straight line through your dots and read off the answer.
That is the whole idea of this chapter. Swap temperature for one neuron’s spike count, and swap cups for hand velocity. The line is a decoder with one weight.
A line has two numbers. The weight w is the slope: how much the predicted velocity changes for one extra spike. The intercept b is the prediction when the neuron fires nothing. The prediction for bin t is
Many lines pass near the dots. Which one is best? We need a score. For each bin, the residual is the truth minus the guess, yt − ŷt: how far off we were. Residuals can be positive or negative, so we square them and add them up. The line with the smallest total is the winner. This rule is least squares.
Why squares? A square counts a miss of 2 as four times worse than a miss of 1, so one big miss cannot hide among many tiny ones. Squares also make the maths clean. The best line has a closed form, no search needed:
Four bins have counts 2, 4, 6, 8 and velocities −1.0, −0.2, 0.4, 1.2 (illustrative numbers). The average count is x̄ = 5 and the average velocity is ȳ = 0.1.
The count offsets are −3, −1, 1, 3. The velocity offsets are −1.1, −0.3, 0.3, 1.1. Multiply pairs: 3.3, 0.3, 0.3, 3.3. The sum is 7.2. The squared count offsets are 9, 1, 1, 9, and their sum is 20. So w = 7.2 / 20 = 0.36, and b = 0.1 − 0.36 × 5 = −1.7.
The line predicts −0.98, −0.26, 0.46 and 1.18. The residuals are −0.02, 0.06, −0.06 and 0.02. Their squares sum to 0.008. The total variation of y around its mean is 1.21 + 0.09 + 0.09 + 1.21 = 2.6. The score R² is 1 − 0.008 / 2.6 = 0.997. It reads as the share of the velocity’s ups and downs that the line explains. Zero means no better than always guessing the average. One means perfect.
Now run the same recipe on a simulated neuron. Pick a neuron, then raise the noise. Watch the cloud fatten and the score fall.
Each dot is one 50 ms bin: the neuron’s count across, the hand’s velocity up. The orange line is least squares. Thin lines are residuals. Choose a neuron, then drag the noise.
Simulated: 200 bins (10 s) from the lesson’s toy recording; “noise” is the spread, in spikes per bin, of the random part of each count. R² here is measured on the same dots the line was fitted to, which flatters it. Chapter 5 shows why that matters.
Look at the direction. Biologists often fit the other way: count as a function of velocity. That is an encoding model, a story of how the neuron responds to the world (our lesson on generalized linear models does it with spikes properly). A decoder runs the other way. It reads the neuron and predicts the world. Least squares of velocity on counts gives the decoder directly.
This mattered in the early experiments. Serruya and colleagues (Nature, 2002) used 7 to 30 motor cortex neurons so the monkey could drive a cursor at once. The fit uses only simple sums of the kind above, so it can run in a fraction of a second.
The symptom: the line barely tilts and the predicted velocity hugs the average. The measurement: R² stays near 0.1 even at low noise for a weakly tuned neuron, and falls toward 0.0 as you raise the noise. The fix is not a cleverer line. It is more neurons, each adding a small piece of the signal, which is the next chapter.
Chapter 3
Stack every neuron’s counts into a table, solve for all the weights at once, and test the result on data it never saw
Back at the lemonade stand, you now also have the day of the week, whether it is a holiday and the price. Each fact nudges the sales. No single fact is enough, but together they give a sharp guess. You want one weight per fact, found together, because the facts overlap. Hot days are often holidays.
A decoder has the same shape. Each neuron is one fact. Neurons overlap too, because nearby cells listen to the same movement. So we find all weights at the same moment. The tool that does it is a table of numbers and a short formula.
Take the count table from Chapter 1: T rows (one per bin) and N columns (one per neuron). Mathematicians call it the design matrix, written X. Put the hand velocities in a column y with T entries. The weights form a column w with N entries. The prediction for every bin at once is
We handle the intercept by subtracting each column’s average from X and the average of y from y before we fit. Then the line passes through zero, and b is put back at the end. Real code does exactly this, or adds a column of ones.
Least squares asks for the w that makes the total squared residual smallest. The total is a bowl-shaped function of w. The bottom of the bowl is where the slope is zero in every direction. Setting the slope to zero gives the normal equations:
Read it in words. The right side asks, “how does each neuron relate to the hand?” The left matrix says, “but the neurons also relate to each other, so untangle that first.” Solving the pair gives weights that share the credit sensibly.
Take four bins, two neurons, counts already centred (illustrative). Rows of X are (1, 1), (1, 0), (−1, −1), (−1, 0). The velocities are y = 1.0, 0.6, −1.0, −0.4. Their average is 0.05, close to zero, so we measure variation around zero to keep the sums short.
First, X⊤X. Neuron 1 against itself: 1 + 1 + 1 + 1 = 4. Neuron 1 against neuron 2: 1 + 0 + 1 + 0 = 2. Neuron 2 against itself: 1 + 0 + 1 + 0 = 2. So X⊤X = [[4, 2], [2, 2]].
Second, X⊤y. Neuron 1 with y: 1.0 + 0.6 + 1.0 + 0.4 = 3.0. Neuron 2 with y: 1.0 + 0 + 1.0 + 0 = 2.0. So X⊤y = (3.0, 2.0).
Third, solve. The determinant of the 2 × 2 table is 4 × 2 − 2 × 2 = 4. The inverse is one quarter of [[2, −2], [−2, 4]]. Multiply by (3.0, 2.0): w1 = (2 × 3 − 2 × 2) / 4 = 0.5 and w2 = (−2 × 3 + 4 × 2) / 4 = 0.5.
Check. The predictions are 1.0, 0.5, −1.0 and −0.5. The residuals are 0, 0.1, 0 and 0.1, so the squared error is 0.02. The variation of y is 2.52, so R² = 1 − 0.02 / 2.52 = 0.992. Neuron 2 alone could not tell bins 1 and 2 from bins 3 and 4 so cleanly, but together the pair does.
Now the key habit. Split the recording. Fit w on the first part, the training set. Then score it on the rest, the test set, which the fitting never touched. The test R² is the honest number, because it measures prediction, not memory. Glaser and colleagues (eNeuro, 2020) did this with 10-fold cross-validation: ten different splits, each using 80% of the data to train, 10% to tune settings and 10% to test, then an average.
Add neurons one at a time below. The left picture is the table X for a short stretch. The right picture is the test result.
Left: the first 60 bins of X, one column per neuron. Right: the hand in blue and the decoder in orange on held-out bins. Drag the number of neurons.
Simulated: 400 training bins (20 s), 300 held-out test bins (15 s), no lags yet, noise 2.5 spikes per bin, ridge strength 1 (nearly plain least squares). Test R² is measured on the held-out block only.
The symptom: two weights with huge values and opposite signs, such as +40 and −39, though each neuron alone has a modest effect. The measurement: X⊤X has an entry that nearly equals what a combination of the others would give, so its determinant is close to zero and its inverse is huge. Tiny noise then swings the weights wildly. The fix is a penalty that keeps the weights small. That is ridge, in Chapter 5.
Chapter 4
Feed the decoder the last few bins of every neuron, so it can use the fact that the brain speaks before the hand moves
You throw a ball to a friend, and a camera films the catch. On the film, her hand closes a moment after her eyes saw the ball. If you want to guess when her hand closes, the useful clue is not what her eyes do now. It is what her eyes did a moment ago.
Motor cortex works the same way. Neurons there fire a little before the arm moves, because they send the command. To guess the velocity in bin t, the most useful spikes may sit in bins t − 1, t − 2 or t − 4. A decoder that looks only at bin t misses them.
The fix is simple. For each neuron, feed the decoder not one count but a short history: the count now, the count one bin ago, two bins ago, and so on up to B bins. Each of these is a lag. B is the number of lags kept, the history window. At 50 ms bins, B = 8 is 400 ms of memory.
Take one neuron with counts 3, 2, 5, 1, 4 in five bins, and keep B = 3. The first usable row is bin 3 (we need two earlier bins). Its row is (5, 2, 3): now, one back, two back. Bin 4 gives (1, 5, 2) and bin 5 gives (4, 1, 5). Each row slides one step to the right in time.
The trick that keeps the maths of Chapter 3 intact: treat each (neuron, lag) pair as its own column. The design matrix grows from T × N to T × (N · B). Then the normal equations are unchanged. Nothing about the solver knows that columns are lagged copies.
This is the Wiener filter of the decoding literature. Glaser and colleagues (eNeuro, 2020) describe it as multiple linear regression that predicts the output from multiple time bins of spikes of all neurons. An engineer calls the same thing a finite impulse response (FIR) filter: a weighted sum over a window of recent inputs. One weight track per neuron slides over time.
Glaser and colleagues chose a window by dataset. For monkey motor cortex they used 700 ms: the current 50 ms bin and the 13 before it, which is 14 bins. With 164 neurons, that is 164 × 14 = 2,296 input columns. For monkey somatosensory cortex they used 650 ms: the current bin, 6 before and 6 after (13 bins), so 52 × 13 = 676 columns. For rat hippocampus they used 2 s of 200 ms bins: the current bin, 4 before and 5 after (10 bins), so 46 × 10 = 460 columns.
Notice the second case includes bins after the output. Somatosensory neurons respond to the limb after it moves, so the useful spikes come later. A motor neuron leads the movement and a sensory neuron follows it. Choose the window to match which way the causation runs.
Now watch it in the toy. In our simulation, each neuron leads the hand by a different delay between 0 and 5 bins (0 to 250 ms), which is made up. Add lags and look at the weights. Each neuron puts its weight at its own delay.
Left: the weight of each neuron (row) at each lag (column). Orange is positive and teal is negative, brighter means larger. Right: held-out R² for each window size. Drag the number of lags.
Simulated: 12 neurons, 800 training bins (40 s), 500 test bins, noise 2.5 spikes per bin, ridge strength 10. Each neuron leads the hand by its own random delay of 0 to 5 bins, built into the simulation. Real delays differ by area and by animal.
With one bin, the decoder sees only the present, so it has to compromise: neurons with a long lead are just off. With a few lags it can pick the right moment for each neuron. The gain slows once the window covers the longest delay in the population. More lags past that point add columns and no new information.
The symptom: test R² stops rising, or falls, while training R² keeps climbing. The measurement: count the columns. With 164 neurons and a 14-bin window the decoder has 2,296 weights. With 21 minutes of data at 50 ms bins (Glaser’s motor set) there are about 25,000 bins. That is only about 11 examples per weight. A noisy recording can fool a decoder with that many knobs. The fix is a smaller window, more data, or a penalty on the weights. The penalty is next.
Chapter 5
Add a small penalty for big weights, and watch a decoder that memorised its training data learn to generalise
A student preparing for an exam gets one practice paper with 200 questions. She memorises the answer to every question, word for word. On the practice paper she scores 100%. On the real exam, with new questions, she scores 45%. She learned the paper and not the subject.
A decoder can do the same. Give it 256 weights and only 200 bins of training data, and it can bend until it matches every wiggle of the training velocity, including wiggles that were pure noise. That is overfitting: a great score on seen data and a poor one on new data. We saw the sign of it in Chapter 3. The gap between training and test is the symptom.
A memorising decoder tends to use huge weights that cancel in pairs, like +40 and −39, to carve out noise. A decoder that generalises tends to use small, shared weights. So we change the score that least squares minimises. We keep the sum of squared residuals. We add a charge for the size of the weights.
Plain least squares has a weak spot. When neurons overlap, or when weights outnumber bins, the squared error has a long flat valley in weight space, and many weight sets fit equally well. Then X⊤X cannot be inverted, or its inverse is wildly unstable. Our 256 weights and 200 bins are exactly that case. The penalty tilts the valley so that one answer wins. This fix is called ridge regression. The same step as before (set the slope to zero) gives a small change to the normal equations. We add λ to the diagonal:
Reuse X⊤X = [[4, 2], [2, 2]] and X⊤y = (3, 2). Pick λ = 1. The new table is [[5, 2], [2, 3]]. Its determinant is 15 − 4 = 11. Its inverse is one eleventh of [[3, −2], [−2, 5]].
Multiply by (3, 2): w1 = (9 − 4) / 11 = 0.455 and w2 = (−6 + 10) / 11 = 0.364. Plain least squares gave 0.5 and 0.5. Both weights shrank, and the stronger overlap between the two neurons is what let ridge pull them in unevenly.
The cost of this is a worse fit on the training bins. The predictions become 0.818, 0.455, −0.818, −0.455. The squared error rises from 0.02 to 0.090, and training R² drops from 0.992 to 0.964. We paid 0.028 of training score. In return, the weights are smaller and less able to chase noise.
Glaser and colleagues state the assumptions behind ridge plainly. It assumes the output changes in proportion to the inputs, that the noise on the output is Gaussian, and that the weights themselves come from a Gaussian centred on zero. That last one is the leash in probability form: before seeing data, we believe small weights are more likely than large ones.
The number λ is a hyperparameter, a setting you pick before fitting. Picking it by the training score is useless, because the training score always prefers λ = 0. Picking it by the test score is cheating, because then the test set has helped train the model. The honest way is a third block, the validation set. Fit with several λ values, score each on validation, and keep the best. Then report the test score once. Glaser and colleagues tuned all their decoders this way, scoring candidate settings on a validation block.
One practical note: ridge charges every weight the same price, so features should have similar scale. Counts do. If your inputs mix units, scale each column to unit spread first.
Drag λ below. The simulated decoder has 32 neurons, 8 lags and only 10 seconds of training: 256 weights for 200 bins.
Top: R² on the training bins (orange line) and on the test bins (green line) as ridge strength grows. The dot marks your setting. Bottom: the size of each of the 256 weights, largest first. Drag the strength.
Simulated: 32 neurons, 8 lags, 200 training bins, 300 test bins, noise 3.5 spikes per bin, so 256 weights chase 200 examples. The slider is logarithmic: λ runs from 0.1 to about 31,600.
At λ = 0.1 the training score is almost perfect and the test score is poor. As λ grows the training score slips and the test score climbs. Past the best point both fall, because the leash is so tight the weights cannot carry the signal. This U shape is the central picture of regularisation. Our lessons on bias and variance and regularisation explain it in general.
Chapter 6
Remove neurons at random, refit, and read off how performance grows with the size of the population
You are planning a surgery. A thin array of electrodes will go into a person’s motor cortex, and each electrode adds cost, risk and wires. A colleague asks a fair question: how many neurons do we need before the cursor feels good? Ten? A hundred? A thousand?
You cannot answer by thinking. You can answer by an experiment on data you already have. Take a recording with many neurons. Throw most of them away. See how well the decoder does with the few that remain. Then put some back. The result is a curve, and it answers the question.
A neuron-dropping curve plots decoder performance against the number of neurons used. Here is the recipe.
The idea belongs to the era of the first large ensemble recordings. Wessberg and colleagues (Nature, 2000) recorded the simultaneous activity of large neuron populations spread over premotor, primary motor and posterior parietal cortex. They predicted arm paths in real time with linear and nonlinear algorithms. Their paper sits behind a paywall that this lesson could not open, so we quote only its abstract and none of its figures. Serruya and colleagues (Nature, 2002) show how small a population can be and still work: 7 to 30 motor cortex neurons were enough for a monkey to move a cursor at once.
Every neuron carries a little of the same hand signal plus its own private noise. Average two neurons and the noise shrinks. Average four and it shrinks more. If each neuron’s noise has a spread of 2 and the noises are independent, then the average of 4 has a spread of 2 divided by the square root of 4, which is 1. That is why the curve rises steeply at first.
Two things then slow it. First, returns diminish: going from 16 neurons to 32 adds far less than going from 2 to 4, because most of the independent noise is already averaged out. Second, there is a floor. Some of what the hand does is simply not in these neurons, and no number of copies of them recovers it.
Chapter 5 warned that weights can outnumber examples. The curve makes the damage visible. Below, with only 7.5 seconds of training data, plain least squares improves up to about 8 neurons and then falls apart, because 32 neurons with 5 lags is 160 weights for 150 bins. Ridge keeps climbing. With 60 seconds of data the two lines agree, because there are plenty of examples.
Each point averages 4 random subsets of neurons. The green line is ridge, the grey line is plain least squares. Choose how much training data the decoder gets, then drag the neuron count to read both lines.
Simulated: 5 lags, noise 3.5 spikes per bin, 300 test bins, ridge strength 300 against 0.01 for plain least squares, 4 random subsets per point (a real analysis uses many more). Points are drawn at 1, 2, 4, 8, 12, 16, 24 and 32 neurons; in between, the read-out uses the nearest drawn point.
Use the curve to make a choice, not just to admire. Find the knee, where adding neurons stops paying. In the long-data panel above, going from 8 to 16 neurons lifts the ridge score from 0.71 to 0.80, a gain of 0.09. Going from 16 to 32 lifts it from 0.80 to 0.85, a gain of 0.05 for twice the neurons. If each neuron costs real effort, the knee is the place to stop.
Glaser and colleagues report the same pattern on real data in a limited form. With only 10 neurons, modern methods beat the traditional ones by a significant margin on motor and somatosensory data. In their rat hippocampus data, no method predicted well from 10 neurons (mean R² below 0.25). The right population size depends on the brain area and on what you decode.
Also check the axis. The curve says how performance grows with this decoder and this much data. A better decoder, or a longer recording, can lift the whole curve. So a neuron-dropping curve compares a population with itself, never one lab’s curve with another’s.
The symptom: a bumpy curve that goes down as neurons are added. The measurement: repeat with different random subsets and see whether the dip moves. In our toy, the plain-least-squares dip is real and repeatable because it comes from the weights-to-bins ratio, and ridge removes it. A dip that wanders from run to run is subset luck. The fix is more repeats, and error bars, before any conclusion.
Chapter 7
Line up the linear decoder against Kalman filters, boosted trees and neural networks, and learn what a fair comparison needs
You have a bicycle that works. A friend shows you a motorbike. It is faster, but it needs fuel, a licence and a mechanic. Is it better? Better for what: a ride to the shop, or a race? The honest answer starts with, “it depends on the road, and on what you can maintain.”
The linear decoder is the bicycle. Over the years, groups built many motorbikes: state-space filters, tree ensembles, deep networks. Which one should you use? In 2020, Glaser and colleagues (eNeuro) ran a careful test to find out, and they released their code so others could repeat it.
They used three recordings. The first: a monkey’s motor cortex during reaching, 21 minutes, 164 neurons, 50 ms bins, predicting hand velocity from 700 ms of activity. The second: a monkey’s somatosensory cortex, 51 minutes, 52 neurons, 50 ms bins, with a window that included 6 bins before and 6 after. The third: a rat’s hippocampus, 75 minutes, 46 neurons, 200 ms bins, predicting position.
They compared ten decoders, and also an ensemble method. The Wiener filter is our lagged linear regression. The Wiener cascade fits that line and then a fitted static curve after it, so it can bend. The Kalman filter assumes the hand follows simple dynamics and the spikes are noisy views of it (see the Kalman filter lesson, and the Kalman decoder lesson for the BCI version). Support vector regression and XGBoost (a tree ensemble) are general nonlinear regressors. The feedforward network, the simple recurrent network, the GRU and the LSTM are neural networks. The last three read the history one bin at a time and keep a memory, so they take N features per step instead of N × B all at once. The tenth, Naive Bayes, scores how likely each hand position is given the spikes.
They scored the decoders with 10-fold cross-validation: in each fold, 80% for training, a contiguous 10% for validation and a contiguous 10% for testing. They chose hyperparameters by Bayesian optimisation on the validation score, and reported R² on the test block.
Neural networks and the ensemble did best. The traditional Wiener and Kalman filters trailed them. In the paper’s words, the LSTM explained more than 40% of the variance that a Wiener filter left unexplained, with R² values of 0.88, 0.86 and 0.62 against 0.78, 0.75 and 0.35. The feedforward network came almost as close as the LSTM in all areas. The Naive Bayes decoder was poor at velocity, with mean R² of 0.26 for motor cortex and 0.36 for somatosensory cortex, though it did relatively well at position in the hippocampus.
Now do the arithmetic that makes “more than 40%” concrete. Take the first pair, 0.78 and 0.88. The Wiener filter leaves 1 − 0.78 = 0.22 unexplained. The LSTM leaves 1 − 0.88 = 0.12. It removed 0.10 of the 0.22, which is 0.10 / 0.22 = 45%. For the second pair, 0.11 of 0.25 is 44%. For the third, 0.27 of 0.65 is 42%. Each is just over 40%.
That framing is more honest than a bare R² gap. Going from 0.78 to 0.88 sounds small. Removing 45% of the remaining error sounds large. Both are true. Pick the one that matches your use. For a cursor, the remaining error is what the user feels.
Pick a pair of scores from the paper, or set your own. The bar splits the hand’s variance into three parts. Read how much of the leftover error the better decoder removed.
The six scores are from the text of Glaser et al. 2020, which lists the LSTM values (0.88, 0.86, 0.62) and the Wiener values (0.78, 0.75, 0.35) without naming the dataset of each; we read them in the paper’s dataset order (motor, somatosensory, hippocampus). The per-decoder figure in the paper holds the full comparison. The sliders let you try other scores.
Glaser and colleagues also shrank the training set. With 2 minutes of data for the motor and somatosensory sets, or 15 minutes for the hippocampus, both modern methods still beat both traditional methods. With 1 minute (or 7.5 minutes for hippocampus), the Kalman filter was sometimes as good as the modern methods, but they still clearly beat the Wiener filter. So flexible models did not need huge data to win here.
Does that kill the linear decoder? Not quite. Two cautions come from the paper itself. First, in the neural networks the authors used dropout and not L1 or L2 penalties, and the part of the text we read does not state whether the Wiener filter had a ridge penalty. A tuned ridge baseline is a fairer bicycle. Second, the authors note that every method makes assumptions. Ridge assumes proportional change, Gaussian noise and Gaussian weights. A neural network makes assumptions too, just less visible ones. The better decoder is the one whose assumptions fit your data.
Chapter 8
See why a decoder that scores well on a recording can still drift as a cursor, and why a person at the controls changes the story
You steer a car on a straight road with your eyes shut, using a map in your head. For ten seconds you do fine. Then a small bias in your steering adds up and you drift into the verge. Now open your eyes. You see the drift and nudge the wheel back. The same hands, the same bias, a different result.
That is the difference between open loop and closed loop. In open loop, the output goes nowhere that can correct it. In closed loop, the output is seen by the user, who corrects errors as they appear.
Everything so far was open loop. We recorded a hand moving, saved the spikes, and asked the decoder to predict the velocity afterwards. The R² of Glaser and colleagues is of this kind. It says how well each moment’s velocity matches. It does not say what happens when you use the decoded velocity to move something.
A cursor does not show velocity. It shows position. The cursor’s position is the running total of the decoded velocities. This is the integral: add up all the steps. Small errors in each step add up too. A bias of 0.1 units in every bin becomes 0.1 × 400 = 40 units after 400 bins (20 seconds). An error that was invisible in R² becomes a cursor far from its target.
Run it below. A fixed ridge decoder predicts velocity for 20 seconds of held-out data. The blue line is the true hand position, the sum of the true velocities. The orange line is the cursor, the sum of the decoded velocities. Then add a user who watches the gap and pulls the cursor back toward the target a little each bin.
Blue is where the cursor should be. Orange is where the decoder puts it. The shaded band is the gap. Drag the correction slider from zero (open loop) upward (closed loop).
Illustrative toy: a ridge decoder (16 neurons, 5 lags, 300 training bins, noise 3.5 spikes per bin) predicts velocity on 400 held-out bins. The cursor moves by the decoded velocity plus a pull of the chosen fraction of the gap to the target each 50 ms bin. Real users are slower and noisier than this rule. Units are arbitrary.
At zero correction the cursor wanders from the target path even though the velocity R² is decent. With 10% correction per bin, the cursor stays within a small band. The decoder did not change. The loop did.
This is why closed-loop work mattered so much. Serruya and colleagues (Nature, 2002) decoded activity from just 7 to 30 motor cortex neurons with a linear filter. The monkey could use that signal at once to move a cursor to any new position in its workspace. It needed no long training period. The monkey could see the cursor, and it adjusted. Wessberg and colleagues (Nature, 2000) also used real-time predictions to drive robotic devices, locally and across the Internet. Serruya's monkeys watched the cursor they moved. For the robot arms of Wessberg, the abstract does not say that the animals saw the result, so we do not call that part closed loop.
Two things follow. First, a person can adapt to a decoder, not only the other way round. The brain learns the weights of the device. Second, a decoder trained on a hand movement may behave differently when the hand is no longer moving. Later work trains the decoder on the user’s intended movement, not the observed one. The intent is inferred from what the user tried to do with the cursor (Gilja and colleagues, Nature Neuroscience, 2012). The Kalman decoder lesson has the full idea. We mention it only as a pointer here.
A BCI decoder runs every bin, all day, on a small chip. Count the work. Take Glaser’s motor setup: 164 neurons, 14 lags, and two outputs (x and y velocity). That is 164 × 14 × 2 = 4,592 multiply-adds per bin. At 20 bins per second, that is 91,840 multiply-adds per second. A modern phone chip does billions per second, so a linear decoder is nearly free to run. The expensive part is the data collection and the calibration, not the arithmetic.
That cheapness is a real design asset. It means the decoder can sit on the implant or next to the electrode, send out only a velocity, and spend almost no power. A larger network costs far more per bin and needs more careful checking in the loop.
The symptom: a decoder with a great R² feels sluggish or drifty in use. The measurement: integrate its output over a trial and compare the endpoints, or log how often the user must correct it. The fix: test in closed loop with people at the controls, recalibrate during use, and treat the offline R² as a screening score, not the verdict.
Chapter 9
Keep the numbers, the checklist and the core code on one page, and see where linear decoders sit in the field
A linear decoder is a weighted sum of recent spike counts. You cut time into bins, count spikes, stack lagged copies of the counts into a design matrix, and solve a penalised least squares problem for the weights. You judge it on bins it never saw. You choose its history window and its penalty on validation data. You read the neuron-dropping curve to see how many neurons you need. And you remember that a person at the controls changes how well it works.
| Term or number | What it means | Where it comes from |
|---|---|---|
| Bin | A slice of time in which spikes are counted. 50 ms for motor and somatosensory cortex, 200 ms for hippocampus. | Glaser et al. 2020 |
| Design matrix X | Shape T × (N · B): one row per bin, one column per neuron at each lag. | Chapters 3 and 4 |
| Wiener filter | Multiple linear regression from several time bins of all neurons to the output. | Glaser et al. 2020 |
| Normal equations | w = (X⊤X)−1 X⊤y. Least squares in closed form. | Chapter 3 |
| Ridge | w = (X⊤X + λI)−1 X⊤y. Shrinks weights, tuned on validation data. | Chapter 5 |
| Motor window | 700 ms: the current bin and 13 before. 164 neurons give 2,296 columns. | Glaser et al. 2020, arithmetic ours |
| Motor data | 21 minutes, 164 neurons, 50 ms bins, about 25,200 bins. | Glaser et al. 2020, arithmetic ours |
| LSTM against Wiener | R² 0.88, 0.86, 0.62 against 0.78, 0.75, 0.35: over 40% of the unexplained variance removed. | Glaser et al. 2020, text |
| Small populations | 7 to 30 motor cortex neurons gave a monkey immediate cursor control. | Serruya et al. 2002, abstract |
| Run cost | 164 neurons × 14 lags × 2 outputs = 4,592 multiply-adds per bin. | Chapter 8, our arithmetic |
The whole lesson fits in a few lines of NumPy. This is a sketch in the spirit of the Wiener filter, not Glaser and colleagues’ own package (theirs is at the link below).
pythonimport numpy as np def bin_spikes(spike_times, n_neurons, t_end, dt=0.05): # spike_times: list of arrays, seconds. Returns counts, shape (T, N) edges = np.arange(0, t_end + dt, dt) return np.stack([np.histogram(s, edges)[0] for s in spike_times], axis=1) def lagged(X, B): # (T, N) -> (T, N*B): the count now and B-1 bins back, for every neuron T, N = X.shape pad = np.vstack([np.zeros((B - 1, N)), X]) return np.hstack([pad[B - 1 - l : B - 1 - l + T] for l in range(B)]) def ridge_fit(F, y, lam): mu, ym = F.mean(0), y.mean() # centre, so the intercept is not penalised Fc, yc = F - mu, y - ym w = np.linalg.solve(Fc.T @ Fc + lam * np.eye(F.shape[1]), Fc.T @ yc) return w, mu, ym def predict(F, w, mu, ym): return (F - mu) @ w + ym def r2(y, yhat): return 1 - ((y - yhat) ** 2).sum() / ((y - y.mean()) ** 2).sum() # X: (T, N) counts at 50 ms, y: (T,) hand velocity. Contiguous blocks, no shuffling. F = lagged(X, B=14) tr, va, te = slice(0, 800), slice(800, 900), slice(900, 1000) best = max([1, 10, 100, 1000, 10000], key=lambda lam: r2(y[va], predict(F[va], *ridge_fit(F[tr], y[tr], lam)))) w, mu, ym = ridge_fit(F[tr], y[tr], best) print("test R2:", r2(y[te], predict(F[te], w, mu, ym)))
Shapes: X is T × N, F is T × (N · B), w is (N · B) × 1 and the output is T × 1. With two velocity axes, y has two columns and w has two columns. Nothing else changes.
The linear decoder is the base of a ladder. One rung up, the Kalman filter adds a model of how the hand moves, so each estimate leans on the last one. The Kalman decoder lesson shows it for BCIs. RTS smoothing and adaptive Kalman filters cover offline and drifting cases. Another rung up, networks learn their own nonlinear features, and in the largest recent efforts they pretrain on many sessions. The ladder is long, but its first rung is the baseline every other rung is measured against.
For the supporting maths, see linear regression, regularisation, bias and variance, and generalized linear models for the encoding direction. To find structure among many neurons before decoding, see PCA.
Now press Present or Teach and explain it from memory: how spikes become a design matrix, why lags help, what ridge changes, and how you would choose the number of neurons. Then go back to the decoder at the top and try each method again.
This lesson was built from primary sources, not slides.
All recordings in the devices are simulations written for this lesson. Their numbers are illustrative, and each device says so.