Spike Sorting

One electrode hears several neurons at once. Sorting cuts out each spike, turns its shape into a point, and groups the points. Add noise and the groups merge.

Learn how a recording of a few thousand voltage wiggles becomes a list of separate neurons, and when a brain-computer interface can skip that step.

Pick how many groups to look for, then drag the noise up. Watch the clouds touch and the neurons blur together. Then we build, piece by piece, everything underneath: the threshold, the alignment, the features, the clustering, the templates and the drift.

You need a feel for averages and spread (the mean, and how far values stray from it) and a little Python for the code. We build the rest from zero.

Lesson 9 of the series Brain-Computer Interfaces, from the spike to the foundation model. Built from primary sources: Quian Quiroga, Nadasdy and Ben-Shaul, Neural Computation 2004; Pachitariu, Steinmetz, Kadir, Carandini and Harris, NeurIPS 2016; Pachitariu, Sridhar, Pennington and Stringer, Nature Methods 2024; Trautmann et al., Neuron 2019.

Sort three neurons

18 µV

Illustrative: 450 made-up spikes, 150 from each of three model neurons with trough depths of 100, 78 and 60 µV, sampled at 30 kHz with 32 samples each. The two features are the first two principal components (Chapter 2), the grouping is k-means (Chapter 4), and both run live in your browser. Fill is the group the computer found, ring is the neuron that really fired.

Chapter 0

One Wire, Many Voices

See why one electrode records a crowd of neurons, and what it means to sort that crowd back into single cells

Picture a microphone hung in the middle of a crowded canteen. You hear a roar. Inside the roar, a few people near the microphone speak clearly. Others, far away, make only a murmur. You want a transcript for each person, but the microphone gives you one sound.

An electrode in the brain is that microphone. It sits in the fluid between cells and records one voltage over time. A nearby neuron fires a brief electrical pulse, a spike (also called an action potential), and the electrode sees a dip of some tens of microvolts that lasts about one millisecond. Another neuron a little further away makes a smaller dip with a different shape. All the dips land on the same wire.

Spike sorting is the job of working out who said what. Given one noisy voltage trace, it returns a list for every neuron: the times at which that neuron fired. A cell found this way is called a unit. A unit that we trust to be exactly one neuron is a single unit. A pile of spikes from several neurons that we could not pull apart is a multi-unit.

Why anyone cares which neuron it was

Suppose two neurons sit beside one electrode. One fires when you plan to move your hand left. The other fires when you plan to move it right. If you count all their spikes together, the two effects cancel and you learn nothing. If you sort them, you see two clean signals. Science often asks questions about one cell, so the field spent decades getting good at sorting.

A brain-computer interface (BCI) asks a different question: what does the whole group say, and can I decode it fast? Sometimes sorting helps. Sometimes it costs more than it gives. Chapter 10 settles which. First we build the machine, because you need to know what you are skipping before you skip it.

The pipeline in one picture

Every classical sorter follows the same chain, and so does this lesson. Each stage takes a table of numbers and returns a smaller, more useful one. Here are the real shapes for one probe with C channels and T samples.

  1. Filter and detect. Input: the voltage table, [T × C]. Output: a list of times where something spiked (Chapter 1).
  2. Cut and align. Output: one short snippet per spike, [N × 32] for N spikes at 32 samples each (Chapter 1).
  3. Features. Output: a few numbers per spike, [N × 2 to 10] (Chapters 2 and 3).
  4. Cluster. Output: one group label per spike, [N] (Chapter 4).
  5. Check. Keep the groups that behave like one neuron (Chapter 5).

Chapters 6 to 9 then fix what this chain gets wrong: spikes that overlap, a neuron seen on many channels, a probe that moves, and a recording so big that the chain is too slow. That is the road to Kilosort4.

First, hear the problem for yourself. The trace below holds three neurons plus noise. Turn them on and off.

Mix three neurons on one wire

The trace is 0.4 seconds of one electrode. Each neuron you switch on adds its spikes. The dashed line is a threshold at four times the noise. Turn on Show who fired to colour each spike by its neuron.

Illustrative: three model neurons firing at about 25, 40 and 18 spikes per second with trough depths of 100, 78 and 60 µV, on top of 12 µV of noise, sampled at 30 kHz. The crossing count is computed from the trace you see, one count per dip, with a one millisecond dead time.

With everything on, the trace is a forest of dips. Switch on Show who fired and the forest sorts itself into three colours, but the electrode never gave you the colours. It gave you only the voltages. Sorting is the art of guessing the colours from the shapes.

Two facts make that guess possible. First, a neuron fires the same shape each time, because its cell body, its width and its distance from the electrode do not change from one spike to the next. Second, different neurons usually differ in shape, in size or in both. Chapters 1 to 4 turn those two facts into an algorithm. Everything after Chapter 4 is about what breaks when the facts are only nearly true.

The road

  1. Cutting the spikes out: a noise-proof threshold and an alignment that lines spikes up.
  2. From shape to a point: principal components keep what tells neurons apart.
  3. Features at many scales: wavelet coefficients chosen by a normality test.
  4. Grouping the points: k-means, and the temperature knob of wave_clus.
  5. Is it one neuron?: the refractory-period check.
  6. Spikes on top of spikes: template matching and matching pursuit.
  7. Many wires, one neuron: probe footprints, common-average reference, whitening.
  8. When the brain moves: drift estimation and correction.
  9. Sorting at scale: the Kilosort4 pipeline, data rates and results.
  10. When to skip sorting: threshold crossings and random projections.
  11. Field guide: numbers, checklist, code and links onward.
What does an extracellular electrode give you, before any sorting?

Chapter 1

Cutting the Spikes Out

Find each spike with a noise-proof threshold, then line the spikes up so they can be compared

You have a wire and half a second of recording. It looks like a hairy line with a few sharp dips. Your first job is plain: say where the dips are. Then cut each one out and stack the cut-outs so they can be compared. This chapter does both, and shows the two ways it goes wrong.

Step one: remove what is not a spike

A raw recording holds slow drifts that are not spikes, such as the brain's slow electrical swells. A spike is fast, so a high-pass filter removes the slow part and keeps the fast part. Kilosort4's default is a high-pass at 300 Hz. The older wave_clus code filters between a lower and an upper limit that you set. Lesson 7 of this series covers filters in depth.

Step two: a threshold that ignores the spikes themselves

The simplest detector draws a horizontal line below the noise and calls every dip that crosses it a spike. The hard part is where to put the line. Too high and you miss the small neurons. Too low and noise crosses it by chance.

A natural plan is to measure the noise by its standard deviation (SD, the typical distance of a value from the average) and to put the line at a few SDs. The trap is that the spikes are in the trace you are measuring. A big spike is a very large value, and a few of them inflate the SD. The busier the electrode, the higher the line climbs, and the more spikes slip under it.

Quian Quiroga and colleagues fixed this with the median, the middle value when you sort a list. A few huge spikes cannot move the middle of a long list. For bell-shaped noise with SD σ, half of all values lie within 0.6745 σ of zero, so the median of the absolute voltage equals 0.6745 σ. Turn that around and you can read σ from the median.

σn
The estimated SD of the noise alone, in microvolts.
median(|x|)
Take the absolute value of every filtered sample, sort, and read the middle one. Spikes are too rare to move it.
0.6745
The share of one SD that contains half of a bell curve's values. It turns a median into an SD.
Thr
The detection line. A spike is a dip below −Thr.
k
A multiplier you choose. The papers and tools use about 4 to 5. wave_clus names it stdmin.

Take nine absolute values in microvolts: 2, 5, 7, 8, 9, 11, 13, 14 and 90. The 90 is a spike. The middle value is 9, so σn = 9 / 0.6745 = 13.3 µV, and with k = 4 the line sits at 53.4 µV. The 90 µV spike crosses it. Now use the plain SD of the same nine numbers instead: 25.8 µV, and four times that is 103 µV. The line has climbed above the spike that raised it, and the spike is lost. Try it with more spikes in the device below.

Set the line, catch the spikes

The trace holds spikes from three model neurons. Green ticks mark caught spikes, red marks mark missed ones. Raise the firing rate, then switch between the two ways of estimating the noise.

300 /s
4.0

Illustrative: 0.5 s of a made-up channel with 10 µV of true noise and the three model neurons from the first device, summed. The estimates are computed from the trace you see. A spike counts as caught when a detection falls within 10 samples (0.33 ms) of its true trough.

At 300 spikes per second the plain SD reads about 17 µV while the truth is 10, and the line drifts up until a third of the spikes pass underneath it. The median reads about 11.5 µV and catches them all. This is the failure as symptom, measurement and fix. Symptom: a busy channel yields fewer units than a quiet one. Measurement: compare the SD estimate with the median estimate. Fix: use the median.

Step three: cut and align

Once a dip is found, cut a short window around it. wave_clus calls the sizes w_pre and w_post: the number of samples kept before and after the spike's peak. We keep 32 samples in all, about one millisecond at 30,000 samples per second. Each spike becomes one row of 32 numbers, so N spikes make a table of shape [N × 32].

Now comes a catch. Where exactly do you cut? The threshold is crossed on the way down, and a big spike crosses early while a small one crosses late. If you align on the crossing, two copies of the same neuron's spike sit a sample or two apart, and they look like different shapes. The fix is to align on the trough itself, the deepest sample.

One more catch: the real trough often falls between two samples. The sampling grid is 33 µs apart, and the trough lands anywhere within it. wave_clus interpolates the spike with cubic splines (it fits a smooth curve through the samples) and finds the trough on the finer curve. Compare three alignments below.

Line the spikes up

Each line is one cut-out spike of the same small neuron. Pick how to align them. The tighter the stack, the more the shapes agree.

4 µV

Illustrative: 80 spikes of the smallest model neuron (60 µV trough) with 30% spread in size, random positions between samples, a threshold of four times the noise, and 40 of them drawn. The error is the aligned position minus the true trough time, its SD taken over all spikes that crossed the threshold. The interpolation is a Catmull-Rom cubic at eight points per sample.

The gains are fractions of a sample, and that is the point. A shift of half a sample is invisible on a plot. It still smears every cluster in the chapters ahead, because every feature we compute will read a slightly different part of the shape. Alignment is cheap, so do it carefully.

An electrode is very busy. You set the threshold from the plain SD of the trace and lose many spikes. What is the most direct cause?

Chapter 2

From Shape to a Point

Turn each 32-number spike shape into two numbers that keep what tells neurons apart

You now hold a table with one row per spike and 32 numbers per row. You want to see which spikes belong together. A person can look at a flat plot with two axes, not at 32. So you must squeeze each row down to two or three numbers without losing what tells one neuron from another.

Each spike is a point in a big room

Here is the picture that makes the rest easy. A spike has 32 voltage samples. Treat each sample as one direction in space, and the whole spike becomes one point in a room with 32 directions. Two spikes with similar shapes land close together. Spikes from one neuron land in a small cloud, because that neuron fires nearly the same shape each time. A second neuron makes a second cloud somewhere else.

You cannot draw a room with 32 directions, but you can look at it from one angle. A good angle shows the clouds apart. A bad angle stacks them on top of each other, like two people who look alike from the front and different from the side. The question is how to choose the angle.

Two ways to pick two numbers

The simplest plan is to pick two samples by hand, say the depth of the trough and the height of the rebound 7 samples later. That works when the neurons differ in exactly those places and fails when they do not.

The better plan is principal component analysis (PCA). It looks at all the spikes together and asks: along which direction in the room do the points spread out the most? That direction is the first principal component (PC1). It then asks for the next direction, at a right angle to the first, with the most spread left over. That is PC2. Think of photographing a long cigar-shaped cloud: you get the most information by looking along the cigar's length. (Our PCA lesson derives this from first principles. Here is the short version.)

PCA needs the covariance matrix C, a 32 by 32 table. Entry (i, j) says how much sample i and sample j move together across all spikes. Its directions of largest spread are its eigenvectors, and the spread along each one is its eigenvalue. A spike's score on a component is its dot product (multiply matching entries, then add) with that direction.

C
The covariance matrix, shape [32 × 32].
(1/N) Σ (x − μ)(x − μ)T
For each spike x, take its distance from the average spike μ, multiply it by itself to get a 32 by 32 table, and average over the N spikes.
v
A unit-length direction: an eigenvector of C. PC1 is the one with the largest eigenvalue.
score
One number per spike per component: how far this spike sits along v. Two scores make the two axes of the plot.

Do it by hand on four spikes with only two samples each, (trough, rebound) in microvolts: (−100, 28), (−98, 30), (−78, 40) and (−76, 38). The mean is (−88, 34). The distances from the mean are (−12, −6), (−10, −4), (10, 6) and (12, 4). Averaging the products gives a variance of 122 for the trough, 26 for the rebound, and a covariance of 55. So C = [[122, 55], [55, 26]].

The eigenvalues of that matrix are 147 and 1. PC1 therefore holds 147 of the total 148 units of spread, which is 99.3%. Its direction is (0.91, 0.41): the trough and the rebound move together, so a big neuron is deep and tall. The first spike scores −12 × 0.91 + −6 × 0.41 = −13.4. Four points in two dimensions became four points on one line, and the two neurons stayed apart.

The real thing has the same steps with 32 samples. The data flow is: [N × 32] table, subtract the mean spike, build C of shape [32 × 32], take the top two eigenvectors [2 × 32], multiply to get [N × 2] scores. Try it against the hand-picked samples.

Pick the angle

450 spikes from three neurons, each drawn as a point. Fill is the group k-means finds, ring is the true neuron. Top strip: the shape of the two directions you chose. Add noise and compare the three choices.

15 µV

Illustrative: the same 450 model spikes as the first device. “Two samples” means the trough (sample 10) and the rebound (sample 17). Groups come from k-means with three groups; the score is the share of spikes whose group is named for their own neuron. The spread figure is the eigenvalue of each component divided by the total variance of the 32 samples.

At 15 µV of noise, the two hand-picked samples sort about 71% of the spikes to the right neuron, and the two principal components sort about 98%. PCA wins because it uses all 32 samples at once. Each sample carries a little of the neuron's signature, and PCA adds the little bits together, while the noise on the samples averages out.

Where PCA fails

PCA looks for the largest spread, not for the best separation. Those are the same only when the spread comes from the neurons. Suppose a few spikes are huge artifacts, or a slow wave leaks through the filter. They stretch the cloud, and PC1 turns toward the artifact instead of toward the neurons.

The symptom is clouds that smear along one axis, or a PC1 shape that looks like a ramp instead of a spike. The measurement is to plot the component shapes, as the device does in its top strip. A spike-shaped PC1 is healthy. The fix is cleaner upstream work: filter harder, align carefully (Chapter 1), reject outliers, and add more components. The next chapter shows a way to choose features by looking at their distribution, not their spread.

Why does PCA usually separate neurons better than two hand-picked samples of the spike?

Chapter 3

Features at Many Scales

Choose shape features by how un-bell-shaped they are, the idea behind wave_clus

PCA asked which direction has the most spread. That question has a weakness: spread comes from many places, and only some of it separates neurons. Quian Quiroga and colleagues asked a different question. Which single numbers about the spike come out in clumps, one clump per neuron? Their answer used wavelets.

A wavelet is a stack of averages and differences

Start with the simplest wavelet, the Haar wavelet. Take the spike's samples two at a time. For each pair, record their average and their difference. The averages form a shorter, smoother copy of the spike, half as long. The differences record what the averaging threw away. Now repeat on the shorter copy, again and again, until one number is left.

Try it on four samples: −100, −40, 20, 28. The pair averages are −70 and 24, and the pair differences are −30 and −4 (half the gap, first minus second). Repeat on (−70, 24): the average is −23 and the difference is −47. The four numbers we keep are −23, −47, −30 and −4. They hold exactly the same information as the four samples, and you can rebuild the samples from them.

What did we gain? Each number now speaks about a different scale and a different place. −47 compares the first half of the spike with the second half, a coarse feature. −30 compares only the first two samples, a fine feature at the start. For a 32-sample spike there are five rounds. That gives 32 coefficients: one overall level, then 1, 2, 4, 8 and 16 differences from coarse to fine. (Our wavelets lesson builds the general case. The real wave_clus code lets you choose the number of scales.)

PCA mixes all 32 samples into every component. A wavelet coefficient looks at one stretch of the spike only. If two neurons differ in a narrow place, such as how fast the rebound starts, one coefficient can carry that difference alone.

Choosing coefficients without labels

Thirty-two coefficients are too many to cluster by eye, so wave_clus keeps only a few. Here is the clever part. The rule uses no knowledge of which spike belongs to which neuron. It asks, for each coefficient, whether its values across all spikes look like a bell curve.

Why a bell curve? If every spike came from one neuron, a coefficient would vary only by noise, and noise is bell-shaped. If a coefficient takes one typical value for neuron A and another for neuron B, its histogram has two bumps, which is not bell-shaped. So a coefficient that departs from a bell curve probably separates neurons. Quian Quiroga's code measures the departure with the Kolmogorov-Smirnov test.

The test is easy to picture. Sort the coefficient's values and rescale them to mean 0 and SD 1. Walk along them, tracking the share of spikes seen so far. Compare that running share with the share a bell curve would have reached at the same point. The largest gap is the statistic D. Zero means a perfect bell. A larger D means a stronger departure. wave_clus ranks the coefficients by D. Its manual says the “Plot all projections” button draws the ten coefficients with the largest deviation from normality, which make 45 pairs.

Choose features by their shape

Top: one bar per Haar coefficient, coarse on the left and fine on the right, as tall as its departure D from a bell curve. Below: the spikes in the two coefficients chosen. Pick a method, or choose Your picks and tap two bars.

12 µV

Illustrative: the same 450 model spikes, orthonormal Haar transform over five scales, D computed from the 450 values of each coefficient against a normal curve with the same mean and SD. Groups come from k-means with three groups, scored as in the last device. On this smooth toy, PCA does as well as or better than the wavelet picks; the text below says why that is not the whole story.

Run it. At 12 µV the best wavelet pair sorts roughly nine in ten spikes correctly, and PCA does a little better. Tap two bars yourself and you can often match the automatic choice, and sometimes beat it.

That result is worth stating plainly. Our toy neurons have smooth, similar shapes, and PCA suits smooth shapes. Wavelets pay off when neurons differ in short, sharp features, or when the biggest source of spread is something other than the neurons. The 2004 paper reports that the wavelet and superparamagnetic method beat conventional methods on simulated data that closely resemble real recordings; the wave_clus manual points to that paper for the comparison of features. Treat the two feature families as tools, and test both on your own data.

When the rule misleads

The rule has a weak spot. A coefficient can depart from a bell curve for reasons that have nothing to do with neurons: a few huge artifacts, or an amplitude that jumps during a burst of firing. The symptom is a top-ranked coefficient whose scatter shows no clumps, only a long tail. The measurement is to look at the scatter of the top pairs, as wave_clus invites you to do. The fix is to reject artifacts first and to compare against PCA.

Why does wave_clus keep the wavelet coefficients that are furthest from a bell curve?

Chapter 4

Grouping the Points

Group spike points with no labels, and turn the temperature knob of superparamagnetic clustering

Now every spike is a point on a flat plot, and the points form clumps. Your eye finds the clumps in a second. A computer needs a rule. The job is called clustering: split the points into groups so that points in one group are close and points in different groups are far. It is unsupervised, which means nobody tells the computer the right groups. That is exactly our situation, because the electrode never told us which neuron fired.

The classic rule: k-means

The oldest recipe is k-means. You choose k, the number of groups. The computer drops k centres onto the plot and repeats two steps. First, give each point to its nearest centre. Second, move each centre to the average of the points it just received. When no point changes group, stop.

Work it on six numbers: 1, 2, 4, 9, 11, 12, with k = 2 and starting centres 1 and 2. Round one: 1 goes to centre 1; 2, 4, 9, 11 and 12 all go to centre 2. The new centres are 1 and 7.6. Round two: the point 2 is closer to 1 than to 7.6, and so is 4 (3 away, against 3.6). So centre 1 takes 1, 2 and 4, and centre 2 takes 9, 11 and 12. The new centres are 2.33 and 10.67, and nothing moves again. The rule found two groups from a poor start.

Every device so far has used it. It is fast and simple, but it asks three things of the data. You must know k, and for neurons you do not. The clumps must be roughly round and similar in size. And one distant outlier can drag a centre away. A soft cousin, the Gaussian mixture fitted by EM, relaxes the roundness; see our EM and Gaussian mixture lesson.

The wave_clus answer: a temperature knob

Quian Quiroga's method borrows an idea from physics. In superparamagnetic clustering (SPC), developed by Eytan Domany's group, every spike is a tiny magnet. Each magnet wants to point the same way as its nearest neighbours. Heat fights that urge by jostling the magnets. The temperature sets the balance.

The wave_clus manual describes the result in plain terms. At a low temperature, all the spikes are assigned to a single cluster. At a high temperature, each spike forms its own cluster. The best clustering lies between these extremes, in what physicists call the superparamagnetic regime. SPC makes no assumption that clusters are round or Gaussian, and it does not need you to pick k.

In practice, wave_clus runs SPC at every temperature between a minimum and a maximum, in steps you set, and saves them all. It picks the largest temperature at which a cluster with at least min_clus members appears. If you disagree, the main correction is to choose another temperature, which only means loading a different saved result. Two more facts from the manual matter. SPC is stochastic, so two runs can differ slightly. And it becomes slow beyond about 30,000 spikes, so the tool clusters the first max_spk spikes and then assigns the rest by template matching, which Chapter 6 explains.

The output shape is simple. Input: the feature table, [N × features]. Output: a list of N whole numbers, one per spike, where 0 means “not assigned to any cluster” and 1, 2, 3 and so on name the clusters, plus the spike times in milliseconds.

The device below is an honest stand-in. We do not run the physics model. We use a simpler rule with the same single knob: link two spikes when they are closer than a distance, and call each connected chain a cluster. A tight distance (hot) leaves spikes alone. A loose distance (cold) chains everything into one blob. Watch the same three regimes appear.

Turn the temperature

Top: the spikes, coloured by cluster. Grey rings are spikes in no cluster of at least 20. Force membership hands each grey spike to the nearest cluster, unless it is too far. Bottom: how many such clusters exist at each setting. Slide from cold to hot.

0.70
15 µV

Toy stand-in, not the Potts model that SPC runs: two spikes are linked when they are closer than a distance that shrinks from 60 to 1.5 feature units as the knob goes from 0 to 1, and each chain is a cluster. The rule that a cluster needs at least 20 members mirrors wave_clus’s min_clus setting; the value 20 is ours. Force membership gives a grey spike to the cluster of its nearest clustered spike when that spike is within 25 feature units, in the spirit of the wave_clus Force button. Features are the first two principal components of the 450 model spikes.

Cold on the left: one big blob holds every spike, so three neurons look like one. Hot on the right: the clouds crumble into dust, and no cluster has enough members to count. In the middle, around 0.65 to 0.75, the clumps separate: three or four clusters stand, and a small fragment can join them. The cost is that more and more spikes sit in the grey, which is why wave_clus offers a Force step that assigns them to the closest cluster unless they are too far. Raise the noise and the middle window shrinks, because the clouds touch.

Over-splitting and merging, as symptom and fix

Too cold gives the merge error. Symptom: a unit whose average waveform is a blur of two shapes. Measurement: its spikes show two bumps along a feature, or it breaks the timing rule of the next chapter. Fix: raise the temperature, or split by hand. Too hot gives the split error. Symptom: several small units with almost the same waveform. Measurement: their average shapes match, and their spike times never overlap suspiciously. Fix: lower the temperature, or merge them. Chapter 5 gives the tests.

In superparamagnetic clustering, what happens at a very low temperature?

Chapter 5

Is It One Neuron?

Check a cluster against the rule that one neuron cannot fire twice within a few milliseconds

The clustering step hands you a label for every spike. A label is a claim: “these 700 spikes came from one neuron.” Before you trust the claim, you need a test that does not look at the shapes again. Shapes were the evidence that made the cluster. We need a second, independent kind of evidence. Timing supplies it.

A neuron needs a rest

After a neuron fires, it cannot fire again straight away. Its ion channels need time to reset. This pause is the refractory period, and it lasts around a millisecond or two. So in the spike times of one real neuron, two spikes almost never sit closer than that.

That gives a test with no shapes in it. List the spike times of a cluster. Take the gap between each spike and the next, the inter-spike interval (ISI). Draw a histogram of the gaps. For one neuron, the bars below the refractory time are empty. If several neurons share the cluster, they fire independently of each other, and some of their spikes will land close together. Those show up as short gaps. wave_clus prints, on every cluster, the number of spikes with an ISI under 3 ms, and its manual calls a relatively large count a sign of a multi-unit cluster.

How many short gaps should contamination make?

We can predict the count. Say the good neuron A fires at a spikes per second, and a contaminating neuron B at b spikes per second, independently. Choose a short time τ. A pair of neighbouring spikes closer than τ must include a B spike, because A cannot fire twice inside τ. An A spike has an A-then-B neighbour within τ with probability about bτ. A B spike has any neighbour within τ with probability about (a + b)τ. Add up the rates of the two cases.

short gaps per second
How many neighbouring spike pairs sit closer than τ each second.
τ
The window, in seconds. wave_clus uses 3 ms.
2ab
The good neuron, rate a, next to the contaminant, rate b, in either order.
b²
Two contaminant spikes next to each other.
share of spikes
Short gaps per second divided by all spikes per second, a + b.

A worked case: a = 10 spikes per second, b = 1 per second, τ = 0.003 s. Short gaps per second = 0.003 × (2 × 10 × 1 + 1) = 0.063. All spikes per second = 11. The share is 0.063 / 11 = 0.57%. In one minute, about 660 spikes and 3.8 short gaps. One spike in eleven came from the wrong neuron, and the test sees only 4 flags in 660. The test is sharp for strong contamination and weak for light.

Count the short gaps

A good neuron fires about 10 times a second. Slide in spikes from a second neuron and watch the empty zone before 3 ms fill up. Red bars are gaps shorter than 3 ms.

9%

Illustrative: 120 s of spikes. The good neuron has a 3 ms dead time and fires near 10 per second. The second neuron fires at random (a Poisson process), independently. The prediction uses the formula above with τ = 3 ms. Real neurons in bursts can fire closer than 3 ms, so a few short gaps do not always mean contamination.

At 9% contamination the histogram shows a few red bars, and the formula predicts the count to within the random scatter of so few events. At 0% the zone is empty. This is the test working. Now the part that matters for practice: the test is a one-way street.

What a clean histogram does not prove

A clean ISI zone means no evidence of contamination. It does not prove one neuron. A light contaminant, say 1% of the spikes, makes only a handful of flags in a minute, and the count is random. Symptom: a unit that passes the test but decodes oddly. Measurement: record longer, because the number of flags grows with time while the unit’s noise averages out. Fix: set an explicit limit on the share of short gaps and test it on the full recording.

The opposite error also hides. If clustering splits one neuron into two groups, each group is clean on its own. The split shows only when you look across the two groups: the pair has no short gaps between them, as if they took turns, and their waveforms match. Kilosort4 uses a refractory test on such cross-correlograms (counts of how often one cluster fires at each delay from the other) when it decides whether to merge two clusters.

The same hole in a second picture

Many tools draw the autocorrelogram instead of the interval histogram. For every spike of a cluster, it counts the other spikes of the same cluster at each delay before and after it, and it plots the counts against the delay. It carries the same message. A single neuron leaves a hole around delay zero, and the hole is as wide as the refractory period. A mixed cluster has no hole, because some neighbours are strangers that owe the neuron nothing.

The picture helps when you must decide quickly. If the hole is deep and wide, trust the cluster more. If the hole is shallow, check the waveforms before you believe it. Treat the number as a score you report, not as a verdict you hide.

A cluster has many spike pairs only 1 ms apart. What is the best reading?

Chapter 6

Spikes on Top of Spikes

Separate overlapping spikes by fitting known shapes and subtracting them one at a time

Two people at the canteen talk at the same moment. Your ears hear one jumbled sound. If you had only ever learned the sound of a single voice, you would call the jumble a stranger. Spikes do the same. When two neurons fire within about a millisecond of each other, their voltages add, and the sum looks like no neuron you know.

How often do spikes collide?

Count the collisions. Let all the neurons on a channel fire together at R spikes per second, at random times. Call two spikes overlapping if they come within a window w of each other, where w is about the length of one spike, 1 ms. The chance that a given spike has a neighbour within w on either side is about 1 − e−2wR. At R = 100 spikes per second and w = 0.001 s, that is 1 − e−0.2 = 18%. Almost one spike in five is tangled. A dense probe with busy neurons sees it constantly.

Threshold-and-cluster handles these spikes badly. A tangled spike lands between clouds in feature space, or beyond the edge of both, and either the tool drops it or gives it the wrong label. Dropping spikes biases every statistic that counts them. Worse, collisions happen mostly when neurons fire together, which is exactly the signal that population decoding wants.

Treat the voltage as a sum

The 2016 Kilosort paper (Pachitariu, Steinmetz, Kadir, Carandini and Harris) starts from a different picture. The recording is not a list of separate spikes. It is a sum of copies of a small set of shapes, each copy placed at a spike time and scaled. The authors state that, because the model is a sum, overlapping spikes can be identified and separated. The shapes are called templates.

x(t)
The recorded voltage.
Σk
Add up one term for every spike k, from all neurons.
ak
How strongly spike k shows up, near 1 when it matches the template.
Tu(k)
The template of the neuron that fired spike k, a short waveform on every channel.
tk
The time of spike k.

Sorting now means solving for the spike times, the amplitudes and the neuron labels so that the sum matches the recording. The paper notes that this works on the raw data, where earlier methods first compressed spikes by PCA, and that doing so gives more accurate waveform models.

Find one, subtract it, look again

Solving it exactly is hard, so the practical method is greedy: matching pursuit. Slide each template along the recording and measure how well it fits at every position. Take the best fit anywhere. Subtract that scaled template. Look at what is left, and do it again. Kilosort4 repeats this for up to 50 rounds, subtracting each detected spike to resolve overlaps.

The fit has a clean formula. If the data window is x and the template is T, the best amplitude is a = (x · T) / (T · T), and the energy removed by subtracting is (x · T)² / (T · T). Try three numbers: T = (−2, −5, −1) and x = (−1.9, −5.2, −1.1). The dot x · T = 3.8 + 26 + 1.1 = 30.9. T · T = 30. So a = 1.03. The leftover x − aT = (0.16, −0.05, −0.07): almost nothing. The template explains the window.

Now an overlap. When two spikes collide, the best fit takes the larger one first. After subtracting, the smaller one stands clear in what is left, and the second round finds it. Step through it below.

Find, subtract, repeat

Top: the recording, two neurons firing close together. Middle: the templates found so far, placed where they fit. Bottom: what is left. Drag the gap, then press Next round.

10 samples

Illustrative: neuron 1 and neuron 2 from the earlier devices fire a given number of samples apart (one sample is 33 µs) on 6 µV of noise. Matching pursuit here tries every template at every sample position, lets each scale between 0.75 and 1.3 times its size, and subtracts the best match. The real Kilosort4 works on many channels at once and uses learned templates.

With a gap of 10 samples (0.33 ms) the recording shows one broad, odd dip. Round one takes the better of the two matches. Round two finds the other spike in the leftover. Round three has nothing worth taking, and the loop stops. Now squeeze the gap below 8 samples (0.27 ms). The greedy rule starts to explain the pair with the wrong templates, because one big dip can be read as many different pairs. Spikes that coincide almost exactly are the hard case for every sorter, and no amount of subtraction creates information that the sum has lost.

The new failure mode

Templates must match. If a neuron's shape changes, the subtraction leaves a ghost behind, and the ghost looks like a small spike. Symptom: a unit with a swarm of tiny extra detections right after each of its real spikes. Measurement: the residual after subtraction is larger than the noise, and it has the shape of the template's error. Fix: update the templates, and correct for the probe's drift (Chapter 8) before matching.

Why can template matching recover two overlapping spikes that a threshold-and-cluster pipeline loses?

Chapter 7

Many Wires, One Neuron

Use a neuron's footprint across a probe, and remove noise that every channel shares

Now hang a whole pole of microphones through the canteen, one every few centimetres. A person speaking near microphone 4 is loud there, quieter on microphones 3 and 5, and faint further away. The pattern of loudness across the pole is a fingerprint. Two people at different places make different fingerprints, even if their voices sound alike. Modern probes give neurons exactly that fingerprint.

What a probe records

A probe is a thin shank with many recording sites along it. The Neuropixels probe (Jun et al., Nature 2017) has 960 sites on a 10 mm shank, and a base chip that filters, amplifies and digitises. Any 384 of the sites can be recorded at once, each as a channel. The recording is a table of shape [T × 384]: one row per sample in time, one column per channel.

When a neuron fires, the spike shows on the nearest channels and fades with distance. Its footprint is the set of amplitudes across channels. The footprint gives two gifts. First, more evidence: a spike is now judged on many channels, so two neurons with similar shapes on one channel still differ in where they sit. Second, a position: the neuron is near the loudest channels. A template in a probe is therefore a small table of shape [channels × time], not a single trace.

A quick way to estimate position is the centre of mass of the amplitudes: the average of the channel depths, each weighted by its amplitude. Take five channels at 0, 20, 40, 60 and 80 µm with amplitudes 5, 30, 80, 55 and 10 µV. The weights add up to 180. The weighted depths add up to 0 + 600 + 3,200 + 3,300 + 800 = 7,900. So the neuron sits near 7,900 / 180 = 43.9 µm, a little past the middle channel because the 60 µm neighbour is louder than the 20 µm one.

Noise that every channel shares

A probe brings a new problem. Interference such as mains hum, a moving cable or the reference electrode shows up almost the same on every channel at once. A spike, in contrast, is local. So we can remove what all channels agree on. The step is common average referencing (CAR): at each moment, subtract one common value from every channel. The robust choice is the middle value, the median. Kilosort4's preprocessing also begins with common average referencing, then applies a temporal filter, then whitening, then drift correction.

Why the middle value, not the average? Take six channels at one instant: 20, 19, 21, 20, −40 and 21 µV. One channel holds a spike. The average is 10.2, so subtracting it leaves about 10 µV of shared noise on the quiet channels. The median is 20. Subtracting it leaves 0, −1, 1, 0, −60 and 1: the shared part is gone and the spike survives. The median ignores the one loud channel, the same trick as in Chapter 1.

Read a neuron's footprint

Left: eight channels, 20 µm apart, over 8 ms. Right: the spike's size on each channel and the position guessed from it. Slide the neuron along the probe, then turn on the shared interference and remove it.

62 µm

Illustrative: spike size on a channel falls as 100 µV × 15² / (distance² + 15²) for a neuron 15 µm off the probe line. This law is ours, not measured. Interference is a slow 34 µV wave shared by all channels. Each channel also has 5 µV of its own noise. The depth guess is the amplitude-weighted average of channel depths, computed from the trace you see.

Move the neuron and the bars shift with it. The depth guess follows, with errors of a few micrometres in the middle of the probe and more near its ends, where no channel lies beyond the neuron to balance the average. Turn on the interference and every trace rides the same slow wave, which makes the spike hard to see on the quiet channels. Subtract the median and the wave vanishes. The spike stays, a little smaller, because the median also sees the spike on the few channels that carry it. That is CAR in one gesture.

Whitening: noise that leans on its neighbours

One more shared problem remains. The background hum from far-away neurons is not independent across channels. Neighbouring channels pick up the same far-away activity, so their noise is correlated. A detector or a template match that ignores this treats a bump on channel 5 and its echo on channel 6 as two separate events.

Whitening is a linear step that makes the noise uncorrelated, with equal spread on every channel. You estimate the noise covariance matrix C and multiply the data by C−1/2. Take two channels whose noise has correlation 0.8, C = [[1, 0.8], [0.8, 1]]. Its eigenvalues are 1.8 and 0.2. The matrix C−1/2 works out to [[1.49, −0.75], [−0.75, 1.49]]. Multiply two channels that moved together and the shared part shrinks, while differences between the channels grow. The specific version Kilosort4 uses is ZCA whitening, computed from each channel's nearest 32 channels. Its article notes that this neighbourhood may need to grow for ultra-dense probes.

The failure mode is whitening a recording that contains strong spikes in the noise estimate. Symptom: real spikes shrink after whitening. Measurement: compare the spike amplitude before and after. Fix: estimate the noise covariance from the quiet stretches, or from robust statistics, in the same spirit as the median.

Why do probe pipelines subtract the median across channels at each moment?

Chapter 8

When the Brain Moves

Track a neuron that slides past the probe and slide the recording back into place

You have built a careful sorter. You run it on the first ten minutes and it finds twenty clean units. You run it on the next ten minutes and half the units are gone and twenty new ones have appeared. The neurons did not change. The probe moved.

Why the picture slides

Brain tissue is soft. It pulses with the heartbeat, swells with each breath, and settles slowly against an implant over minutes, hours and days. A probe fixed to the skull sees the tissue slide past it. A neuron that sat at depth 62 µm now sits at 80 µm. Its footprint moves to different channels. A template tuned to the old position no longer fits. This slow shift is drift.

The Kilosort4 paper names it as one of the two things that make sorting hard: nonstationarity, which means the recording’s statistics change with time, and overlapping electrical fields from nearby neurons. Drift ruins sorting quietly. Clusters smear along the depth direction. A unit seems to fade out. Its twin appears later under a new label. Counting units, or following one neuron over an hour, becomes impossible without fixing it.

Put a number on the damage with the footprint law of Chapter 7. In that toy, a spike on its best channel is 100 µV at the centre of the footprint, and the size falls as 15² / (d² + 15²) with the distance d. A drift of 40 µm, two channel steps, leaves the old best channel at 225 / (1,600 + 225) = 12% of its old size. A template that expects 100 µV there now sees 12. The neuron is the same, and the match is poor. The law is our toy, but any footprint that falls with distance shows the same trouble.

Estimate the slide from the spikes themselves

The fix is called registration: measure how far the tissue moved at each moment, then move the data back. You do not need a separate sensor. The spikes tell you.

Detect spikes, and for each one estimate its depth along the probe (the centre-of-mass trick from Chapter 7 does it). Cut time into short bins; Kilosort4 uses 2-second bins for its drift estimate. In each bin, count spikes at each depth to make a depth histogram. A probe with several steady neurons gives a histogram with several bumps. When the tissue slides, all the bumps slide together. So compare each bin’s histogram with a reference histogram, and find the shift that makes them overlap best.

The comparison is a cross-correlation: slide one histogram over the other, multiply the matching bars, add them up, and keep the shift with the biggest sum. A tiny case: reference bars [0, 5, 0, 3, 0]. A later bin shows [0, 0, 5, 0, 3]. At shift 0, the products are all zero, so the score is 0. At a shift of one bin, the bars line up: 5 × 5 + 3 × 3 = 34. The best shift is one bin, so the tissue moved one bin and we move the data back by one bin.

Moving the data back means resampling the signal at shifted channel positions. Kilosort4 uses linear interpolation for this, where earlier versions used a method called Makima. It estimates the drift in advance and applies the alignment on demand, one batch at a time. Try the whole idea below.

Straighten the drift

Each dot is one spike: time across, depth up. Seven steady neurons should draw seven flat bands. Increase the drift, then switch on the correction. The teal line is the estimated drift.

40 µm

Illustrative: 7 model neurons at fixed depths firing 10 to 20 spikes per second for 120 s, depth measured with 3 µm of error. The whole probe slides together (rigid drift) as a slow wave plus a step. The estimate uses 2 s bins, 2.5 µm depth bins and a cross-correlation against the first 4 seconds. Real drift is rarely this rigid.

At 40 µm of drift the bands wave up and down, and a band from one neuron crosses the place where its neighbour used to sit. Switch the correction on and they flatten. The spread of one neuron’s depths falls from tens of micrometres to a few, and the bands are separate again.

Where registration stops

The Kilosort4 article lists its own limits, and they teach the method. Tetrodes and single electrodes have no well-defined geometry along a line, so there is little to register. A vertical site spacing above 40 µm limits drift correction. Drift that is faster than the 2-second bin is not well handled by the estimate, though the article reports that its clustering still held up. Chronic recordings across many days were not explicitly tested.

A physical limit sits beside these: the tissue does not always move as one block. If the top of the probe slides differently from the bottom, a single offset per moment is wrong for part of the probe, and it needs a separate estimate for each stretch of depth. The symptom, the measurement and the fix make a pattern. Symptom: units that fade and reappear. Measurement: the drift map of spike depth against time. Fix: register first, then sort.

How does a sorter estimate drift without any extra sensor?

Chapter 9

Sorting at Scale

Follow the Kilosort4 pipeline from raw samples to units and see where the time and the bytes go

Put the pieces together and run them on a real probe for an hour. The first surprise is the size. Thirty thousand samples per second on 384 channels is a flood of numbers: tens of gigabytes before sorting even starts. Everything we built has to run on that flood in about the time it took to record. That is the problem Kilosort was made for.

How the field got here

The 2016 paper (Pachitariu, Steinmetz, Kadir, Carandini and Harris, NeurIPS) set the direction. Its claims, in its own abstract: template matching is used for both detection and clustering; it models the voltage as a sum of templates, so overlapping spikes can be separated; it works on raw data rather than PCA-compressed spikes; and batch-based optimisation on GPUs makes it faster than earlier algorithms. A merging step based on template continuity reduces manual work. The authors present this as a step toward fully automated sorting of multichannel recordings.

Kilosort4 (Pachitariu, Sridhar, Pennington and Stringer, Nature Methods 2024) is the newest version. Its new part is the clustering: it is inspired by graph-based approaches, and the abstract says it performed best in all the simulated conditions the authors tested, including neurons with low amplitudes and small spatial extents under high drift.

The pipeline, stage by stage

The stages below follow the order in the Kilosort4 article. Step through them to see what goes in and what comes out at each one, and to see how big the data is. Each stage is something you have already built, scaled up.

Walk the pipeline

Pick a stage. The panel shows what goes in, what comes out, and the detail the article gives. Then set the probe and the recording time to see how much data flows through.

384
1 hour

Stage facts are from the Kilosort4 article (Pachitariu et al., Nature Methods 2024): the 300 Hz high-pass, ZCA whitening from the nearest 32 channels, linear-interpolation drift alignment, 46,080 simple templates for Neuropixels 1 and 50 matching-pursuit rounds, a nearest-neighbour graph with 200 k-means++ starting clusters, and merges tested with refractory correlograms. The data size is our arithmetic: 30,000 samples per second, 2 bytes per sample (an assumption), times channels times seconds. The batch shape uses the 65,536-sample batch the article names for its IBL data and 4 bytes per sample after filtering (our assumption).

One hour on 384 channels is about 83 GB at 2 bytes per sample. A batch of 65,536 samples (a bit over two seconds) as 4-byte numbers is about 101 MB, small enough to sit on a GPU. That is why batching works: the stages up to matching only look at a short time window at a time, so the recording can stream through. Only the spike list, a few numbers per spike, has to be held whole.

Why the clustering became a graph

K-means needs k and round clumps. The graph approach needs neither. Each spike becomes a node, and each node links to its nearest neighbours in feature space. A cluster is then a group of nodes that link mostly to each other. The measure of that is modularity: the share of links inside groups, minus the share you would expect if links were wired at random.

Work it on two triangles joined by one bridge: 6 nodes, 7 links. Each triangle has 3 internal links and its degrees (links per node) add to 7. For each group: 3/7 − (7/14)² = 0.429 − 0.25 = 0.179. Add the two groups: modularity 0.357. If you merged all six nodes into one group, modularity is 0, so splitting is better. The article starts from 200 k-means++ clusters and moves nodes between neighbours to raise this score, then builds a tree of merges. It subsamples 25,000 spikes per section to keep this fast.

The article checks each proposed split or merge with two tests. One asks whether the points are bimodal (two bumps) along the line between the two groups. The other is the refractory test of Chapter 5, run on the cross-correlogram of the pair.

What the numbers say

The article scores a sorted unit against ground truth as 1 − false positives − false negatives, with a spike matching when the times agree within 0.2 ms, and counts a unit as identified above 0.8. On hybrid simulations from three low-drift recordings with 300 ground-truth units, all Kilosort versions except Kilosort1 beat the other algorithms and Kilosort4 was best. On drift simulations built from 11 recordings with at least 40 µm of drift, IronClust generally found about 50% of the units, against 80 to 90% for Kilosort4. Every algorithm ran within twice the duration of the simulated recordings.

The same article is frank about limits. Chronic multi-day recordings were not explicitly tested. Some stages repair problems that others leave, so the benefit of each stage varies by recording. And users should still check results by hand in Phy, the curation tool. The failure mode of a fast automatic sorter is trust. Symptom: many units whose refractory-violation rate is high. Measurement: that rate per unit, which the article uses with a cut-off of 0.2 to call a unit “good”. Fix: curate, and report which units you kept.

Why can a sorter keep up with a 384-channel, 30 kHz recording?

Chapter 10

When to Skip Sorting

Decode from threshold crossings and learn when merging neurons costs nothing

Imagine a person who moves a cursor with their mind. Every 20 milliseconds the decoder needs fresh numbers from the implant. A full sorting pipeline wants the whole recording, a GPU and a curator. The implant has a tiny power budget, the radio link is narrow, and the cursor must not lag. So a design question follows: do we need to know which neuron fired, or only that something near this electrode fired?

The cheap alternative: count threshold crossings

The cheap route skips Chapters 2 to 9. Detect spikes with the threshold of Chapter 1, but do not cut them out or cluster them. For each channel and each short time bin, say 20 ms, just count the crossings. The output is a table of whole numbers with shape [bins × channels], and that is the decoder’s input. Each count is the activity of a multi-unit, every neuron near that electrode added together.

The data flow shrinks a lot. Take 96 channels. The raw signal is 96 × 30,000 samples per second × 2 bytes, about 5.76 MB every second. Counts in 20 ms bins are 50 per channel per second, so 4,800 numbers per second, about 4.8 kB if each fits in a byte. That is roughly 1,200 times less to send, store and decode. The figures are our arithmetic, and the byte sizes are our assumption.

This is also friendly to hardware: a comparator and a counter are far cheaper than filters, feature extraction and clustering. A company-authored 2019 platform paper from Neuralink, “An Integrated Brain-Machine Interface Platform With Thousands of Channels” (J Med Internet Res 21(10):e16194), is reported in press coverage to describe detecting spikes on the chip itself. Treat any chip figures as company claims; this lesson has not checked them against the paper.

Why merging neurons can cost so little

Trautmann and colleagues (Neuron 2019) asked the question directly: does combining the neurons on each electrode, without sorting, distort estimates of the population’s state and dynamics? They recorded from monkey motor cortex with Neuropixels probes and re-analysed data from three earlier studies. Using multi-unit threshold crossings in place of sorted neurons gave similar dynamics and the same scientific conclusions.

Their argument uses random projection theory. Picture the population as following a few hidden knobs, the low-dimensional dynamics. Every neuron is a weighted copy of those knobs plus noise. A channel that merges three neurons is another weighted copy: a different mix, but still a mix of the same knobs. A hundred such mixes of ten knobs still carry all ten knobs. Merging throws away neuron identity but keeps the shared structure, provided there are enough channels compared with the number of knobs.

The argument has a price tag, and the toy below shows it. Each model neuron prefers a direction of motion, and fires faster when the hand moves that way. A linear decoder turns spike counts in 50 ms bins into the 2-D hand velocity. Compare three inputs: every neuron sorted, every channel merged, and sorted with a share of spikes given to the wrong neuron of the same channel.

Sort, merge or sort badly

Lines show how well a linear decoder recovers velocity on fresh data, as the number of channels grows. Choose how many neurons share a channel and how often sorting is wrong. The marker shows your channel count.

8
3
30%

Toy, not data: each neuron has a random preferred direction, a random baseline of 6 to 12 and a gain of 4 to 8 spikes per second per unit of speed, counts in 50 ms bins (Poisson), velocity drawn from a 2-D bell curve, ridge regression trained on 800 bins and scored on 800 new ones (the score is R², 1 is perfect). Sorting errors move that share of spikes to a random neuron on the same channel. The curves use one random draw per setting, so small gaps are noise.

Read the picture in three parts. With one neuron per channel the lines coincide, because merging changes nothing. With three neurons per channel and few channels, sorting buys a lot. As channels grow, the merged line climbs toward the sorted one: more mixes, less lost. And bad sorting is not free: at 30 to 50% errors the sorted line falls toward the merged one, because the neuron identities it pays for are partly wrong.

When sorting still earns its cost

Merging suits decoding population activity. It does not suit every question. To ask what one cell prefers, how cell types differ, whether two cells are connected, or how a single neuron changes over weeks, you need units. Sorting also helps when channels are few and each carries several useful neurons, as the toy shows.

It has costs of its own in a BCI. A sorted unit’s identity can change between days when drift moves the probe, and a decoder trained on yesterday’s labels then reads the wrong inputs. Threshold crossings fail more gently: the channel is still the channel. This is part of why recalibration across days is such a practical problem, and why the decoder lessons (linear decoders and the Kalman decoder) take it seriously. Symptom: a decoder that degrades each morning. Measurement: compare channel-level counts across days. Fix: decode from stable channel-level inputs, or realign the latent space each day, a topic for later in the series.

In the toy, why does the merged-channel decoder catch up with the sorted decoder as channels increase?

Chapter 11

Field Guide

Keep the numbers, the checklist, the core code and the next lessons

Spike sorting turns one noisy voltage trace into a list of neurons. It detects spikes with a threshold that spikes cannot inflate, cuts and aligns them, turns each shape into a few features, groups the features, and checks each group by timing. Template matching handles collisions. Probes add footprints, referencing and whitening. Registration undoes drift. Kilosort4 does all of it at scale. Many BCIs skip the grouping and count threshold crossings instead.

The numbers and terms

ItemValueSource
Noise estimatemedian(|x|) / 0.6745Quian Quiroga et al. 2004; wave_clus manual
Detection linek times the noise estimate, k about 4 to 5 (wave_clus calls it stdmin)wave_clus manual; the exact k is your choice
Spike window32 samples at 30 kHz, about 1 ms, trough at sample 10Our choice; wave_clus sets it with w_pre and w_post
AlignmentOn the trough, after cubic-spline interpolationwave_clus manual
Wavelet featuresCoefficients picked by Kolmogorov-Smirnov distance from a normal curve; the manual also allows the first 3 principal componentswave_clus manual
Clustering temperatureLow: one cluster. High: every spike alone. Best: in between. Results saved for every temperaturewave_clus manual
Slow regimeSuperparamagnetic clustering slows past about 30,000 spikes; cluster the first max_spk, template-match the restwave_clus manual
Multi-unit flagMany gaps under 3 ms in a clusterwave_clus FAQ
Overlap rate1 minus e to the power of minus 2wR; 18% at 100 spikes per second and w = 1 msOur derivation from random firing times
Neuropixels 1384 channels chosen from 960 sites on a 10 mm shankJun et al., Nature 2017
Kilosort4 front endCommon average reference, 300 Hz high-pass, ZCA whitening from the nearest 32 channels, drift correctionPachitariu et al., Nature Methods 2024
Kilosort4 matching46,080 simple templates (Neuropixels 1), 50 matching-pursuit roundsPachitariu et al. 2024
Kilosort4 clusteringNearest-neighbour graph, 200 k-means++ starts, subsample of 25,000 spikes per sectionPachitariu et al. 2024
Drift estimate2 second bins; faster drift is not well handledPachitariu et al. 2024
Score1 minus false positives minus false negatives; identified above 0.8; spike match within 0.2 msPachitariu et al. 2024
Drift benchmarkAbout 50% of units found by IronClust, 80 to 90% by Kilosort4, in 11 drifting recordingsPachitariu et al. 2024
Merged channelsThreshold crossings gave similar dynamics and conclusions in three re-analysed studiesTrautmann et al., Neuron 2019
Data rate384 channels × 30,000 samples/s × 2 bytes = 23.04 MB/s, about 83 GB per hourOur arithmetic; 2 bytes per sample is our assumption

Company claims in this lesson: the Neuralink chip description in Chapter 10 comes from a company-authored paper and press coverage, and we did not check its figures.

A checklist before you trust a sorted unit

  1. Filtered. High-pass applied; slow waves gone.
  2. Thresholded by the median. The line does not rise on busy channels.
  3. Aligned on the trough, with interpolation.
  4. Features inspected. Several pairs of components or coefficients, not only the first two.
  5. Setting justified. k, or the temperature, chosen by looking, not by habit.
  6. Timing checked. The share of gaps under 3 ms is small, and you reported it.
  7. Collisions handled. Template matching, or a note that overlaps were dropped.
  8. Drift map looked at. Spike depth against time is flat, or was registered.
  9. Curated. You kept a list of the units that passed and the ones that did not.
  10. Question matched. Single neurons need units; population decoding may need only crossings.

The core as code

This is the classic chain, short enough to read in one go. Input: a filtered voltage trace [T]. Output: for each spike its time and its group. It is a teaching sketch for one channel, not a replacement for Kilosort4.

python
import numpy as np
from scipy.signal import butter, sosfiltfilt
from sklearn.cluster import KMeans

FS, PRE, POST = 30_000, 10, 22                    # a spike is 32 samples, about 1 ms

def detect(x, k=4.0):
    """x: [T] filtered voltage in uV. Returns trough indices and the threshold."""
    sigma = np.median(np.abs(x)) / 0.6745         # noise SD that spikes cannot inflate
    thr = k * sigma
    below = x < -thr
    onset = np.flatnonzero(below[1:] & ~below[:-1]) + 1      # first sample under the line
    idx = []
    for i in onset:
        j = i + int(np.argmin(x[i:i + 8]))        # align on the trough, not the crossing
        if PRE <= j < len(x) - POST and (not idx or j - idx[-1] > 30):
            idx.append(j)
    return np.array(idx), thr

def cut(x, idx):
    return np.stack([x[i - PRE:i + POST] for i in idx])      # [N, 32]

def features(snips, n=2):
    z = snips - snips.mean(0)
    _, _, vt = np.linalg.svd(z, full_matrices=False)         # PCA by SVD
    return z @ vt[:n].T                                      # [N, n]

def isi_flags(times_s, tau=0.003):
    return float(np.mean(np.diff(np.sort(times_s)) < tau))   # share of gaps under 3 ms

# usage, with raw a [T] voltage in microvolts sampled at FS:
x = sosfiltfilt(butter(3, 300, 'high', fs=FS, output='sos'), raw)
idx, thr = detect(x)                       # trough times, line height
F = features(cut(x, idx), 2)               # [N, 2]
labels = KMeans(3, n_init=5, random_state=0).fit_predict(F)
for c in range(3):
    print(c, (labels == c).sum(), isi_flags(idx[labels == c] / FS))

Where this sits in the field

Quian Quiroga’s 2004 pipeline (threshold, wavelets, superparamagnetic clustering) is the classic single-channel answer. The 2016 Kilosort paper moved to templates and GPUs for probes with hundreds of channels, and Kilosort4 in 2024 added graph-based clustering and stronger drift handling. In BCIs, much practice moved to channel-level counts, with sorting kept for science. The next lessons use these inputs: implant compression asks how many bits per spike the radio can carry, and linear decoders and the Kalman decoder read the counts you just learned to make. For the tools behind this chapter, see PCA, wavelets, Gaussian mixtures and the Kalman filter. The step before this one is spike detection.

Sources

  1. R. Quian Quiroga, Z. Nadasdy, Y. Ben-Shaul. Unsupervised spike detection and sorting with wavelets and superparamagnetic clustering. Neural Computation 16(8):1661-1687, 2004. doi. We read the abstract and the author’s wave_clus manual for every parameter named here.
  2. M. Pachitariu, N. A. Steinmetz, S. N. Kadir, M. Carandini, K. D. Harris. Fast and accurate spike sorting of high-channel count probes with KiloSort. NeurIPS 2016. abstract page. We read the abstract.
  3. M. Pachitariu, S. Sridhar, J. Pennington, C. Stringer. Spike sorting with Kilosort4. Nature Methods 21(5):914-921, 2024. doi, open access. We read the first part of the text, not the supplement or the final pages of the drift simulations.
  4. J. J. Jun et al. Fully integrated silicon probes for high-density recording of neural activity. Nature, 2017. doi. We read the abstract record.
  5. E. M. Trautmann et al. Accurate estimation of neural population dynamics without spike sorting. Neuron 103(2):292-308, 2019. doi. We read the abstract record.
  6. E. Musk and Neuralink. An integrated brain-machine interface platform with thousands of channels. J Med Internet Res 21(10):e16194, 2019. doi. Company-authored; not read in full for this lesson.

Spike sorting, and every device on this page, is written from those sources and from our own toy simulations. All numbers marked illustrative are made up to teach the idea. The model neurons, the noise levels and the decoding task are not data.

The takeaway. A sorted unit is a claim built from three kinds of evidence: shape, position and timing. Know which of them each stage uses, and which question needs the claim at all.

Go back to the three neurons at the top and push the noise up. Now you know which stage fails first, and what repairs it. Try Teach mode to explain the pipeline on a whiteboard, or open the Sparks for a fast review.

BCI 09: Spike Sorting
Previous: spike detection · Next: implant compression · Back to Gleams