CEBRA draws a map of brain activity by pulling together moments that share a behavior and pushing the rest apart. It read a rat’s position to within 5.8 cm, where pi-VAE had reported 12 cm.
Learn how a small neural network turns hundreds of noisy neurons into a few numbers that mean something, the same few numbers for every animal.
Pick what pulls the points together, then press Train 300 steps. Watch a ring form when the pull signal is real, and fail when it is shuffled. Then we build, piece by piece, the pairs, the loss, the encoder, the tests and the results.
You need the idea of a vector (a list of numbers) and what a neuron’s firing rate is. We build the rest from zero.
Paper: Steffen Schneider, Jin Hwa Lee and Mackenzie Weygandt Mathis (EPFL), Learnable latent embeddings for joint behavioral and neural analysis, arXiv 2204.00673 (this lesson follows version 2, October 2022), published in Nature in 2023. Every idea is re-taught here in our own words and devices. This paper belongs to the lecture Latent Variable Models for Neural Data.
A toy lab, not the paper’s data: 16 simulated neurons with sharp place fields and added noise, a rat running 3 laps of a circular track in 600 time bins, a linear encoder with 2 outputs scaled to length 1, 96 positive and 96 negative samples per step, temperature 0.3, the Adam optimizer. Position is read back with a nearest-neighbour decoder (k = 9) fit on laps 1 and 2 and scored on lap 3. A guess at random is off by 90 degrees (median). The paper’s numbers are in Chapter 8.
Chapter 0
See why a list of noisy firing rates hides a simple shape, and what a good map must do that older tools could not
A rat runs back and forth along a track 1.6 meters long. A thin probe in its hippocampus (a deep brain region that helps an animal know where it is) listens to many neurons at once. Every 25 milliseconds, you write down how many times each neuron fired.
That count is the neuron’s firing rate in one time bin, a short slice of the recording. If the probe hears 16 neurons, one time bin is a list of 16 numbers. A list of numbers is a vector, and this one is the population vector: what the whole group said in that slice.
One recording gives thousands of these vectors, one per time bin. Your question is simple. What do they say about the rat? Where is it? Which way is it running? Is it the same story on every lap?
Some hippocampal neurons are place cells. Each one fires most when the animal stands at one place on the track, its favourite spot. If you know where the rat is, you can guess which neurons are loud. If you hear which neurons are loud, you can guess where the rat is.
The device below uses a simulated version of this. The track is a circle, so position is an angle from 0 to 360 degrees. Slide the rat around the track and watch the 16 bars. Then press Next lap. The rat stands at the same place, yet the bars change, because neurons are noisy.
Left: how loud each neuron is at this position (bars for the lap you picked, dots for the other two laps). Right: every time bin as one dot, seen through just two neurons. The rings mark this position on each lap.
Simulated, the same 16-neuron toy as the hero: sharp place fields plus noise, 3 laps, 200 time bins per lap, each neuron’s average subtracted. The paper’s rats ran a straight 1.6 m track and each rat had its own number of neurons. Dot color is track position.
The right panel is the problem in one picture. Two neurons give a flat view of a 16-dimensional cloud. The dots overlap, loop over each other and smear. The shape is in the cloud, but no pair of neurons shows it cleanly.
A latent embedding is a short list of numbers, say 3, that stands in for the long list of 16 or 100. Latent means hidden: the rat’s position was never in the data, yet the neurons carry it. Embedding means each time bin gets a point in a small space. A good embedding puts bins from the same place close together.
The authors list what they want from such a map. It should be nonlinear, because the link between neurons and behavior bends. It should use behavior and time, because both are always recorded. It should be consistent: two animals, or two runs of the method, give the same map. And it should not need a generative model, an explicit recipe for how hidden causes produce spikes.
PCA (principal component analysis) lays a flat sheet through the cloud along its widest directions. It is easy to read, but a flat sheet cannot follow a bend, and the paper notes that this costs performance. See our PCA lesson.
tSNE and UMAP are nonlinear and make pretty cluster pictures. But they ignore the order of time bins, though time is always known. They also drift when you stack animals together: the paper says concatenating animals can shift the clusters.
pi-VAE and LFADS are generative. A VAE (variational autoencoder) learns by rebuilding its input from a hidden code; see our VAE lesson. These models assume how the data is made, and the authors argue they do not guarantee the same map for every animal.
The name says it: Consistent EmBeddings of high-dimensional Recordings using Auxiliary variables. An auxiliary variable is any extra recorded signal: the rat’s position, a trial number, the time stamp, or features of a movie the animal watched. The method is a form of nonlinear ICA (independent component analysis, which looks for hidden sources; see ICA) trained by contrastive learning (our lesson).
Chapter 1
Measure whether two maps of the same brain function agree, and see which changes keep them the same map
Two friends each draw a map of the same city. One puts north at the top. The other turns the sheet sideways and uses miles instead of kilometers. The maps look different, yet every street is in the same place relative to the others. They are the same map.
A third friend draws a map where the river bends the wrong way and two districts swap places. That one is a different map. The rule that separates the two cases is the heart of this chapter, and it is what the authors mean by consistent.
An embedding turns each time bin into a short list of D numbers. D is the embedding dimension. With D = 3, each bin is a point in 3-D space, and a recording of 15,000 bins is a cloud of 15,000 points.
CEBRA scales every point to length 1. The points then lie on a hypersphere, the set of all points at distance 1 from the origin. For D = 2 that is a circle. For D = 3 it is the surface of a ball. This is why the paper says that 3 dimensions of CEBRA are equal to 2 dimensions of the other methods: the length-1 rule removes one degree of freedom.
Why a sphere? It gives a clean way to compare two points: the angle between them. Chapter 2 uses exactly that.
The paper records four rats. Each rat has different neurons under the probe, and a different number of them. So the raw input differs from rat to rat. Yet all four ran the same track. If CEBRA works, each rat should give the same map of the track.
But a trained network has no reason to pick the same orientation twice. One run may put the start of the track at the top of the ring, another at the bottom. The authors do not ask for that. They ask for something weaker that is still useful: the two maps must match after a linear change. A linear change is any mix of rotating, mirroring, stretching and shifting. It never bends or tears.
This is called linear identifiability. Earlier theory (Hyvärinen and colleagues, and Roeder and colleagues, cited in the paper) shows that contrastive models trained to convergence on the same data agree up to such a linear map. A downstream tool that is itself linear, such as a linear regression, then gives the same score on either map.
To measure it, take two embeddings of the same moments, A and B. Fit the best linear map from A to B, with an intercept (a shift). Then ask how much of B the fit explains. The score is R².
B = 1, 2, 3, 4; fit = 1.1, 1.9, 3.2, 3.8: misses 0.1, 0.1, 0.2, 0.2 give 0.01 + 0.01 + 0.04 + 0.04 = 0.10; mean 2.5 gives 2.25 + 0.25 + 0.25 + 2.25 = 5.00; so R2 = 1 − 0.10 / 5.00 = 0.98
The paper uses this score in two ways. Across subjects: embed rat A and rat B, fit one to the other, and report R² for all pairs (12 pairs of four rats). Across runs: train the same method 10 times on the same data and compare every ordered pair (90 comparisons). A method that is consistent has high scores and little spread in both.
The left ring is map A. The right ring is map B, made by changing A in the way you choose, plus a little noise. The score is the R² of the best linear fit from A to B. Try each change and push the slider.
Illustrative: 120 points on a circle, map B gets noise of 0.03 on each coordinate. Rotate turns B by up to 180 degrees. Stretch squeezes one axis down to 25% and shears. Twist adds a position-dependent turn of up to 1.6 radians (a nonlinear bend). Scramble swaps the order of points by an amount you set. R² is computed live with a least-squares fit with intercept.
Rotate, stretch and shear all keep R² near 1. Twist and scramble pull it down. The linear fit forgives a change of viewpoint. It does not forgive a change of shape.
A low score has a symptom: two rats give rings with different structure, or one run fails to close the loop. The measurement is R² between embeddings of the same behavior bins. A common fix is more data or the right labels. Chapter 7 shows a second fix: train on all animals at once so their maps share one space.
Chapter 2
Teach a network a map by telling it which moments are friends and which are strangers
Spread a day’s photos across a table. Pick one photo. Find another taken a minute later, and place the two side by side. Now take a handful of photos from other days and slide them to the far edge. Do this a thousand times, with a different photo each time, and the table sorts itself by the time of day.
Nobody told you the rule for sorting. You only said who is close to whom. That is contrastive learning: learning from comparisons instead of from answers. CEBRA uses exactly this move on neural data, with a network in the place of your hands.
Each training step picks a reference sample x: one time bin of neural activity. It also picks a positive sample y+, a bin that should end up close to the reference, and several negative samples y1 to yn, bins that should end up far away. A reference with its positive is a positive pair. A reference with a negative is a negative pair.
Who counts as a friend? That is where the auxiliary variable enters, in Chapter 4. For now, take it as given. The network, called f, maps every bin to a unit-length point z = f(x) on the sphere from Chapter 1.
For two unit-length vectors, the dot product z·z′ equals the cosine of the angle between them. Same direction: cos 0° = 1. A right angle: cos 90° = 0. Opposite: cos 180° = −1. So the dot product is a score from −1 to 1 that says how close two points are on the sphere. It is called the cosine similarity.
For two bins 25 degrees apart, the score is cos 25° = 0.91. For 90 degrees it is 0.00. The paper divides the score by a number τ, the temperature. A small τ makes the score louder. At τ = 0.5, the 25-degree pair scores 0.91 / 0.5 = 1.81, and the 90-degree pair still scores 0. We call this scaled score ψ.
Suppose we only pulled. The easiest way to make every positive pair close is to put every point in one spot. All friends are perfectly close, and the map says nothing. This is collapse.
The push prevents it. If a negative sits near the reference, the loss punishes that. The only way to please both forces is to spread points over the sphere, keeping friends together and strangers apart. The map ends up as informative as the labels allow.
There is a detail worth seeing. Not every negative is pushed equally. A negative already on the far side barely matters. A negative close to the reference matters a lot. The weight of each negative is its share of the total eψ, a share that Chapter 3 derives. Use the device to feel it.
The teal point is the reference. The purple point is its positive. The pink points are six negatives. Arrows show the pull and the push, and the bars show each negative’s share of the push. Move the positive, pick where the negatives sit, and watch the loss.
Illustrative placement. The loss is the paper’s Equation 2 for one reference with n = 6 negatives, with similarity = dot product of unit vectors divided by τ. Push share of negative i is eψi divided by the sum of all six; the arrow length is that share.
Put the negatives close to the reference and the loss jumps: the network must work harder. Spread them evenly or on the far side and the loss falls. Slide the positive away from the reference and the loss grows too, because the pull has further to go.
Take six negatives at 30, 90, 150, 210, 270 and 330 degrees from the reference, with τ = 0.5. Their scores ψ are 1.73, 0, −1.73, −1.73, 0 and 1.73. Raise e to each: 5.65, 1, 0.18, 0.18, 1 and 5.65. The total is 13.66. The negatives at 30 and 330 degrees each take 5.65 / 13.66 = 41% of the push. The two on the far side take 1% each.
So the loss works like a teacher who spends time on the hardest mistakes. Each step spends almost all of its push on the two bins that look most like the reference, which are exactly the confusions that blur the map.
The network never sees the points on the sphere as a picture. It sees neural data x and, through f, produces z. The loss compares only the z values. Gradient descent then nudges the weights of f so that every reference moves toward its positive and away from its hard negatives. Each step changes f a little. After thousands of steps, the sphere is organized.
Two things in this picture are not data. The first is the choice of friends. The second is the choice of f. The paper keeps both open for the user, and that freedom is its main design idea.
Chapter 3
Turn pull and push into one number by treating each step as a multiple-choice question
Imagine a quiz. You see one photo, the reference. Below it are several answer photos. Exactly one was taken a minute away. The others come from other days. Your job is to point at the friend.
A network can take this quiz. It scores every answer photo by its closeness to the reference. A high score means “I think this one is the friend”. The better the network, the higher the friend’s score compared with the rest. That is the whole idea of the InfoNCE loss. NCE stands for noise contrastive estimation, a way to learn by telling real pairs from noise pairs.
Here is the criterion the paper minimizes (its Equation 2), for one reference x, one positive y+ and n negatives y1 to yn.
Why this shape? Start from the quiz. A softmax turns scores into probabilities that add up to 1. The chance that the network picks the positive is eψ+ divided by the sum of eψ over all answers. The cross-entropy loss is the minus log of that chance. Expanding the log of a fraction gives −ψ+ plus the log of the sum. That is the equation above.
One detail. The usual textbook form includes the positive in the sum. The paper’s Equation 2 sums over the negatives only, as an estimate of its Equation 1, where the sum is an integral over the negative distribution. The authors’ code at the end of this chapter does the same.
Take temperature τ = 0.5. The reference points along (1, 0). The positive is 30 degrees away: its similarity is cos 30° = 0.866, so ψ+ = 0.866 / 0.5 = 1.73. Three negatives sit at 90, 120 and 180 degrees. Their similarities are 0, −0.5 and −1, so their ψ values are 0, −1 and −2.
push = log( e0 + e−1 + e−2 ) = log( 1 + 0.368 + 0.135 ) = log 1.503 = 0.41
L = −1.73 + 0.41 = −1.32
A collapsed map gives every pair the same ψ. Then the loss is −ψ + log(n eψ) = log n, which for n = 3 is 1.10. Our map beats that by 2.42. The positive scores well, the negatives score badly, and the loss shows it.
The paper makes a useful point. Call the loss with n negatives Ln. As the number of samples grows, Ln − log n settles on a fixed value that depends only on the pair distributions, not on the batch size.
This turns the loss into a test. Train with two label choices, wait until the same point in training, and compare their gaps. The lower gap is the label that explains more of the neural data. Chapter 6 uses this.
One reference at 0 degrees, one positive at the angle you set, and n negatives at fixed random angles. The bars show the pull term, the push term and their sum, the loss. The dashed line is log n, the score of a collapsed map.
Illustrative: the negatives are n seeded random angles on a circle (a 2-D stand-in for the sphere), so they are spread evenly on average, not learned. The slider sets n as 2 to the power of its value, from 2 to 256. Even evenly spread negatives cost about log I0(1/τ) above log n, so a positive that is not close can leave the gap above zero.
Try moving the positive far from the reference. The pull term turns from a gain into a cost, and the loss rises above the dashed line. The network would do better to collapse than to keep that pair. This is why labels that disagree with the neural data give a high loss.
pythonimport torch def info_nce(ref, pos, neg, tau=1.0): # ref, pos: (n, d) unit vectors. neg: (m, d) unit vectors. pos_sim = (ref * pos).sum(dim=1) / tau # (n,) psi(x, y+) neg_sim = ref @ neg.T / tau # (n, m) psi(x, y_i) c = neg_sim.max(dim=1).values.detach() # subtract the max so exp() cannot overflow pos_loss = -(pos_sim - c).mean() neg_loss = torch.logsumexp(neg_sim - c[:, None], dim=1).mean() return pos_loss + neg_loss # same value as the equation
Chapter 4
Decide who counts as a friend, because that single choice shapes everything the map can show
The rat is on its second lap, at 120 degrees round the track. Who are this moment’s friends? You have at least three honest answers. The moments just before and after. Or every moment, on any lap, when the rat stood near 120 degrees. Or moments picked at random.
Each answer defines a different map. The paper calls this choice sampling and names it “CEBRA’s key feature”. The network, the loss and the optimizer stay the same. Only the rule that picks positive and negative samples changes.
Three rules define the whole scheme. p(x) says how to pick the reference. p(y|x) says how to pick a positive for a given reference; the bar reads “given”. q(y|x) says how to pick a negative. The user writes these rules. The loss from Chapter 3 then does the rest.
Sampling happens in two steps. First pick locations in the recording, using a context variable called the index. Then fetch the neural data at those locations.
The simplest negative is a random time bin from the whole dataset. That works well, because a random bin is rarely a friend.
There is one useful twist. Suppose you have a discrete variable k, such as which session or trial a bin came from. You can draw negatives so that every value of k is equally likely. If you also draw the positives that way, the embedding cannot use k: carrying k would not help the loss. The map becomes invariant to k, meaning it ignores it. This is how you remove nuisance factors. The paper’s example: for a brain-machine interface that should not depend on short-term changes, treat trial ID as a task-irrelevant variable.
A small example makes the invariance idea concrete. Say two recording sessions differ only by a slow drift in the equipment. Draw positives and negatives evenly from both sessions. A map that tells the sessions apart gains nothing, since half of every set of negatives comes from the same session as the reference. The best the network can do is to ignore the drift and keep what both sessions share.
Time. Take a short time offset and pick the bin that far from the reference. This is CEBRA-Time. It uses no behavior at all. It rests on a simple bet: the hidden state changes smoothly, so bins close in time are close in the map. The paper calls it discovery-driven, because it needs no hypothesis.
Discrete behavior. If bins carry a label k from a small set, such as “leftward” or “rightward”, filter the data to the reference’s value and pick any bin at random from that set. This is close to supervised contrastive learning.
Continuous behavior. Here the label c is a number, such as position. Choose a time offset τ from a set Δ. Compute the step d = ct − ct+τ that the behavior usually takes over that offset. Then look through the whole dataset for the bin whose behavior is closest to the reference’s behavior moved by d. That bin is the positive. This is CEBRA-Behavior, called hypothesis-driven, since you chose the label.
Notice what the third rule does. It can pick a bin from another lap. Two moments far apart in time become friends because the rat stood in the same place. That is how the map ties the laps together.
The paper also allows hybrid designs. In one test, a 5-D latent space had its first 3 dimensions shaped by behavior and time, and its last 2 by time alone. The idea is to hold what the label explains in some dimensions and let the rest absorb leftover variation.
The saw-tooth line is the rat’s position over three laps. The teal ring is your reference. The dots are 30 positives drawn under the rule you pick. Slide the reference, press Draw again, and compare the three rules.
Illustrative: the same 600-bin, 3-lap toy as the hero, with time offsets of 1 to 10 bins (as in the paper’s runs with time_offsets = 10). In Shuffled mode the position labels are permuted over time before the behavior rule is applied, so the “friends” stand all over the track.
In Time mode, every positive sits inside the same lap, within a few bins. In Behavior mode, the positives come from all three laps and all sit near the reference’s position. In Shuffled mode, the positives scatter. The network is then asked to place far-apart positions together, and nothing in the neurons supports that, so no consistent ring can form.
The symptom of a bad pairing is a map with no ring, or a ring that decodes badly. The measurement is the gap Ln − log n from Chapter 3, which stays high, together with a poor decoding score. The usual fixes are to check that the label is synchronized with the neural recording, and to try a different label or a time-only run. The paper reports exactly this contrast: a shuffled label gives a high loss and an unstructured embedding.
Chapter 5
Follow the numbers from raw spike counts to a unit vector, and see why the network reads a window of time instead of one bin
Look at one frame of a film. A blur could be a runner, a dog or a bush. Look at ten frames in a row and the blur moves, and now you know. A single time bin of spike counts is like that one frame: noisy, and easy to misread. A short stretch of bins tells a steadier story.
So the CEBRA network does not take a single bin. It takes a window of recent bins. The number of bins it looks at is its receptive field. This chapter follows the data through the network, one tensor shape at a time. A tensor is just a grid of numbers, and its shape lists the size of each side.
A session gives three signals, all aligned in time. The signal st holds the neural data, with shape (T, N): T time bins by N neurons. The continuous context ct holds behavior such as position, shape (T, m). The discrete context kt holds labels such as direction, shape (T,). The paper says s and c can have up to hundreds or thousands of dimensions. For raw video or audio it advises extracting features first, then passing those in.
The method uses two feature extractors, f for the reference and f′ for the other samples. In a single session they are the same network. In a multi-session setup they differ, as Chapter 7 shows.
The paper uses three encoders, picked by how long a window the data needs.
Where does 10 come from? A convolution with kernel size k sees k bins, and each further layer adds k − 1 more. Start from 1 bin. Layer one (k = 2) adds 1. Layers two to five (k = 3) add 2 each. So 1 + 1 + 2 + 2 + 2 + 2 = 10. For the 40-bin model the paper states the window size without a sum, so take 40 as given. The device below walks one 40-bin window through the layers with no padding: 40 bins shrink to 19, 9, 7, 5, 3 and finally 1.
A skip connection adds a layer’s input to its output, so the network can keep what it already has. Every layer except the last uses a GELU activation, a smooth function that lets small values pass a little. The last layer maps to the embedding size D, and the result is scaled to length 1.
The windows match in real time. Ten bins of the rat data are 250 ms. The mouse calcium frames last about 33 ms, so ten of them are 330 ms. The 40 Neuropixels bins span the same 330 ms. In each case the network sees about a quarter to a third of a second.
Each row is a layer. The squares show how many time steps are left; the shape is channels by time. Choose a receptive field and move the sizes. The window shrinks to one step, and that step is the embedding.
Layer sizes follow the Methods text: 10-bin model with kernels 2, 3, 3, 3, 3 (the lengths shown assume no padding, which is what one output per 10-bin window needs); 40-bin model with a strided kernel-4 and a strided kernel-3 convolution, then four kernel-3 layers. The weight count is my own sum of the convolution weights and biases for equal hidden width; the paper gives no total, and the real code may differ in small details. For 40 bins the strided layers’ widths are not stated, so no total is shown.
Let B be the batch size (512 in the paper’s runs). One step goes like this.
For small datasets, the paper draws pairs from the whole dataset instead of splitting a mini-batch into pairs. Doing so gives a better estimate of the pair distributions, and the authors say it considerably improves the embeddings. If the data fit in GPU memory, the whole dataset can serve as the batch. At test time you drop all pairs: pass any window through f and read the unit vector.
Failure, measured. Symptom: the ring is fuzzy or two nearby positions swap. Measurement: decoding error and consistency R² versus the window size. Fix: widen the window until it covers a few hundred milliseconds, but not much further, since a long window blurs fast changes.
Chapter 6
Use the loss as a referee between ideas about the brain, and use topology to check that the shape of the map is real
Two scientists argue about the hippocampus. One says its neurons map space. The other says they mostly track the direction the animal runs. They could argue for years. CEBRA offers a quicker way: train one model with position as the label, one with direction, one with both, and compare how well each fits.
This is what the paper calls hypothesis-driven analysis. The label you pass in is the hypothesis. If the neural data truly carry that variable, the network can build a clean map, and the loss falls far below the collapsed-map line. If the neurons do not carry it, the loss stays high.
Recall the gap Ln − log n from Chapter 3. Lower is better, and it does not depend on batch size. The rule is: train each model the same number of steps, then compare gaps. The label with the lowest gap explains the most.
The paper tests position, direction, both together and shuffled versions of each on its rat data. It finds that position plus direction is the most informative label. Shuffled labels, which break the link between neurons and behavior, converge to considerably higher loss values and give an unstructured embedding.
The decoding scores back this up. With a nearest-neighbour decoder on a 32-dimensional embedding, position plus direction gives an R² of 73.35% and a median absolute error of 5.8 cm. The shuffled run gives an R² of −49.90% and an error of 44.7 cm. A negative R² means the prediction is worse than always guessing the average position. The authors note that the pi-VAE paper reported an error of 12 cm on the same rat.
A small example, with made-up numbers. Suppose that after 1,000 steps the position model has a gap of −1.3, the direction model −0.1, and a shuffled-position model +0.4. Position explains much more than direction. The shuffled model shows what a useless label looks like on this data. Always keep a shuffled control: it tells you what “no relation” costs.
The failure to watch for is comparing at different points in training. The symptom is a model that looks better only because it trained longer. The measurement is the gap at the same step count for every label. The fix is to fix the step budget first, then read the gaps.
What if you have no hypothesis? Train with time alone. The paper finds that the CEBRA-Time embedding of the rat data looks much like the position embedding. A smooth loop forms around the track. The structure is in the neural activity, and time is enough to bring it out. The paper advises running both variants, since each tells you something.
There is a theory fact behind the loss test. If some mapping exists between the label and the neural signal, CEBRA recovers it. If none exists, CEBRA learns a default embedding: points spread evenly over the hypersphere. A uniform blob is the sign of a label that says nothing.
A simulated rat runs a circular track in two directions. Some neurons prefer one direction. Press Run the four models. Each trains for 400 steps with a different label. Bars show the gap Ln − log n; left of the line is a better fit.
Toy lab, not the paper’s data: 20 neurons, 600 bins, direction flips half-way, neurons of the other direction fire at 35% of their gain, a linear encoder with 3 outputs on the sphere, 400 steps each. Gap = mean loss of the last 30 steps minus log n. Error = median absolute angle error of a nearest-neighbour decoder that trains on the first two thirds of the recording and is scored on the last third. In this toy a linear encoder cannot use direction much, so position alone and position plus direction come out close; the paper’s deep encoder did better with both.
Read the bars as a referee would. Position pulls the gap well below zero. Direction alone stays above the line. Shuffled position lands even further on the wrong side. In this toy, adding direction to position changes little, because a linear encoder has few ways to use it. The paper’s deep convolutional encoder had more room, and there the extra label gave the lowest loss.
A pretty ring in 3-D could still be an accident. The authors add a second check from topology, the study of shape that ignores stretching. Their tool is persistent cohomology.
The idea: put a small ball around every embedded point. Grow all balls together. When two balls touch, their points join into one piece. When balls close up a hole, a loop is born. Later the loop fills and dies. A loop that lives for a long time is real. A loop that dies at once is noise.
The counts of pieces, loops and voids are the Betti numbers, written (H0, H1, H2).
| Shape | Pieces (H0) | Loops (H1) | Voids (H2) |
|---|---|---|---|
| One blob | 1 | 0 | 0 |
| A ring | 1 | 1 | 0 |
| A sphere surface | 1 | 0 | 1 |
Points spread evenly over a 2-sphere give (1, 0, 1). That is the signature of a random embedding. The paper finds (1, 1, 0), a ring, for CEBRA-Behavior and CEBRA-Time on the rat data, as you would expect for place cells on a track. They set the cut-off between a real loop and noise from shuffled embeddings, so a loop must outlive whatever shuffling produces. They then use the first loop to give every time bin an angle on the ring. That angle matches the rat’s position, in every dimension they tried.
Chapter 7
Train one map across animals and recording types, then adapt it to a new animal in a handful of steps
A new patient arrives and wears a recording array. Their neurons are not the neurons of anyone before them. There are more or fewer, and each fires for different things. Yet you would like to start from what you learned on earlier patients, and be useful in minutes, not weeks.
The paper sets up this problem with rats, mice and monkeys. A single recording is a session. Sessions differ in neuron count, in day, in animal and even in recording method. How can one map serve them all?
The neural signal changes size from session to session. Session i has Ni neurons, so its data has shape (Ti, Ni). But the behavior label, say position, has the same shape in all of them: (Ti, m) with the same m. The label is the bridge. Two sessions at the same position should land at the same place on the sphere.
The paper builds this with three changes to the single-session recipe.
Without the shared pairs, two separately trained networks put the ring at arbitrary angles. Each ring is correct, and the two are consistent up to a linear change. But a decoder trained on one cannot read the other. With joint training, a given position lands at the same spot in both, and a decoder carries over.
Three simulated rats (A, B, C) have 16, 22 and 19 neurons. Choose how to train A and B, press Train A and B, then adapt the new rat C. The score is the error of a position decoder fitted on rat A and applied to another rat.
Toy lab, not the paper’s data: 600 steps, 96 positives and negatives per step, a linear encoder to the unit circle per rat. Adapting C trains only its own encoder, with rats A and B frozen and used for positives and negatives. A random guess is off by 90 degrees (median). The paper’s own figures are in the text below; results here vary with the random seed, and the page uses a fixed one.
Compare the two modes. Trained separately, the rings point in unrelated directions, and the decoder from A fails on B. Trained jointly, the colors line up around the ring, and the decoder from A works on B almost as well as on A itself.
Joint training raised consistency across subjects on the Allen Institute mouse data and on the rat hippocampus data. The paper used one-sided paired t-tests: t = −5.80 with p = 6.0 × 10−5 for Allen, and t = −2.22 with p = 0.024 for hippocampus. A fair worry is that a shared space loses what is special to one animal. The authors checked: decoding from the shared “pseudo-subject” space showed no loss in decoding performance.
This is the brain-machine-interface case. Pretrain on a subset of subjects. Then take the unseen one and train only its encoder, keeping the rest. The paper reports that adaptation runs at 50 to 100 steps per second, and that the position decoding error already dropped by 10 cm after adapting for one step. It also reaches a lower error more quickly than training on the new animal alone. The authors argue that this speed matters for real-time and adaptive decoding, and that pi-VAE is far slower, since its sampling to approximate the test label is costly.
Try it in the device. Before any adaptation, rat C’s encoder is random, and the error sits near chance. A handful of steps pulls its ring into line with the shared space.
The paper pushes the idea further. It takes mouse visual cortex recorded two ways, from different mice watching the same movie. One way is Neuropixels probes, which record electrical spikes. The other is two-photon calcium imaging, a fluorescent signal that follows activity slowly. The label is a feature vector for each movie frame from a vision transformer called DINO.
The two modalities gave similar embeddings, and joint training across them improved consistency without lowering consistency within a modality. The authors also compared areas. With joint training, consistency within an area was higher than across areas (Welch’s t-test, T = 4.55, p = 0.00019). The shared space did not erase real biological differences between areas.
Failure, measured. Symptom: rat B’s ring comes out rotated against rat A’s. Measurement: the error of a decoder fitted on A and applied to B. Fix: train A and B in one run with a shared behavior label.
Chapter 8
Read each result against its baseline, see which controls rule out the easy explanations, and know where the claims stop
A method is only as convincing as its comparisons. Each of the paper’s five figures asks one question of one dataset and sets CEBRA against a baseline or a control. This chapter walks through them in order, with the numbers the text gives.
The authors first make data where the answer is known. A hidden 2-D latent variable depends on a behavior label (a number from 0 to 2π). Four random invertible networks (RealNVP blocks) turn the latent into the firing rates of 100 neurons, and Poisson noise turns rates into spike counts. That gives 15,000 points. A method is then scored by how well its embedding, after a linear fit, recovers the true latent (R²).
CEBRA-Behavior scored significantly higher than pi-VAE, tSNE and UMAP over 100 runs each (one-way ANOVA F(3, 396) = 278.31, p = 3.95 × 10−97, post hoc Tukey p < 0.001). On the rat hippocampus data, CEBRA also gave higher consistency across animals than conv-pi-VAE with or without test-time labels (F(5, 67) = 28, p = 3.4 × 10−15).
Note the fairness step. The original pi-VAE reads one time bin. The authors added a convolutional front end so it could use a 10-bin window, which improved pi-VAE’s own results (median error 11 cm instead of 12 cm). They benchmarked against that stronger version, called conv-pi-VAE, and said so.
The paper checks position, direction and both, then adds shuffled controls. Position plus direction wins, with a decoding R² of 73.35% against −49.90% for shuffled and a median error of 5.8 cm against 44.7 cm. Time alone beats PCA, tSNE and UMAP at decoding (F = 12,091). A persistent-cohomology check finds a ring (1, 1, 0).
A monkey moves a handle to eight targets, or has its arm bumped by the handle. Recordings come from area 2 of the somatosensory cortex. CEBRA-Behavior trained on hand position gives a clean trajectory. Active and passive movements land in different parts of the map. A 16-D CEBRA-Behavior model recovers the hand path with R² of 88% on 26 held-out trials. An L1 regression from all neurons gets 74%, and 16-D conv-pi-VAE gets 82%.
Mice watch a natural movie ten times. The label of each frame is its DINO feature vector. CEBRA trained on Neuropixels or calcium data gives smooth trajectories that follow the movie, and the two recording types agree (Chapter 7). For decoding, the paper trains on nine repeats and holds out the tenth. With a nearest-neighbour or naive-Bayes decoder on the CEBRA embedding, frame accuracy is above 95%, where a frame counts as right if the guess falls within a 1-second window, or if the right scene is identified. Baselines using the raw population vector with the same decoders are significantly lower; the text prints no values for them, and the supplement gives pairwise tests.
Pick a dataset. The bars are numbers printed in the paper, with the figure that shows them. Note which way is better: error falls, R² and accuracy rise.
Numbers copied from the arXiv v2 text: rat, Fig. 2d and Methods (CEBRA-Behavior with position and direction 5.8 cm, shuffled 44.7 cm, conv-pi-VAE 11 cm, pi-VAE as reported in its own paper 12 cm); monkey, Fig. 3i text (88%, 82%, 74%); mouse, text and Fig. 5 (“greater than 95%” for CEBRA; baselines are lower but their values are not printed in the text). Runs, seeds and test sets differ between figures, so compare bars only within one dataset.
Several controls rule out the easy stories. Shuffled labels give a high loss and poor decoding, so the label carries the information (Chapters 4 and 6). Shuffled DINO features give poor movie decoding, so the features alone do not drive the result. Time-only CEBRA already gives a similar ring, so the structure is in the neurons. Higher dimensions can raise consistency (Extended Data). And layers matter: in V1, layers 2/3 and 5/6 decode better than layer 4 (F(2, 12) = 9.88, p = 0.003). The authors suggest that the non-thalamic input layers make frame information more explicit, perhaps through feedback or predictive processing. That is their interpretation, not a tested claim.
Chapter 9
Keep the numbers, the checklist and the core code, and find where to go next
CEBRA trains a small network to map windows of neural activity to unit vectors. It pulls together windows that share a label or sit close in time, and pushes apart the rest, using the InfoNCE loss. The label and the sampling rule are the scientist’s hypothesis. The loss then tells you how well that hypothesis fits, and the maps from different animals agree up to a linear change.
Pick what you want to do. The card shows the label, the positives, the negatives and the check that tells you it worked.
A summary of the Methods and Results of the paper, written as a recipe. The rule for invariance is the one in Chapter 4; the checks are the ones used in Chapters 1, 6 and 7.
| Quantity | Value | Where it comes from |
|---|---|---|
| Embedding size | unit-length vectors; n + 1 CEBRA dimensions match n dimensions of other methods | Methods, hypersphere |
| Loss of a collapsed map | log n for n negatives | Methods, goodness of fit |
| Goodness of fit | −DKL(p‖q) ≤ Ln − log n ≤ 0, lower is better | Equation 3 |
| Receptive fields | 1 bin; 10 bins (250 ms for the rat); 40 bins (Neuropixels at 120 Hz) | Methods, CEBRA models |
| Batch size in the paper’s runs | 512 | Methods, model parameters |
| Rat position error | 5.8 cm with position + direction; 44.7 cm shuffled; 12 cm reported for pi-VAE | Fig. 2d and text |
| Rat decoding R² | 73.35% against −49.90% shuffled | Fig. 2d |
| Monkey hand path R² | 88% CEBRA; 82% conv-pi-VAE; 74% L1 regression | Fig. 3i text |
| Mouse V1 movie frames | over 95% accuracy | Fig. 5 text |
| Adapting to a new animal | 50 to 100 steps per second; error down 10 cm after one step | Results, multi-session |
| Consistency tests | 12 pairs across 4 rats; 90 comparisons across 10 runs | Methods |
| Topology of the rat map | Betti numbers (1, 1, 0), a ring; a random sphere gives (1, 0, 1) | Fig. 2f, 2g |
python# One training step, written out. f maps a window (N, RF) to a unit vector (D,). for step in range(steps): t = rng.integers(0, T, B) # B references t_pos = sample_positive(t, c, k) # Chapter 4: time offset or closest behavior t_neg = rng.integers(0, T, B) # random negatives ref, pos, neg = f(window(s, t)), f(window(s, t_pos)), f(window(s, t_neg)) # (B, D) each loss = info_nce(ref, pos, neg, tau) # Chapter 3 opt.zero_grad(); loss.backward(); opt.step() # The package, as printed in the paper (check the docs for the current API). import cebra model = cebra.CEBRA(output_dimension=8, num_hidden_units=32, batch_size=1024, learning_rate=3e-4, max_iterations=1000) model.fit(s) # time only (discovery) model.fit(s, k) # discrete behavior model.fit(s, c) # continuous behavior (hypothesis) model.fit([s, s2], [c, c2]) # two sessions, one space model.fit(s2, c2, adapt=True) # adapt to a new session z = model.transform(s_test) # (T, 8) embedding, no labels needed
CEBRA is one answer to a question the course keeps asking: how do we find the few numbers that matter in a large population of neurons? It sits between linear tools such as PCA and ICA, generative sequence models such as LFADS, and the contrastive methods of self-supervised learning. Its consistency across sessions bears on a hard BCI problem, drift, treated in Drift and Recalibration.
Steffen Schneider and Jin Hwa Lee (co-first authors) and Mackenzie Weygandt Mathis, EPFL Brain Mind Institute and Neuro-X. Paper: arXiv 2204.00673, published in Nature in 2023 (doi 10.1038/s41586-023-06031-6). This lesson follows the arXiv v2 text, so numbers may differ from the journal version. Code: github.com/AdaptiveMotorControlLab/CEBRA; docs: cebra.ai/docs. Data used in the paper: the CRCNS hc-11 rat hippocampus set, the DANDI 000127 primate set, and the Allen Institute Brain Observatory mouse sets. Toy labs on this page are ours and are not the authors’ code or data.
Now press Teach and explain CEBRA at the whiteboard, from memory: the pairs, the loss, the gap to a collapsed map, and what makes two animals’ maps consistent. Then go back to the hero and train all three pull signals once more.