CEBRA

CEBRA draws a map of brain activity by pulling together moments that share a behavior and pushing the rest apart. It read a rat’s position to within 5.8 cm, where pi-VAE had reported 12 cm.

Learn how a small neural network turns hundreds of noisy neurons into a few numbers that mean something, the same few numbers for every animal.

Pick what pulls the points together, then press Train 300 steps. Watch a ring form when the pull signal is real, and fail when it is shuffled. Then we build, piece by piece, the pairs, the loss, the encoder, the tests and the results.

You need the idea of a vector (a list of numbers) and what a neuron’s firing rate is. We build the rest from zero.

Paper: Steffen Schneider, Jin Hwa Lee and Mackenzie Weygandt Mathis (EPFL), Learnable latent embeddings for joint behavioral and neural analysis, arXiv 2204.00673 (this lesson follows version 2, October 2022), published in Nature in 2023. Every idea is re-taught here in our own words and devices. This paper belongs to the lecture Latent Variable Models for Neural Data.

Train a tiny CEBRA

A toy lab, not the paper’s data: 16 simulated neurons with sharp place fields and added noise, a rat running 3 laps of a circular track in 600 time bins, a linear encoder with 2 outputs scaled to length 1, 96 positive and 96 negative samples per step, temperature 0.3, the Adam optimizer. Position is read back with a nearest-neighbour decoder (k = 9) fit on laps 1 and 2 and scored on lap 3. A guess at random is off by 90 degrees (median). The paper’s numbers are in Chapter 8.

Chapter 0

Why a Map of Neural Activity Is Hard

See why a list of noisy firing rates hides a simple shape, and what a good map must do that older tools could not

A rat runs back and forth along a track 1.6 meters long. A thin probe in its hippocampus (a deep brain region that helps an animal know where it is) listens to many neurons at once. Every 25 milliseconds, you write down how many times each neuron fired.

That count is the neuron’s firing rate in one time bin, a short slice of the recording. If the probe hears 16 neurons, one time bin is a list of 16 numbers. A list of numbers is a vector, and this one is the population vector: what the whole group said in that slice.

One recording gives thousands of these vectors, one per time bin. Your question is simple. What do they say about the rat? Where is it? Which way is it running? Is it the same story on every lap?

A shape hides inside the list

Some hippocampal neurons are place cells. Each one fires most when the animal stands at one place on the track, its favourite spot. If you know where the rat is, you can guess which neurons are loud. If you hear which neurons are loud, you can guess where the rat is.

The device below uses a simulated version of this. The track is a circle, so position is an angle from 0 to 360 degrees. Slide the rat around the track and watch the 16 bars. Then press Next lap. The rat stands at the same place, yet the bars change, because neurons are noisy.

One moment, sixteen numbers

Left: how loud each neuron is at this position (bars for the lap you picked, dots for the other two laps). Right: every time bin as one dot, seen through just two neurons. The rings mark this position on each lap.

120

Simulated, the same 16-neuron toy as the hero: sharp place fields plus noise, 3 laps, 200 time bins per lap, each neuron’s average subtracted. The paper’s rats ran a straight 1.6 m track and each rat had its own number of neurons. Dot color is track position.

The right panel is the problem in one picture. Two neurons give a flat view of a 16-dimensional cloud. The dots overlap, loop over each other and smear. The shape is in the cloud, but no pair of neurons shows it cleanly.

What a good map must do

A latent embedding is a short list of numbers, say 3, that stands in for the long list of 16 or 100. Latent means hidden: the rat’s position was never in the data, yet the neurons carry it. Embedding means each time bin gets a point in a small space. A good embedding puts bins from the same place close together.

The authors list what they want from such a map. It should be nonlinear, because the link between neurons and behavior bends. It should use behavior and time, because both are always recorded. It should be consistent: two animals, or two runs of the method, give the same map. And it should not need a generative model, an explicit recipe for how hidden causes produce spikes.

Why the older tools fall short

PCA (principal component analysis) lays a flat sheet through the cloud along its widest directions. It is easy to read, but a flat sheet cannot follow a bend, and the paper notes that this costs performance. See our PCA lesson.

tSNE and UMAP are nonlinear and make pretty cluster pictures. But they ignore the order of time bins, though time is always known. They also drift when you stack animals together: the paper says concatenating animals can shift the clusters.

pi-VAE and LFADS are generative. A VAE (variational autoencoder) learns by rebuilding its input from a hidden code; see our VAE lesson. These models assume how the data is made, and the authors argue they do not guarantee the same map for every animal.

The CEBRA idea in one line. Skip the recipe for how spikes are made. Instead, teach a network which time bins should sit close, using a label you already have, and let the geometry follow.

The name says it: Consistent EmBeddings of high-dimensional Recordings using Auxiliary variables. An auxiliary variable is any extra recorded signal: the rat’s position, a trial number, the time stamp, or features of a movie the animal watched. The method is a form of nonlinear ICA (independent component analysis, which looks for hidden sources; see ICA) trained by contrastive learning (our lesson).

The road

  1. What an embedding is, and what “consistent” means: a number for the sameness of two maps.
  2. Pull and push: reference, positive and negative samples on a sphere.
  3. The InfoNCE loss: the equation, its symbols and a worked batch.
  4. Choosing the pairs: time, behavior, hybrid, and what shuffling breaks.
  5. The encoder and the data flow: shapes in, shapes out.
  6. Hypotheses and topology: the loss as a test, and counting loops.
  7. Many animals, many sessions: one shared space and quick adaptation.
  8. The results: what each figure shows, the ablations, the limits.
  9. Field guide: numbers, checklist, the core code.
The right panel of the device shows a smeared cloud, although the neurons clearly carry the rat’s position. What is the best reading?

Chapter 1

What an Embedding Is, and What “Consistent” Means

Measure whether two maps of the same brain function agree, and see which changes keep them the same map

Two friends each draw a map of the same city. One puts north at the top. The other turns the sheet sideways and uses miles instead of kilometers. The maps look different, yet every street is in the same place relative to the others. They are the same map.

A third friend draws a map where the river bends the wrong way and two districts swap places. That one is a different map. The rule that separates the two cases is the heart of this chapter, and it is what the authors mean by consistent.

An embedding, made concrete

An embedding turns each time bin into a short list of D numbers. D is the embedding dimension. With D = 3, each bin is a point in 3-D space, and a recording of 15,000 bins is a cloud of 15,000 points.

CEBRA scales every point to length 1. The points then lie on a hypersphere, the set of all points at distance 1 from the origin. For D = 2 that is a circle. For D = 3 it is the surface of a ball. This is why the paper says that 3 dimensions of CEBRA are equal to 2 dimensions of the other methods: the length-1 rule removes one degree of freedom.

Why a sphere? It gives a clean way to compare two points: the angle between them. Chapter 2 uses exactly that.

Two animals, two sets of neurons

The paper records four rats. Each rat has different neurons under the probe, and a different number of them. So the raw input differs from rat to rat. Yet all four ran the same track. If CEBRA works, each rat should give the same map of the track.

But a trained network has no reason to pick the same orientation twice. One run may put the start of the track at the top of the ring, another at the bottom. The authors do not ask for that. They ask for something weaker that is still useful: the two maps must match after a linear change. A linear change is any mix of rotating, mirroring, stretching and shifting. It never bends or tears.

This is called linear identifiability. Earlier theory (Hyvärinen and colleagues, and Roeder and colleagues, cited in the paper) shows that contrastive models trained to convergence on the same data agree up to such a linear map. A downstream tool that is itself linear, such as a linear regression, then gives the same score on either map.

The consistency score

To measure it, take two embeddings of the same moments, A and B. Fit the best linear map from A to B, with an intercept (a shift). Then ask how much of B the fit explains. The score is R².

R2
The share of B’s variation that the linear fit explains. 1 means a perfect match. 0 means no better than always guessing the mean.
∑(bi − b̂i)2
What the fit gets wrong. For each point i, take the true value bi of B, subtract the fit’s prediction b̂i, square it, and add up.
∑(bi − b̄)2
How much B varies at all: the squared distance of each value from the mean b̄.

B = 1, 2, 3, 4; fit = 1.1, 1.9, 3.2, 3.8: misses 0.1, 0.1, 0.2, 0.2 give 0.01 + 0.01 + 0.04 + 0.04 = 0.10; mean 2.5 gives 2.25 + 0.25 + 0.25 + 2.25 = 5.00; so R2 = 1 − 0.10 / 5.00 = 0.98

The paper uses this score in two ways. Across subjects: embed rat A and rat B, fit one to the other, and report R² for all pairs (12 pairs of four rats). Across runs: train the same method 10 times on the same data and compare every ordered pair (90 comparisons). A method that is consistent has high scores and little spread in both.

Is it the same map?

The left ring is map A. The right ring is map B, made by changing A in the way you choose, plus a little noise. The score is the R² of the best linear fit from A to B. Try each change and push the slider.

60

Illustrative: 120 points on a circle, map B gets noise of 0.03 on each coordinate. Rotate turns B by up to 180 degrees. Stretch squeezes one axis down to 25% and shears. Twist adds a position-dependent turn of up to 1.6 radians (a nonlinear bend). Scramble swaps the order of points by an amount you set. R² is computed live with a least-squares fit with intercept.

Rotate, stretch and shear all keep R² near 1. Twist and scramble pull it down. The linear fit forgives a change of viewpoint. It does not forgive a change of shape.

When the score is low

A low score has a symptom: two rats give rings with different structure, or one run fails to close the loop. The measurement is R² between embeddings of the same behavior bins. A common fix is more data or the right labels. Chapter 7 shows a second fix: train on all animals at once so their maps share one space.

Two training runs give rings that are mirror images of each other. What does the consistency score say?

Chapter 2

Pull and Push on a Sphere

Teach a network a map by telling it which moments are friends and which are strangers

Spread a day’s photos across a table. Pick one photo. Find another taken a minute later, and place the two side by side. Now take a handful of photos from other days and slide them to the far edge. Do this a thousand times, with a different photo each time, and the table sorts itself by the time of day.

Nobody told you the rule for sorting. You only said who is close to whom. That is contrastive learning: learning from comparisons instead of from answers. CEBRA uses exactly this move on neural data, with a network in the place of your hands.

Three roles in every comparison

Each training step picks a reference sample x: one time bin of neural activity. It also picks a positive sample y+, a bin that should end up close to the reference, and several negative samples y1 to yn, bins that should end up far away. A reference with its positive is a positive pair. A reference with a negative is a negative pair.

Who counts as a friend? That is where the auxiliary variable enters, in Chapter 4. For now, take it as given. The network, called f, maps every bin to a unit-length point z = f(x) on the sphere from Chapter 1.

Closeness is an angle

For two unit-length vectors, the dot product z·z′ equals the cosine of the angle between them. Same direction: cos 0° = 1. A right angle: cos 90° = 0. Opposite: cos 180° = −1. So the dot product is a score from −1 to 1 that says how close two points are on the sphere. It is called the cosine similarity.

For two bins 25 degrees apart, the score is cos 25° = 0.91. For 90 degrees it is 0.00. The paper divides the score by a number τ, the temperature. A small τ makes the score louder. At τ = 0.5, the 25-degree pair scores 0.91 / 0.5 = 1.81, and the 90-degree pair still scores 0. We call this scaled score ψ.

Two forces keep the map honest

Suppose we only pulled. The easiest way to make every positive pair close is to put every point in one spot. All friends are perfectly close, and the map says nothing. This is collapse.

The push prevents it. If a negative sits near the reference, the loss punishes that. The only way to please both forces is to spread points over the sphere, keeping friends together and strangers apart. The map ends up as informative as the labels allow.

There is a detail worth seeing. Not every negative is pushed equally. A negative already on the far side barely matters. A negative close to the reference matters a lot. The weight of each negative is its share of the total eψ, a share that Chapter 3 derives. Use the device to feel it.

Place the pairs

The teal point is the reference. The purple point is its positive. The pink points are six negatives. Arrows show the pull and the push, and the bars show each negative’s share of the push. Move the positive, pick where the negatives sit, and watch the loss.

25
0.50

Illustrative placement. The loss is the paper’s Equation 2 for one reference with n = 6 negatives, with similarity = dot product of unit vectors divided by τ. Push share of negative i is eψi divided by the sum of all six; the arrow length is that share.

Put the negatives close to the reference and the loss jumps: the network must work harder. Spread them evenly or on the far side and the loss falls. Slide the positive away from the reference and the loss grows too, because the pull has further to go.

Where the push goes, in numbers

Take six negatives at 30, 90, 150, 210, 270 and 330 degrees from the reference, with τ = 0.5. Their scores ψ are 1.73, 0, −1.73, −1.73, 0 and 1.73. Raise e to each: 5.65, 1, 0.18, 0.18, 1 and 5.65. The total is 13.66. The negatives at 30 and 330 degrees each take 5.65 / 13.66 = 41% of the push. The two on the far side take 1% each.

So the loss works like a teacher who spends time on the hardest mistakes. Each step spends almost all of its push on the two bins that look most like the reference, which are exactly the confusions that blur the map.

What the network sees, and what it does not

The network never sees the points on the sphere as a picture. It sees neural data x and, through f, produces z. The loss compares only the z values. Gradient descent then nudges the weights of f so that every reference moves toward its positive and away from its hard negatives. Each step changes f a little. After thousands of steps, the sphere is organized.

Two things in this picture are not data. The first is the choice of friends. The second is the choice of f. The paper keeps both open for the user, and that freedom is its main design idea.

You train with only the pull on positive pairs and no negatives. What do you expect?

Chapter 3

The InfoNCE Loss, Step by Step

Turn pull and push into one number by treating each step as a multiple-choice question

Imagine a quiz. You see one photo, the reference. Below it are several answer photos. Exactly one was taken a minute away. The others come from other days. Your job is to point at the friend.

A network can take this quiz. It scores every answer photo by its closeness to the reference. A high score means “I think this one is the friend”. The better the network, the higher the friend’s score compared with the rest. That is the whole idea of the InfoNCE loss. NCE stands for noise contrastive estimation, a way to learn by telling real pairs from noise pairs.

The equation the paper trains

Here is the criterion the paper minimizes (its Equation 2), for one reference x, one positive y+ and n negatives y1 to yn.

L
The loss: one number that is small when the map is good. Training pushes it down.
E[ … ]
Average over many draws of a reference, a positive and a set of negatives. The batch average is our estimate.
−ψ(x, y+)
The pull. ψ is the temperature-scaled cosine similarity of the two embeddings (Chapter 2). The minus sign means a high similarity lowers the loss.
log ∑ eψ(x, yi)
The push. Each negative adds eψ to a sum, so a close negative adds a lot. The log keeps the numbers small. It is a smooth “largest negative score”.

Why this shape? Start from the quiz. A softmax turns scores into probabilities that add up to 1. The chance that the network picks the positive is eψ+ divided by the sum of eψ over all answers. The cross-entropy loss is the minus log of that chance. Expanding the log of a fraction gives −ψ+ plus the log of the sum. That is the equation above.

One detail. The usual textbook form includes the positive in the sum. The paper’s Equation 2 sums over the negatives only, as an estimate of its Equation 1, where the sum is an integral over the negative distribution. The authors’ code at the end of this chapter does the same.

A batch, worked by hand

Take temperature τ = 0.5. The reference points along (1, 0). The positive is 30 degrees away: its similarity is cos 30° = 0.866, so ψ+ = 0.866 / 0.5 = 1.73. Three negatives sit at 90, 120 and 180 degrees. Their similarities are 0, −0.5 and −1, so their ψ values are 0, −1 and −2.

push = log( e0 + e−1 + e−2 ) = log( 1 + 0.368 + 0.135 ) = log 1.503 = 0.41

L = −1.73 + 0.41 = −1.32

A collapsed map gives every pair the same ψ. Then the loss is −ψ + log(n eψ) = log n, which for n = 3 is 1.10. Our map beats that by 2.42. The positive scores well, the negatives score badly, and the loss shows it.

Reading the loss as goodness of fit

The paper makes a useful point. Call the loss with n negatives Ln. As the number of samples grows, Ln − log n settles on a fixed value that depends only on the pair distributions, not on the batch size.

Ln − log n
The gap to the collapsed map. 0 means no better than collapse. Lower is a better fit. It does not change if you change n.
DKL(p‖q)
How different the positive distribution p is from the negative distribution q (the KL divergence, a measure of how far apart two distributions are). It is the best gap any map could reach. More informative labels make p differ more from q.

This turns the loss into a test. Train with two label choices, wait until the same point in training, and compare their gaps. The lower gap is the label that explains more of the neural data. Chapter 6 uses this.

Score a batch

One reference at 0 degrees, one positive at the angle you set, and n negatives at fixed random angles. The bars show the pull term, the push term and their sum, the loss. The dashed line is log n, the score of a collapsed map.

20
64
0.50

Illustrative: the negatives are n seeded random angles on a circle (a 2-D stand-in for the sphere), so they are spread evenly on average, not learned. The slider sets n as 2 to the power of its value, from 2 to 256. Even evenly spread negatives cost about log I0(1/τ) above log n, so a positive that is not close can leave the gap above zero.

Try moving the positive far from the reference. The pull term turns from a gain into a cost, and the loss rises above the dashed line. The network would do better to collapse than to keep that pair. This is why labels that disagree with the neural data give a high loss.

pythonimport torch

def info_nce(ref, pos, neg, tau=1.0):
    # ref, pos: (n, d) unit vectors.  neg: (m, d) unit vectors.
    pos_sim = (ref * pos).sum(dim=1) / tau            # (n,)  psi(x, y+)
    neg_sim = ref @ neg.T / tau                          # (n, m) psi(x, y_i)
    c = neg_sim.max(dim=1).values.detach()            # subtract the max so exp() cannot overflow
    pos_loss = -(pos_sim - c).mean()
    neg_loss = torch.logsumexp(neg_sim - c[:, None], dim=1).mean()
    return pos_loss + neg_loss                        # same value as the equation
You double the number of negatives from 64 to 128. Why do researchers compare Ln − log n rather than Ln?

Chapter 4

Choosing the Pairs: Time, Behavior, Hybrid

Decide who counts as a friend, because that single choice shapes everything the map can show

The rat is on its second lap, at 120 degrees round the track. Who are this moment’s friends? You have at least three honest answers. The moments just before and after. Or every moment, on any lap, when the rat stood near 120 degrees. Or moments picked at random.

Each answer defines a different map. The paper calls this choice sampling and names it “CEBRA’s key feature”. The network, the loss and the optimizer stay the same. Only the rule that picks positive and negative samples changes.

The three distributions

Three rules define the whole scheme. p(x) says how to pick the reference. p(y|x) says how to pick a positive for a given reference; the bar reads “given”. q(y|x) says how to pick a negative. The user writes these rules. The loss from Chapter 3 then does the rest.

Sampling happens in two steps. First pick locations in the recording, using a context variable called the index. Then fetch the neural data at those locations.

Negatives: usually just random

The simplest negative is a random time bin from the whole dataset. That works well, because a random bin is rarely a friend.

There is one useful twist. Suppose you have a discrete variable k, such as which session or trial a bin came from. You can draw negatives so that every value of k is equally likely. If you also draw the positives that way, the embedding cannot use k: carrying k would not help the loss. The map becomes invariant to k, meaning it ignores it. This is how you remove nuisance factors. The paper’s example: for a brain-machine interface that should not depend on short-term changes, treat trial ID as a task-irrelevant variable.

A small example makes the invariance idea concrete. Say two recording sessions differ only by a slow drift in the equipment. Draw positives and negatives evenly from both sessions. A map that tells the sessions apart gains nothing, since half of every set of negatives comes from the same session as the reference. The best the network can do is to ignore the drift and keep what both sessions share.

Positives: three ways to pick a friend

Time. Take a short time offset and pick the bin that far from the reference. This is CEBRA-Time. It uses no behavior at all. It rests on a simple bet: the hidden state changes smoothly, so bins close in time are close in the map. The paper calls it discovery-driven, because it needs no hypothesis.

Discrete behavior. If bins carry a label k from a small set, such as “leftward” or “rightward”, filter the data to the reference’s value and pick any bin at random from that set. This is close to supervised contrastive learning.

Continuous behavior. Here the label c is a number, such as position. Choose a time offset τ from a set Δ. Compute the step d = ct − ct+τ that the behavior usually takes over that offset. Then look through the whole dataset for the bin whose behavior is closest to the reference’s behavior moved by d. That bin is the positive. This is CEBRA-Behavior, called hypothesis-driven, since you chose the label.

Notice what the third rule does. It can pick a bin from another lap. Two moments far apart in time become friends because the rat stood in the same place. That is how the map ties the laps together.

The paper also allows hybrid designs. In one test, a 5-D latent space had its first 3 dimensions shaped by behavior and time, and its last 2 by time alone. The idea is to hold what the label explains in some dimensions and let the rest absorb leftover variation.

Who gets picked as a friend?

The saw-tooth line is the rat’s position over three laps. The teal ring is your reference. The dots are 30 positives drawn under the rule you pick. Slide the reference, press Draw again, and compare the three rules.

260

Illustrative: the same 600-bin, 3-lap toy as the hero, with time offsets of 1 to 10 bins (as in the paper’s runs with time_offsets = 10). In Shuffled mode the position labels are permuted over time before the behavior rule is applied, so the “friends” stand all over the track.

In Time mode, every positive sits inside the same lap, within a few bins. In Behavior mode, the positives come from all three laps and all sit near the reference’s position. In Shuffled mode, the positives scatter. The network is then asked to place far-apart positions together, and nothing in the neurons supports that, so no consistent ring can form.

When it goes wrong

The symptom of a bad pairing is a map with no ring, or a ring that decodes badly. The measurement is the gap Ln − log n from Chapter 3, which stays high, together with a poor decoding score. The usual fixes are to check that the label is synchronized with the neural recording, and to try a different label or a time-only run. The paper reports exactly this contrast: a shuffled label gives a high loss and an unstructured embedding.

With CEBRA-Behavior on position, why can a positive for a lap-2 reference come from lap 1?

Chapter 5

The Encoder and the Data Flow

Follow the numbers from raw spike counts to a unit vector, and see why the network reads a window of time instead of one bin

Look at one frame of a film. A blur could be a runner, a dog or a bush. Look at ten frames in a row and the blur moves, and now you know. A single time bin of spike counts is like that one frame: noisy, and easy to misread. A short stretch of bins tells a steadier story.

So the CEBRA network does not take a single bin. It takes a window of recent bins. The number of bins it looks at is its receptive field. This chapter follows the data through the network, one tensor shape at a time. A tensor is just a grid of numbers, and its shape lists the size of each side.

What goes in

A session gives three signals, all aligned in time. The signal st holds the neural data, with shape (T, N): T time bins by N neurons. The continuous context ct holds behavior such as position, shape (T, m). The discrete context kt holds labels such as direction, shape (T,). The paper says s and c can have up to hundreds or thousands of dimensions. For raw video or audio it advises extracting features first, then passing those in.

The method uses two feature extractors, f for the reference and f′ for the other samples. In a single session they are the same network. In a multi-session setup they differ, as Chapter 7 shows.

Three encoders, one rule

The paper uses three encoders, picked by how long a window the data needs.

Where does 10 come from? A convolution with kernel size k sees k bins, and each further layer adds k − 1 more. Start from 1 bin. Layer one (k = 2) adds 1. Layers two to five (k = 3) add 2 each. So 1 + 1 + 2 + 2 + 2 + 2 = 10. For the 40-bin model the paper states the window size without a sum, so take 40 as given. The device below walks one 40-bin window through the layers with no padding: 40 bins shrink to 19, 9, 7, 5, 3 and finally 1.

A skip connection adds a layer’s input to its output, so the network can keep what it already has. Every layer except the last uses a GELU activation, a smooth function that lets small values pass a little. The last layer maps to the embedding size D, and the result is scaled to length 1.

The windows match in real time. Ten bins of the rat data are 250 ms. The mouse calcium frames last about 33 ms, so ten of them are 330 ms. The 40 Neuropixels bins span the same 330 ms. In each case the network sees about a quarter to a third of a second.

Follow one window through

Each row is a layer. The squares show how many time steps are left; the shape is channels by time. Choose a receptive field and move the sizes. The window shrinks to one step, and that step is the embedding.

120
32
8

Layer sizes follow the Methods text: 10-bin model with kernels 2, 3, 3, 3, 3 (the lengths shown assume no padding, which is what one output per 10-bin window needs); 40-bin model with a strided kernel-4 and a strided kernel-3 convolution, then four kernel-3 layers. The weight count is my own sum of the convolution weights and biases for equal hidden width; the paper gives no total, and the real code may differ in small details. For 40 bins the strided layers’ widths are not stated, so no total is shown.

One training step, with shapes

Let B be the batch size (512 in the paper’s runs). One step goes like this.

  1. Pick B references, and for each one a positive and a negative, by the rules of Chapter 4. Each is a window of shape (N, RF), so each of the three sets has shape (B, N, RF).
  2. Encode all three sets. Each comes out as (B, D) with every row of length 1.
  3. Compute B pull scores (reference with its positive) and B × B push scores (every reference against every negative), each divided by τ.
  4. Apply the loss of Chapter 3 to get one number. Take a gradient step.

For small datasets, the paper draws pairs from the whole dataset instead of splitting a mini-batch into pairs. Doing so gives a better estimate of the pair distributions, and the authors say it considerably improves the embeddings. If the data fit in GPU memory, the whole dataset can serve as the batch. At test time you drop all pairs: pass any window through f and read the unit vector.

Failure, measured. Symptom: the ring is fuzzy or two nearby positions swap. Measurement: decoding error and consistency R² versus the window size. Fix: widen the window until it covers a few hundred milliseconds, but not much further, since a long window blurs fast changes.

The 10-bin model has kernels 2, 3, 3, 3, 3 without padding. After the last layer, how many time steps remain, and why?

Chapter 6

Testing Hypotheses with the Loss, and Counting Loops

Use the loss as a referee between ideas about the brain, and use topology to check that the shape of the map is real

Two scientists argue about the hippocampus. One says its neurons map space. The other says they mostly track the direction the animal runs. They could argue for years. CEBRA offers a quicker way: train one model with position as the label, one with direction, one with both, and compare how well each fits.

This is what the paper calls hypothesis-driven analysis. The label you pass in is the hypothesis. If the neural data truly carry that variable, the network can build a clean map, and the loss falls far below the collapsed-map line. If the neurons do not carry it, the loss stays high.

The loss as a goodness-of-fit score

Recall the gap Ln − log n from Chapter 3. Lower is better, and it does not depend on batch size. The rule is: train each model the same number of steps, then compare gaps. The label with the lowest gap explains the most.

The paper tests position, direction, both together and shuffled versions of each on its rat data. It finds that position plus direction is the most informative label. Shuffled labels, which break the link between neurons and behavior, converge to considerably higher loss values and give an unstructured embedding.

The decoding scores back this up. With a nearest-neighbour decoder on a 32-dimensional embedding, position plus direction gives an R² of 73.35% and a median absolute error of 5.8 cm. The shuffled run gives an R² of −49.90% and an error of 44.7 cm. A negative R² means the prediction is worse than always guessing the average position. The authors note that the pi-VAE paper reported an error of 12 cm on the same rat.

A small example, with made-up numbers. Suppose that after 1,000 steps the position model has a gap of −1.3, the direction model −0.1, and a shuffled-position model +0.4. Position explains much more than direction. The shuffled model shows what a useless label looks like on this data. Always keep a shuffled control: it tells you what “no relation” costs.

The failure to watch for is comparing at different points in training. The symptom is a model that looks better only because it trained longer. The measurement is the gap at the same step count for every label. The fix is to fix the step budget first, then read the gaps.

Discovery-driven: no label at all

What if you have no hypothesis? Train with time alone. The paper finds that the CEBRA-Time embedding of the rat data looks much like the position embedding. A smooth loop forms around the track. The structure is in the neural activity, and time is enough to bring it out. The paper advises running both variants, since each tells you something.

There is a theory fact behind the loss test. If some mapping exists between the label and the neural signal, CEBRA recovers it. If none exists, CEBRA learns a default embedding: points spread evenly over the hypersphere. A uniform blob is the sign of a label that says nothing.

Referee four hypotheses

A simulated rat runs a circular track in two directions. Some neurons prefer one direction. Press Run the four models. Each trains for 400 steps with a different label. Bars show the gap Ln − log n; left of the line is a better fit.

Toy lab, not the paper’s data: 20 neurons, 600 bins, direction flips half-way, neurons of the other direction fire at 35% of their gain, a linear encoder with 3 outputs on the sphere, 400 steps each. Gap = mean loss of the last 30 steps minus log n. Error = median absolute angle error of a nearest-neighbour decoder that trains on the first two thirds of the recording and is scored on the last third. In this toy a linear encoder cannot use direction much, so position alone and position plus direction come out close; the paper’s deep encoder did better with both.

Read the bars as a referee would. Position pulls the gap well below zero. Direction alone stays above the line. Shuffled position lands even further on the wrong side. In this toy, adding direction to position changes little, because a linear encoder has few ways to use it. The paper’s deep convolutional encoder had more room, and there the extra label gave the lowest loss.

Is the shape real? Counting loops

A pretty ring in 3-D could still be an accident. The authors add a second check from topology, the study of shape that ignores stretching. Their tool is persistent cohomology.

The idea: put a small ball around every embedded point. Grow all balls together. When two balls touch, their points join into one piece. When balls close up a hole, a loop is born. Later the loop fills and dies. A loop that lives for a long time is real. A loop that dies at once is noise.

The counts of pieces, loops and voids are the Betti numbers, written (H0, H1, H2).

ShapePieces (H0)Loops (H1)Voids (H2)
One blob100
A ring110
A sphere surface101

Points spread evenly over a 2-sphere give (1, 0, 1). That is the signature of a random embedding. The paper finds (1, 1, 0), a ring, for CEBRA-Behavior and CEBRA-Time on the rat data, as you would expect for place cells on a track. They set the cut-off between a real loop and noise from shuffled embeddings, so a loop must outlive whatever shuffling produces. They then use the first loop to give every time bin an angle on the ring. That angle matches the rat’s position, in every dimension they tried.

You train CEBRA with the shuffled position as the label. What do you expect, and why?

Chapter 7

Many Animals, Many Sessions, One Shared Space

Train one map across animals and recording types, then adapt it to a new animal in a handful of steps

A new patient arrives and wears a recording array. Their neurons are not the neurons of anyone before them. There are more or fewer, and each fires for different things. Yet you would like to start from what you learned on earlier patients, and be useful in minutes, not weeks.

The paper sets up this problem with rats, mice and monkeys. A single recording is a session. Sessions differ in neuron count, in day, in animal and even in recording method. How can one map serve them all?

What stays the same across sessions

The neural signal changes size from session to session. Session i has Ni neurons, so its data has shape (Ti, Ni). But the behavior label, say position, has the same shape in all of them: (Ti, m) with the same m. The label is the bridge. Two sessions at the same position should land at the same place on the sphere.

The paper builds this with three changes to the single-session recipe.

Without the shared pairs, two separately trained networks put the ring at arbitrary angles. Each ring is correct, and the two are consistent up to a linear change. But a decoder trained on one cannot read the other. With joint training, a given position lands at the same spot in both, and a decoder carries over.

One space for three animals

Three simulated rats (A, B, C) have 16, 22 and 19 neurons. Choose how to train A and B, press Train A and B, then adapt the new rat C. The score is the error of a position decoder fitted on rat A and applied to another rat.

Toy lab, not the paper’s data: 600 steps, 96 positives and negatives per step, a linear encoder to the unit circle per rat. Adapting C trains only its own encoder, with rats A and B frozen and used for positives and negatives. A random guess is off by 90 degrees (median). The paper’s own figures are in the text below; results here vary with the random seed, and the page uses a fixed one.

Compare the two modes. Trained separately, the rings point in unrelated directions, and the decoder from A fails on B. Trained jointly, the colors line up around the ring, and the decoder from A works on B almost as well as on A itself.

What the paper measured

Joint training raised consistency across subjects on the Allen Institute mouse data and on the rat hippocampus data. The paper used one-sided paired t-tests: t = −5.80 with p = 6.0 × 10−5 for Allen, and t = −2.22 with p = 0.024 for hippocampus. A fair worry is that a shared space loses what is special to one animal. The authors checked: decoding from the shared “pseudo-subject” space showed no loss in decoding performance.

A new animal in a few steps

This is the brain-machine-interface case. Pretrain on a subset of subjects. Then take the unseen one and train only its encoder, keeping the rest. The paper reports that adaptation runs at 50 to 100 steps per second, and that the position decoding error already dropped by 10 cm after adapting for one step. It also reaches a lower error more quickly than training on the new animal alone. The authors argue that this speed matters for real-time and adaptive decoding, and that pi-VAE is far slower, since its sampling to approximate the test label is costly.

Try it in the device. Before any adaptation, rat C’s encoder is random, and the error sits near chance. A handful of steps pulls its ring into line with the shared space.

Spikes and calcium in one space

The paper pushes the idea further. It takes mouse visual cortex recorded two ways, from different mice watching the same movie. One way is Neuropixels probes, which record electrical spikes. The other is two-photon calcium imaging, a fluorescent signal that follows activity slowly. The label is a feature vector for each movie frame from a vision transformer called DINO.

The two modalities gave similar embeddings, and joint training across them improved consistency without lowering consistency within a modality. The authors also compared areas. With joint training, consistency within an area was higher than across areas (Welch’s t-test, T = 4.55, p = 0.00019). The shared space did not erase real biological differences between areas.

Failure, measured. Symptom: rat B’s ring comes out rotated against rat A’s. Measurement: the error of a decoder fitted on A and applied to B. Fix: train A and B in one run with a shared behavior label.

Rats A and B have different numbers of neurons. How can one embedding space serve both?

Chapter 8

What the Figures Show, the Ablations, and the Limits

Read each result against its baseline, see which controls rule out the easy explanations, and know where the claims stop

A method is only as convincing as its comparisons. Each of the paper’s five figures asks one question of one dataset and sets CEBRA against a baseline or a control. This chapter walks through them in order, with the numbers the text gives.

Figure 1: does it recover a known truth?

The authors first make data where the answer is known. A hidden 2-D latent variable depends on a behavior label (a number from 0 to 2π). Four random invertible networks (RealNVP blocks) turn the latent into the firing rates of 100 neurons, and Poisson noise turns rates into spike counts. That gives 15,000 points. A method is then scored by how well its embedding, after a linear fit, recovers the true latent (R²).

CEBRA-Behavior scored significantly higher than pi-VAE, tSNE and UMAP over 100 runs each (one-way ANOVA F(3, 396) = 278.31, p = 3.95 × 10−97, post hoc Tukey p < 0.001). On the rat hippocampus data, CEBRA also gave higher consistency across animals than conv-pi-VAE with or without test-time labels (F(5, 67) = 28, p = 3.4 × 10−15).

Note the fairness step. The original pi-VAE reads one time bin. The authors added a convolutional front end so it could use a 10-bin window, which improved pi-VAE’s own results (median error 11 cm instead of 12 cm). They benchmarked against that stronger version, called conv-pi-VAE, and said so.

Figure 2: hypotheses, decoding, topology

The paper checks position, direction and both, then adds shuffled controls. Position plus direction wins, with a decoding R² of 73.35% against −49.90% for shuffled and a median error of 5.8 cm against 44.7 cm. Time alone beats PCA, tSNE and UMAP at decoding (F = 12,091). A persistent-cohomology check finds a ring (1, 1, 0).

Figure 3: a monkey reaching

A monkey moves a handle to eight targets, or has its arm bumped by the handle. Recordings come from area 2 of the somatosensory cortex. CEBRA-Behavior trained on hand position gives a clean trajectory. Active and passive movements land in different parts of the map. A 16-D CEBRA-Behavior model recovers the hand path with R² of 88% on 26 held-out trials. An L1 regression from all neurons gets 74%, and 16-D conv-pi-VAE gets 82%.

Figures 4 and 5: a mouse watching a movie

Mice watch a natural movie ten times. The label of each frame is its DINO feature vector. CEBRA trained on Neuropixels or calcium data gives smooth trajectories that follow the movie, and the two recording types agree (Chapter 7). For decoding, the paper trains on nine repeats and holds out the tenth. With a nearest-neighbour or naive-Bayes decoder on the CEBRA embedding, frame accuracy is above 95%, where a frame counts as right if the guess falls within a 1-second window, or if the right scene is identified. Baselines using the raw population vector with the same decoders are significantly lower; the text prints no values for them, and the supplement gives pairwise tests.

The headline numbers

Pick a dataset. The bars are numbers printed in the paper, with the figure that shows them. Note which way is better: error falls, R² and accuracy rise.

Numbers copied from the arXiv v2 text: rat, Fig. 2d and Methods (CEBRA-Behavior with position and direction 5.8 cm, shuffled 44.7 cm, conv-pi-VAE 11 cm, pi-VAE as reported in its own paper 12 cm); monkey, Fig. 3i text (88%, 82%, 74%); mouse, text and Fig. 5 (“greater than 95%” for CEBRA; baselines are lower but their values are not printed in the text). Runs, seeds and test sets differ between figures, so compare bars only within one dataset.

Controls and ablations that matter

Several controls rule out the easy stories. Shuffled labels give a high loss and poor decoding, so the label carries the information (Chapters 4 and 6). Shuffled DINO features give poor movie decoding, so the features alone do not drive the result. Time-only CEBRA already gives a similar ring, so the structure is in the neurons. Higher dimensions can raise consistency (Extended Data). And layers matter: in V1, layers 2/3 and 5/6 decode better than layer 4 (F(2, 12) = 9.88, p = 0.003). The authors suggest that the non-thalamic input layers make frame information more explicit, perhaps through feedback or predictive processing. That is their interpretation, not a tested claim.

Where the claims stop

With shuffled DINO features as labels, movie decoding from V1 is poor. What does that control show?

Chapter 9

Field Guide: CEBRA on One Page

Keep the numbers, the checklist and the core code, and find where to go next

CEBRA trains a small network to map windows of neural activity to unit vectors. It pulls together windows that share a label or sit close in time, and pushes apart the rest, using the InfoNCE loss. The label and the sampling rule are the scientist’s hypothesis. The loss then tells you how well that hypothesis fits, and the maps from different animals agree up to a linear change.

Pick a recipe for your goal

Choose your goal

Pick what you want to do. The card shows the label, the positives, the negatives and the check that tells you it worked.

A summary of the Methods and Results of the paper, written as a recipe. The rule for invariance is the one in Chapter 4; the checks are the ones used in Chapters 1, 6 and 7.

The numbers and terms

QuantityValueWhere it comes from
Embedding sizeunit-length vectors; n + 1 CEBRA dimensions match n dimensions of other methodsMethods, hypersphere
Loss of a collapsed maplog n for n negativesMethods, goodness of fit
Goodness of fit−DKL(p‖q) ≤ Ln − log n ≤ 0, lower is betterEquation 3
Receptive fields1 bin; 10 bins (250 ms for the rat); 40 bins (Neuropixels at 120 Hz)Methods, CEBRA models
Batch size in the paper’s runs512Methods, model parameters
Rat position error5.8 cm with position + direction; 44.7 cm shuffled; 12 cm reported for pi-VAEFig. 2d and text
Rat decoding R²73.35% against −49.90% shuffledFig. 2d
Monkey hand path R²88% CEBRA; 82% conv-pi-VAE; 74% L1 regressionFig. 3i text
Mouse V1 movie framesover 95% accuracyFig. 5 text
Adapting to a new animal50 to 100 steps per second; error down 10 cm after one stepResults, multi-session
Consistency tests12 pairs across 4 rats; 90 comparisons across 10 runsMethods
Topology of the rat mapBetti numbers (1, 1, 0), a ring; a random sphere gives (1, 0, 1)Fig. 2f, 2g

A checklist before you trust a map

  1. Write down the label and the hypothesis it stands for.
  2. Check that labels and neural data share the same clock.
  3. Pick a window that covers a few hundred milliseconds.
  4. Train a shuffled-label control for the same number of steps.
  5. Compare the gap Ln − log n, not the raw loss.
  6. Decode held-out data with a simple nearest-neighbour decoder.
  7. Repeat the run and measure consistency (R²) across runs, then across animals.
  8. For topology, count only loops that outlive the shuffled reference.

The core as code

python# One training step, written out. f maps a window (N, RF) to a unit vector (D,).
for step in range(steps):
    t     = rng.integers(0, T, B)                  # B references
    t_pos = sample_positive(t, c, k)                # Chapter 4: time offset or closest behavior
    t_neg = rng.integers(0, T, B)                # random negatives
    ref, pos, neg = f(window(s, t)), f(window(s, t_pos)), f(window(s, t_neg))   # (B, D) each
    loss  = info_nce(ref, pos, neg, tau)            # Chapter 3
    opt.zero_grad(); loss.backward(); opt.step()

# The package, as printed in the paper (check the docs for the current API).
import cebra
model = cebra.CEBRA(output_dimension=8, num_hidden_units=32,
                    batch_size=1024, learning_rate=3e-4, max_iterations=1000)
model.fit(s)                         # time only (discovery)
model.fit(s, k)                      # discrete behavior
model.fit(s, c)                      # continuous behavior (hypothesis)
model.fit([s, s2], [c, c2])          # two sessions, one space
model.fit(s2, c2, adapt=True)        # adapt to a new session
z = model.transform(s_test)              # (T, 8) embedding, no labels needed

Where it sits in the field

CEBRA is one answer to a question the course keeps asking: how do we find the few numbers that matter in a large population of neurons? It sits between linear tools such as PCA and ICA, generative sequence models such as LFADS, and the contrastive methods of self-supervised learning. Its consistency across sessions bears on a hard BCI problem, drift, treated in Drift and Recalibration.

The takeaway. CEBRA swaps a model of how the brain makes spikes for a statement of what should be close. The label is a hypothesis you choose. The loss scores it, a shuffled control sets the floor, and the shared space across animals makes the result reusable.

Credit

Steffen Schneider and Jin Hwa Lee (co-first authors) and Mackenzie Weygandt Mathis, EPFL Brain Mind Institute and Neuro-X. Paper: arXiv 2204.00673, published in Nature in 2023 (doi 10.1038/s41586-023-06031-6). This lesson follows the arXiv v2 text, so numbers may differ from the journal version. Code: github.com/AdaptiveMotorControlLab/CEBRA; docs: cebra.ai/docs. Data used in the paper: the CRCNS hc-11 rat hippocampus set, the DANDI 000127 primate set, and the Allen Institute Brain Observatory mouse sets. Toy labs on this page are ours and are not the authors’ code or data.

You want to know whether a neural population carries the animal’s speed, and you have no other claim. Which plan follows the paper?

Now press Teach and explain CEBRA at the whiteboard, from memory: the pairs, the loss, the gap to a collapsed map, and what makes two animals’ maps consistent. Then go back to the hero and train all three pull signals once more.

CEBRA: learnable latent embeddings for joint behavioral and neural analysis
Read the paper · Code · Docs · Lecture: latent variable models · Back to Gleams