EEG Foundation Models

EEG caps differ in channels and length. LaBraM cuts every recording into 1-second patches, learns to name hidden ones, and lifts TUEV balanced accuracy from 0.5281 to 0.6409.

Learn how one pre-trained model reads brain recordings from any cap: the patches, the spectrum tokenizer, the masked prediction, and the criss-cross attention that followed.

Pick a cap, then drag the share of hidden patches. Watch one 256-patch budget hold a 23, 32, 62 or 64 channel recording. Then we build every piece, one mechanism at a time: the patch, the encoder, the Fourier target, the codebook, the mask, the results and the ablations, and CBraMod’s criss-cross attention.

You need a rough idea of what a Transformer does (it lets every piece of an input look at every other piece) and a little Python for the code. We teach EEG, the Fourier transform and vector quantization from zero.

Based on Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI (LaBraM), Wei-Bang Jiang, Li-Ming Zhao and Bao-Liang Lu, ICLR 2024, and on CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding, Jiquan Wang, Sha Zhao and colleagues, ICLR 2025. EEGPT (Wang et al., NeurIPS 2024) appears as context. This paper is a reading of the lecture Spatial Filters and CSP. Every idea is re-taught here in our own words and devices.

Fit any cap into one budget

50%

The caps are datasets named in the LaBraM paper (channel counts from its Appendix D and Section 3.1). The rule is the paper’s: patches of 200 samples (1 second at 200 Hz) and at most 256 patches per sample, so 64 channels give 4 seconds and 32 channels give 8 seconds. Which patches hide is a seeded toy draw. The paper hides 50% in its main setting.

Chapter 0

Why One Model Per Dataset Breaks

See why every EEG network built so far fits one cap, one task and one recording length, and what a foundation model would change

Picture two labs. A hospital lab records patients with a cap of 23 electrodes, sampled 256 times a second. A psychology lab shows film clips to volunteers and records with 62 electrodes, sampled 1,000 times a second.

A team trains a neural network on the hospital data to tell normal brain recordings from abnormal ones. It works. Now someone hands it a file from the psychology lab. The network expects 23 rows of numbers. The file has 62. The network cannot even read it.

This is the daily situation in EEG research, and it is the problem the two papers in this lesson attack. First we need the basic words.

What an EEG recording is

Electroencephalography (EEG) measures the tiny electrical activity of the brain with metal discs on the scalp. Each disc is an electrode. The voltage at one electrode, followed over time, is one channel. Think of each channel as one microphone in a large hall, each hearing the crowd from its own seat.

Electrodes sit at standard spots named by the international 10–20 system, such as Fp1, Cz or O2. The letters name a brain region and the number names a side and a place. A recording is then a table of numbers.

X
One recording: a table of voltages.
C
The number of channels, one row per electrode. It changes with the cap.
T
The number of time samples in each row. It equals the recording time in seconds times the sampling rate, the count of voltage readings per second, in hertz (Hz).

The paper states that EEG values usually fall between −0.1 mV and 0.1 mV. That is a ten-thousandth of a volt. Every number in X is tiny, and noise from muscles, eyes and mains power can be as large as the signal.

Why every model so far is a one-off

Both papers list the same walls. First, labels are scarce. A label is an expert’s answer for a recording, such as “abnormal”. Experts are slow and expensive, so most labeled EEG sets cover a few dozen people and less than tens of hours.

Second, formats clash. Caps differ in channel count, labs differ in sampling rate, and tasks differ in recording length. A network with a fixed input shape fits exactly one of these.

Third, the signal is weak. The LaBraM authors call out the low signal-to-noise ratio (SNR), the size of the useful signal against the noise. Many researchers still design features by hand because learning from raw EEG is hard.

Try the second wall yourself. The chart lists ten datasets named in the LaBraM paper. Each bar is the count of numbers in ten seconds of that dataset. Pick the dataset your imagined network targets.

Count the recordings your network can read

Each bar is channels × 10 seconds × sampling rate. A green tick means the shape matches the network you picked. A red cross means the network cannot take that file as it is.

Channel counts and sampling rates are from the LaBraM paper (Appendix D, and Section 3.1 for Temple’s TUAB and TUEV). “Fits” is the toy rule of a fixed input: the same channel count and the same sampling rate. Real caps also differ in electrode names and reference wiring, which makes the gap wider.

Almost nothing fits. The only dataset that matches is the one the network targets. A lab that wants to reuse a model must rebuild it, retrain it, and find new labels.

The bet: learn first, label later

Text faced a similar wall. A language model reads huge amounts of unlabeled text, learns by filling in hidden words, and only then gets a small labeled task. This recipe is self-supervised pre-training: the data supplies its own exercises, so no expert is needed. The model that results is called a foundation model because many tasks build on it.

LaBraM, the Large Brain Model, tries the same recipe on EEG. The authors collected about 2,500 hours of EEG from about 20 datasets, mostly public and some recorded by themselves. The authors kept the four test datasets out of pre-training. They trained three sizes, from 5.8 million to 369 million parameters, and call these the largest models in the field so far.

To make the recipe work they must answer three questions. How do we cut any recording into a standard unit? What do we ask the model to predict, given the noise? And how do we hide pieces so the exercise teaches something? Each answer is one chapter.

The road

  1. Channel patches: cut any recording into one-second pieces, and fix the budget of 256.
  2. The patch encoder: turn 200 samples into one vector, then tag it with time and place.
  3. The Fourier target: why the model predicts a spectrum and not the raw wave.
  4. The neural tokenizer: a codebook of 8,192 EEG words learned by vector quantization.
  5. Masked EEG modeling: hide half the patches, name their codes, and use the mask twice.
  6. Results and scale: Tables 1 and 2, and what more data and parameters buy.
  7. Ablations: which design choice carries the gain.
  8. CBraMod and EEGPT: criss-cross attention and the field one year later.
  9. Field guide: numbers, a checklist, the core in code, and where to go next.
A network is trained on 23 channel EEG at 256 Hz. Which file can it read without any change?

Chapter 1

Cut Any Recording into Channel Patches

Turn a table with any number of rows into a list of equal tiles that a Transformer can read

Imagine tiling a floor. Rooms come in every shape, but if every tile is the same one-foot square, you can cover any room. You only need to count the tiles.

LaBraM does the same to EEG. A recording has any number of channels and any length. The model never sees that table. It sees a list of identical tiles, each one a single channel for a single second.

From a table to a list of tokens

A Transformer reads a list of equal-size pieces called tokens. In text a token is a word piece. In images the Vision Transformer made a token from a 16 by 16 block of pixels (the Vision Transformer lesson builds this). The LaBraM paper copies that idea.

Take a sample x with C channels and t time points. Pick a window length w. Cut every channel into non-overlapping windows of w samples. Each window is an EEG channel patch.

xc,k
One patch: the w voltage readings of channel c during window number k. It is a plain vector of w numbers.
|x| = C · ⌊t / w⌋
The patch count. Each of the C channels gives ⌊t/w⌋ windows (the floor drops any partial window at the end).

The paper sets w to 200 samples. After resampling to 200 Hz, one patch is exactly 1 second of one channel. A longer recording is first cut into samples of t time points, moving ahead by a stride. During pre-training the stride is 4 seconds.

Clean the signal just enough

The authors apply “very little” preprocessing. Each step has one job.

  1. Band-pass 0.1 to 75 Hz. A band-pass filter keeps only frequencies inside a range. This drops slow drift and fast noise. The EEG preprocessing lesson goes deeper.
  2. Notch at 50 Hz. A notch filter removes one narrow frequency. The paper uses 50 Hz, the mains hum of the power grid. CBraMod later uses 60 Hz, since its data came from a US hospital.
  3. Resample to 200 Hz. Every dataset now runs on the same clock, so every patch holds 200 numbers.
  4. Scale by 0.1 mV. Voltages are divided by 0.1 mV so most values land between −1 and 1, a range networks train well on.

The budget of 256

The patches of one sample become one sequence. The model compares every patch with every other patch, so the cost grows with the square of the sequence length. For stable cost the paper caps the sequence at 256 patches. Then 256 × 256 = 65,536 pairs are compared in each attention head.

A fixed budget in patches means a flexible length in seconds. The paper gives two cases: a 64 channel cap gets 4 seconds per sample, and a 32 channel cap gets 8 seconds.

Now play with it. Choose a channel count and a number of seconds. The grid shows the patches the model would receive, and what breaks when you ask for too many.

Count the patches in a sample

Each tile is one channel for one second, 200 numbers. Blue tiles fit the budget. Red tiles do not. Drag the sliders, or press a preset.

23
5 s

The rule (200 samples per patch, at most 256 patches) and the 64 and 32 channel examples are the paper’s (Section 3.2). The 5 s and 10 s presets are the TUEV and TUAB sample lengths from Section 3.1. The pair count is the exact number of query-key pairs in full attention over the sequence.

Try the last preset. A 64 channel cap with 10 seconds makes 640 patches. That is 2.5 times the budget and 6.25 times the pairs. The fix is the stride: cut the 10 seconds into windows of 4 seconds and treat each as its own sample.

What does a patch lose? Inside one patch, the model sees one electrode only. Links between electrodes appear later, in attention, where all patches meet. Links inside the one second appear earlier, in the convolutions of Chapter 2. The cut is a trade: a standard unit now, in return for pushing every cross-channel question into the Transformer.

The failure mode is easy to meet. Symptom: you feed a 64 channel, 10 second file and the sequence is 640 patches. Measurement: 640 × 640 = 409,600 pairs, 6.25 times the pairs the budget allows, and 2.5 times the memory. Fix: cut the file into 4 second samples before patching.

A worked example with shapes

Take one sample of SEED, an emotion dataset with 62 channels recorded at 1,000 Hz. The data flow runs as follows.

  1. Raw: (62, 4000) for 4 seconds. Resample to 200 Hz and the shape becomes (62, 800).
  2. Patch: 800 / 200 = 4 windows per channel. The shape becomes (62, 4, 200).
  3. Flatten: 62 × 4 = 248 patches, each a vector of 200 numbers. 248 is below 256, so it fits.

Notice what the model does not know yet. After flattening, a patch has no label for which channel it came from or when. A patch from Cz at second 3 looks like a bag of 200 numbers, same as any other. Chapter 2 fixes that.

After resampling to 200 Hz, a 32 channel recording of 8 seconds becomes a sequence of how many patches, and why does that suit the budget?

Chapter 2

Encode Each Patch, Then Tell It Where It Sits

Compress 200 voltages into one vector, add a time tag and a place tag, and feed the Transformer

A mail room receives 115 envelopes. Each holds a sheet with 200 numbers, but no return address. The sorter cannot tell which electrode sent a sheet, or at which second. Before sorting, someone must summarize each sheet and stamp it with a place and a time.

That someone is the first part of LaBraM, shown in the paper’s Figure 1. Three steps turn a patch into a token: encode, tag, then pass to the Transformer.

Step 1: the temporal encoder

Why not feed the 200 raw numbers straight to attention? The authors note that EEG is rich in fine time detail. So they pull out time features first, patch by patch, using convolutions.

A 1-D convolution slides a short window of weights, the kernel, along the signal. At each stop it multiplies and adds, so it lights up where the signal looks like the kernel. The stride is how far the window jumps. Padding adds zeros at both ends. The output length follows one formula.

L
The number of steps in the output.
n
The number of steps going in. For the first layer, n = 200 samples.
p
Padding on each side, in zeros.
k
The kernel length.
s
The stride.

The paper’s Table 3 gives three convolution blocks. Each block is a convolution, then group normalization (it rescales groups of channels inside one sample, so it does not depend on batch size), then GELU (a smooth switch that passes positive values and softly blocks negative ones).

The settings are: input channels 1, 8, 8; output channels 8, 8, 8; kernel 15, 3, 3; stride 8, 1, 1; padding 7, 1, 1. Work the first block: L = ⌊(200 + 14 − 15) / 8⌋ + 1 = ⌊24.875⌋ + 1 = 25. The next two blocks keep 25, since (25 + 2 − 3) / 1 + 1 = 25.

After three blocks the patch is a table of 8 channels by 25 steps. Flattened, that is 8 × 25 = 200 numbers, the same as the Base hidden size d = 200. The Large model uses 16 channels and d = 400. The Huge model uses 32 channels and d = 800. The match holds all three times. The paper does not write out the flatten, so this is our reading of its tables.

Set the dials below to check the arithmetic, and break it.

Trace one patch through the encoder

Choose a model size, then change the first block’s kernel, stride and padding. The last bar must land on the hidden size.

15
8
7

Kernel 15, stride 8, padding 7 and the blocks after it (kernel 3, stride 1, padding 1), with 8, 16 and 32 output channels, are Tables 3 and 4 of the paper. The hidden sizes 200, 400 and 800 are Table 4. That the encoder output is flattened to d numbers is inferred from these tables, not stated in the text.

Only the paper’s setting lands on 200. Slide the stride to 4 and the patch becomes 50 steps, so the flatten gives 400. The block would no longer fit the Transformer without another layer.

Step 2: tag each patch with a time and a place

Now each patch is a vector e of d numbers. The paper adds two embeddings, which are learnable vectors looked up from a table.

The temporal embedding list TE has one entry per time slot, up to a limit tmax. The spatial embedding list SE has one entry per electrode of the universal 10–20 set. Cz has one entry, whatever the cap.

e
The patch vector from the temporal encoder, d numbers.
tek
The learned vector for time slot k. It says “this is second k”.
seij
The learned vector for electrode ij of the 10–20 set. It says “this came from Cz”.

This is how one model serves many caps. A 23 channel and a 62 channel recording each pick their own rows of SE. The same electrode name always gets the same vector. Missing electrodes simply never appear in the sequence.

Step 3: the Transformer encoder, with two changes

The tagged tokens go into a standard Transformer encoder (the Transformer lesson builds it). Base has 12 layers, 10 heads and a feed-forward width of 800. Two edits come from the ViT-22B recipe of Dehghani and colleagues.

First, a layer norm (LN, which rescales a vector to zero mean and unit spread) is applied to queries and keys before the dot product. Second, the bias terms in the Q, K and V projections are dropped, which speeds up training with no loss.

Attention
For each patch, a weighted mix of all patches’ values V. The weights come from the softmax.
LN(Q) LN(K)T
The match score of every query with every key, after layer norm. These are the attention logits.
dhead
The width of one head: d divided by the head count. For Base it is 200 / 10 = 20.

Why this helps: a normalized vector of width 20 has length √20. By the Cauchy–Schwarz bound, a dot product of two such vectors is at most 20. Divide by √20 and every logit stays within ±4.5. Without the LN, logits can grow without limit, and the softmax collapses onto one patch. (The bound is exact at unit gain, the layer norm’s starting setting.)

For downstream tasks the paper averages the output vectors over all patches and adds a task head. Base has 5.8 million parameters.

What does the spatial embedding add to a patch, and why does it let one model handle different caps?

Chapter 3

Predict the Spectrum, Not the Wiggles

See why LaBraM asks its tokenizer to rebuild the Fourier amplitude and phase of a patch instead of its raw voltages

You record the same piano chord twice in a noisy room. The two sound files look different, wiggle by wiggle, because the room noise differs. But a musician hears the same three notes both times. The notes are the stable part.

EEG has the same trouble, and worse. A model that must rebuild every wiggle is asked to copy noise. The LaBraM authors report that when they tried this, the loss did not converge. So they changed the target.

The Fourier transform in plain words

Any wave is a sum of pure sine waves of different pitches. The Fourier transform finds the recipe: for each frequency, how much of that sine is present, and where in its cycle it starts. The discrete Fourier transform (DFT) does this for a list of samples. (The DFT lesson derives it from scratch.)

The brain makes rhythms at typical pitches. The alpha rhythm sits near 10 Hz and appears when you close your eyes. The DFT shows it as a tall spike at 10 Hz. See the brain rhythms lesson.

The paper writes the DFT of one patch x with w samples. It tests each frequency index m.

X̃m
A complex number for frequency index m. It tests how well the patch matches a sine that completes m cycles in the patch. For 200 samples at 200 Hz, index m means m Hz.
Am
The amplitude: how strong that frequency is. It is the length of the complex number.
φm
The phase: where in its cycle that sine starts. It is the angle of the complex number.

A worked number. Take a patch that is a pure 10 Hz sine of height 1, sampled 200 times. Index 10 gives X̃ = 100, so the amplitude is 100 (half of the 200 samples, times the height). Every other index gives 0. A sine of height 0.5 would give 50. The spectrum tells us what is present and how strongly.

Because the voltages are real numbers, the second half of the spectrum mirrors the first (the DFT is conjugate symmetric). So a patch of 200 samples has 100 useful frequencies, 1 Hz to 100 Hz. The paper’s Figure 7 shows half the spectrum for this reason.

What the tokenizer is asked to rebuild

Every patch gets an amplitude list and a phase list. Within each sample the paper applies a z-score to both: subtract the mean, divide by the spread. This makes the numbers comparable and the training stable. A decoder then predicts both lists, and the loss is a plain mean squared error on each.

Phase is the harder half. The paper reports that the amplitude trend is rebuilt well, the phase less so, though the reconstruction loss still falls steadily. Try the toy below to feel why amplitude is the steadier target.

Compare two noisy copies of one rhythm

Top: two recordings of the same 10 Hz and 20 Hz rhythm with different random noise. Bottom: the amplitude of each, 1 to 60 Hz. Raise the noise, then draw new noise.

1.0

Illustrative toy, not EEG data: a 10 Hz sine of height 1 plus a 20 Hz sine of height 0.5, 200 samples at 200 Hz, plus white noise. Real EEG noise is not white and not stationary. The patch length and the DFT definition are the paper’s (Eqs. 5 to 8).

At noise level 1.0 the two waveforms agree only weakly, while the two spectra still show the same two peaks. That is the argument for the spectrum: it keeps the part of the patch that is repeatable.

Why phase is the harder half

Phase is an angle, so it wraps around: 359 degrees and 1 degree are close, but their difference looks huge. It also reacts to tiny timing shifts. Shifting a patch by Δt seconds moves the phase of frequency f by 360 × f × Δt degrees. A 1 ms shift moves a 10 Hz wave by 3.6 degrees. It moves an 80 Hz wave by 28.8 degrees.

Amplitude has no such trouble. A 1 ms shift leaves it untouched. This is why the paper’s Figure 7 shows amplitude rebuilt well and phase less so. The training still gains from phase, since the loss falls steadily.

The failure mode, then, has a clear shape. Symptom: a tokenizer whose decoder must copy raw voltages stalls, and its loss does not converge (the authors report this for their tokenizer, not for the later masked prediction). Measurement: in the toy above, two noisy copies of one rhythm barely correlate as waves. Fix: aim at the part that repeats, the spectrum.

Where this target goes next

The spectrum is not the final prediction. It is the exam the tokenizer must pass. The tokenizer squeezes each patch into one code from a small list. The decoder must rebuild the spectrum from those codes alone. If the codes keep what the spectrum needs, they keep the rhythm content of the patch. Chapter 4 builds that tokenizer.

Why does LaBraM train its tokenizer to rebuild the Fourier amplitude and phase and not the raw voltages?

Chapter 4

Give EEG a Vocabulary of 8,192 Codes

Build the neural tokenizer: match each patch to its nearest code, and train the codes by rebuilding the spectrum

A paint shop sells exactly 8,192 colors. You bring a photo of a wall, and the clerk names the closest shop color for every square inch. You walk out with a list of color numbers. You lost fine shades, but you kept what mattered, and the list is short.

A language model predicts a word out of a fixed vocabulary. EEG has no words. The LaBraM authors build a vocabulary by learning it, and call the result a neural tokenizer with a neural codebook.

Vector quantization, step by step

The tokenizer reuses the encoder of Chapter 2 (a temporal encoder, tags, and 12 Transformer layers) to turn each patch into a vector p of fixed width. The codebook V holds K = 8,192 learned vectors, each of width D = 64 (Table 3 of the paper). Vector quantization replaces p with the nearest entry of the codebook.

zi
The chosen code for patch i. It is just a whole number from 0 to 8,191, the patch’s token.
ℓ2(·)
L2 normalization: divide a vector by its length, so it has length 1. Distance between two such vectors then ranks them by the angle between them, which is the same as ranking by cosine similarity.

Why normalize? Without it a long vector and a short vector pointing the same way would pick different codes, since distance also measures length. After normalization only direction counts. The paper cites earlier work showing this improves codebook utilization, meaning more of the 8,192 codes actually get used. Try it on two dimensions.

Match a patch vector to its nearest code

Pink dots are 16 toy codes. The teal dot is a patch vector. Turn it, then stretch it. Strip: the code chosen as the length sweeps from short to long.

100°
1.20

Toy: 16 codes in 2 dimensions with seeded positions and lengths. The real codebook has 8,192 codes of width 64 (Table 3). The nearest-code rule with L2 normalization is the paper’s Eq. 4. That normalization improves utilization is the paper’s citation of Peng et al. 2022, not something this toy measures.

In raw mode, stretching the vector can send it to a far code, so the strip changes color along the way. In L2 mode the strip is one color from end to end, because length no longer enters.

A tiny worked case. Let a patch vector be (3, 4), which has length 5. Divided by 5 it becomes (0.6, 0.8). A code vector (0.8, 0.6) has cosine similarity 0.6 × 0.8 + 0.8 × 0.6 = 0.96 with it. A code (1, 0) scores only 0.6. The patch picks the first code, and would pick it again as (6, 8) or (0.3, 0.4).

Training the codes by rebuilding the spectrum

The codes must mean something. The paper trains them with a rebuild task. Each patch becomes its nearest code vector. A three-layer Transformer decoder reads the code vectors, averages them, and two small heads predict the patch’s z-scored amplitude and phase (Chapter 3). Training makes the codes carry what the spectrum needs.

LT
The tokenizer loss, summed over all training samples and patches.
codebook term
Pulls the chosen code toward the patch. sg means stop-gradient: the value is used, but no learning signal flows through it. Here the patch is held still.
commitment term
Pulls the patch toward its code. Here the code is held still. This keeps the tokenizer from drifting away from the codebook.

The first two terms are mean squared errors on amplitude and phase. The last two are the standard VQ-VAE pair (the VQ-VAE lesson derives them). The paper also updates the codebook with an exponential moving average, a running average of the patches assigned to each code. That makes the codebook update steadier.

What comes out, with shapes

Take a TUEV sample: 23 channels, 5 seconds. Patches: (115, 200). Tokenizer encoder: (115, 64). Nearest code, per row: 115 whole numbers between 0 and 8,191. This list is the sample’s neural token sequence, a sentence in the EEG vocabulary. It is the answer key for the next chapter.

The tokenizer trains first, with 12 encoder layers, 3 decoder layers, a batch of 1,024 and 100 epochs. After that it is frozen. The paper’s Appendix B views the whole recipe as a two-stage variational autoencoder, a discrete one: stage one learns the codes, stage two learns to predict them from masked input.

One failure to know. If all patches fall on a handful of codes, the vocabulary collapses, and a model that predicts the common code looks accurate while learning nothing. The symptom is a few codes dominating the counts. L2 normalization and the moving-average update are the paper’s guards.

What does the neural tokenizer output for a sample with 115 patches, and what is it trained to do?

Chapter 5

Hide Half the Patches and Name Their Codes

Pre-train the Transformer with masked EEG modeling, and reuse every mask twice with symmetric masking

A teacher hands out a page of text with some words blacked out. You fill in each blank from the words around it. Do this a million times and you learn the language, without anyone telling you what any sentence means.

LaBraM plays this game on brain recordings. It covers some patches of the grid, and the model must name them. The twist is what “name” means here: the model must give the tokenizer’s code for the hidden patch.

One masked training step

Use the sample from Chapter 4: 115 patches, and for each one a code from the tokenizer. The step has five moves.

  1. Encode. The temporal encoder gives 115 patch vectors e, each of width d = 200.
  2. Mask. Draw a random mask M. A share r of the 115 patches are marked hidden. Each hidden vector is replaced by one learnable mask token eM, a vector the model learns to read as “blank”.
  3. Tag. Add the temporal and spatial embeddings (Chapter 2). The blank now knows which channel and which second it stands for.
  4. Transformer. The 115 tagged vectors go through the encoder. Out come 115 hidden vectors h, still of width 200.
  5. Predict. A linear layer turns each h into 8,192 scores, and a softmax makes them probabilities over the codebook.
p(v′ | eM)
The model’s probability for each of the 8,192 codes v′, given the corrupted input eM.
LM
The masked loss. For each hidden patch (mi = 1), take minus the log of the probability the model gave to the true code vi from the tokenizer. Sum over all hidden patches of all training samples D. Visible patches add nothing.

This is a cross-entropy loss: a classification with 8,192 classes. Guessing at random scores 1/8,192, about 0.012%. Training raises the score. The paper’s Figure 3 shows the loss falling and the masked accuracy rising, with a larger model reaching a lower loss and a higher accuracy.

Look at why the spatial embedding matters here. A blank carries no signal. Only its place tag says which patch to guess. The paper reports that without spatial embeddings the pre-training loss could not converge.

What must the model learn to answer? A hidden patch at Cz in second 3 has neighbors: Cz in seconds 2 and 4, and other electrodes in second 3. The model can only guess well by using both kinds of context. Slow rhythms carry across seconds and across nearby electrodes. That is the knowledge pre-training stores.

The paper’s Appendix B gives a second view. The whole recipe is a variational autoencoder over discrete codes. Stage one trains the tokenizer to rebuild the spectrum from codes. Stage two, this chapter, learns to predict the codes from a damaged input. Splitting it in two keeps each stage simple.

Symmetric masking: use each mask twice

The tokenizer costs one extra forward pass per sample, since it must produce the answer key. The authors want to get more from that pass. So they flip the mask.

Take the mask M. Its inverse M̃ hides exactly the patches M left visible. Run the same sample again with M̃. The same codes are the answer key, so the tokenizer is not called again.

L
The full pre-training loss: the loss under mask M plus the loss under its inverse. The two share one tokenizer output.

The paper gives two reasons. It reuses the discrete codes, so training is more efficient. And it gives the batch more masking views, which acts like extra data. Appendix I tests it, and Chapter 7 shows what it found.

See both masks at work on a toy grid of 8 channels by 8 seconds.

Mask once, then use the mask twice

Orange tiles are hidden and become targets. Blue tiles are seen. Switch on symmetric masking to add the inverse pass, then change the share.

0.5

A toy grid of 64 patches, not a real sample (real samples hold up to 256). The rule is the paper’s (Sections 2.3 and Appendix I): the mask hides a share r, the inverse hides 1 − r, and the tokenizer is called once. The paper’s default is r = 0.5.

With r = 0.5 the two masks split the grid in half and every patch becomes a target once. With another r the inverse mask hides the rest, such as 60% after 40%, as in the paper’s Table 8 where “0.4 (0.6)” names the pair.

The mask share is a dial with two failure modes. Hide too little and the guess is trivial, since neighbors give it away. Hide too much and nothing is left to reason from. The paper tested r from 0.1 to 0.9 and found a broad plateau, with 0.5 the safest (Chapter 7).

The settings

The paper pre-trains with a batch of 512 samples, a peak learning rate of 5e-4, 50 epochs and the AdamW optimizer. It uses eight A800 GPUs. All of this runs on the 2,534.78 hours of EEG, with the four test datasets held out. Chapter 6 shows what that bought.

What does symmetric masking add to LaBraM’s pre-training step?

Chapter 6

Read the Report Card: Tables 1 and 2

Fine-tune LaBraM on two clinical exams, learn what each metric means, and see what more parameters and more data buy

A student finishes years of general reading. Now two exams arrive, each from a different subject. The honest question is not “did they pass” but “by how much, against whom, and how steady was the score”. This chapter reads the paper’s two main tables that way.

The two exams

TUAB asks: is this recording normal or abnormal? It comes from the Temple University Hospital corpus. It has 23 channels at 256 Hz and 409,455 samples of 10 seconds each.

TUEV asks a harder question. Which of six event types is in this 5-second window: spike and sharp wave, generalized periodic discharge, periodic lateralized discharge, eye movement, artifact, or background? It has 112,491 samples. Both sets are held out of pre-training.

How fine-tuning works

Fine-tuning means continuing to train a pre-trained model on a labeled task. LaBraM averages the output vectors over all patches, adds a small task head, and trains the whole network. It uses binary cross-entropy for TUAB and cross-entropy for TUEV.

Settings from the paper’s Table 5: batch 512, peak learning rate 5e-4, 50 epochs for Base (30 for Large and Huge), and a layer-wise learning rate decay of 0.65 for Base. That decay gives layers near the input a smaller learning rate, so pre-trained features move gently. The splits follow the BIOT paper, with training patients divided 80% and 20% into training and validation. Every number is the mean over five random seeds, with its spread.

The metrics, with a worked case

Balanced accuracy is the average recall of each class. Say a test set has three classes with recalls 0.9, 0.5 and 0.1. Balanced accuracy is (0.9 + 0.5 + 0.1) / 3 = 0.5. Plain accuracy could look far better if the first class dominates. Balanced accuracy refuses to hide a neglected class.

Cohen’s kappa measures agreement beyond chance. If the model matches the experts 80% of the time and chance alone would match 50% of the time, kappa = (0.8 − 0.5) / (1 − 0.5) = 0.6. AUROC and AUC-PR are areas under the receiver-operating and precision-recall curves, both across all decision thresholds. Weighted F1 combines precision and recall per class and weights each by its size.

Now read the tables. Pick an exam and a metric. The bars start above zero so differences show, which the axis label states.

Compare nine models on one exam

Warm bars are LaBraM. Blue bars are the earlier models the paper compares against. The thin line on each bar is plus or minus one standard deviation over five seeds.

Values are Tables 1 and 2 of the LaBraM paper (mean over five seeds; the baselines are the best results from the BIOT paper). The bars start at 0.70, 0.80 and 0.80 for the TUAB metrics, at 0.30, 0.30 and 0.60 for the TUEV metrics, not at zero.

What the tables say

On TUEV, LaBraM-Base reaches 0.6409 balanced accuracy where BIOT, the strongest earlier model, reaches 0.5281. Kappa goes from 0.5273 to 0.6637 and weighted F1 from 0.7492 to 0.8312. The jump is large on the harder, six-class exam.

On TUAB the gain is smaller. AUROC moves from 0.8815 (BIOT) to 0.9022, and balanced accuracy from 0.7966 (ST-Transformer) to 0.8140. The authors suggest TUAB is a lower-level task, where normal and abnormal can often be told apart by eye.

Size helps a little. Going from Base (5.8M) to Huge (369M) adds 0.0207 balanced accuracy on TUEV and 0.0118 on TUAB. That is small beside the leap from earlier models to Base.

For the regression task in Appendix F the metrics change. Pearson’s correlation says how well predictions rise and fall with the truth. R2 says what share of the variation the model explains. RMSE is the typical error size, where lower is better.

More tests, and what data buys

Appendix F adds two tasks. On SEED-V emotion recognition (five classes), Base scores 0.4095 accuracy against BIOT’s 0.3802. On MoBI gait regression, Base is on par with the best baseline (R2 0.2876 versus 0.2911 for ST-Transformer), and Large and Huge pass it (0.3093 and 0.3145).

The paper also scales the pre-training data. Per Figure 5, the Base model at 500 hours already beats its own 2,500 hour version on TUAB and reaches over 90% of it on TUEV. Large grows more slowly after 1,000 hours. Huge keeps rising. From this and the scaling law of Kaplan and colleagues, the authors infer that Huge would need on the order of at least ten thousand hours.

Two more checks. Adding TUAB and TUEV themselves to pre-training changed little (Figure 4), and training from scratch without pre-training dropped steeply (Figure 8).

Read the table with care

Three cautions. The baselines come from the BIOT paper, and the authors chose its best numbers. The pre-training pool holds much Temple hospital data: TUAR, TUEP, TUSZ and TUSL add up to 1,842.56 of the 2,534.78 hours, or 72.7%. The paper says TUAB and TUEV recordings are disjoint from these, but the source is the same hospital family. And the whole network is fine-tuned, which costs compute and memory. The paper lists this as one of its three limits.

On TUEV, LaBraM-Base reaches 0.6409 balanced accuracy and BIOT reaches 0.5281. Which reading is most accurate?

Chapter 7

Remove One Part at a Time

Read the paper’s ablation tables to learn which design choices carry the gain, and which are within the noise

A cook thinks a stew needs seven ingredients. To test the idea, she makes the stew seven more times, each time without one ingredient, and tastes. Where the taste collapses, that ingredient mattered. Where nothing changes, it may be decoration.

An ablation does exactly this to a model: remove or swap one part, keep everything else, and compare scores. The LaBraM appendix runs five of them. Each one answers a question you may ask of any design.

Reading an ablation without fooling yourself

Every score has a spread. The paper repeats each run with five seeds and prints mean and standard deviation. On TUEV the kappa spread is often 0.01 to 0.03, so a gap of 0.01 is not a finding. On TUAB the AUROC spread is 0.001 to 0.004.

Use two numbers per variant: AUROC on TUAB, and Cohen’s kappa on TUEV. These are the monitor metrics the authors chose. Pick an experiment below, then read the rows.

Run the five ablations

Each row is a variant of LaBraM-Base. Blue bar: TUAB AUROC. Warm bar: TUEV kappa. Look for gaps bigger than the spread.

Values are Tables 7 to 11 of the LaBraM paper (Appendices G, H, I, K and L), mean of five seeds. The TUAB bars run from 0.86 to 0.93 and the TUEV bars from 0.30 to 0.70, so they do not start at zero. Large and Huge rows in the symmetric masking experiment are those models, not Base.

A rule of thumb, with the paper’s numbers

Compare a gap with the combined spread of the two runs, which is the square root of the sum of the squared spreads. Take TUEV kappa in Table 7. Codes score 0.6637 ± 0.0093. Rebuilding raw patches scores 0.5910 ± 0.0156.

The gap is 0.0727. The combined spread is the square root of 0.00932 + 0.01562, about 0.0182. The gap is 4.0 spreads wide. That is a real difference.

Now try “predict tokenizer output” (0.6376 ± 0.0168). The gap is 0.0261, the combined spread 0.0192, so the gap is only 1.4 spreads. That is weak evidence. And on TUAB the 0.0045 AUROC edge of raw patches (0.9067 against 0.9022) is about 1.8 spreads, so it is a hint, not a result. Five seeds is a small number.

What each experiment found

1. The target (Table 7). Four ways to teach the same Transformer. LaBraM predicts tokenizer codes. Setting 1 predicts the tokenizer’s output vectors by cosine similarity instead of codes. Setting 2 drops the tokenizer and rebuilds raw patches. Setting 3 drops it and rebuilds the spectrum with mean squared error.

On TUEV, kappa is 0.6637 for codes against 0.6376, 0.5910 and 0.5643. On TUAB, plain reconstruction does slightly better (0.9067 and 0.9053 AUROC against 0.9022). The authors explain it this way. TUAB is a low-level task, where bad EEG looks different. TUEV needs the semantic grouping that codes provide.

2. Symmetric masking (Table 9). Without it, Base even gains a little on TUAB (0.9065 against 0.9022). But on TUEV every size drops, and Huge drops most: kappa 0.6745 falls to 0.6391. The authors say the larger the model, the more it needs the extra data views.

3. Mask ratio (Table 8). On TUAB all ratios score close together, with r = 0.4 best (0.9049). On TUEV, r = 0.5 leads the others by about 0.05 kappa. The authors pick 0.5 because its spread is smaller on both exams.

4. How much to fine-tune (Table 10). Training the last 8 of 12 blocks gives the best TUEV kappa (0.6820), a gain about the size of the run-to-run spread, so treat it as level with full fine-tuning. A linear probe, which trains only a new output layer on frozen features, holds TUAB at 0.8835 but drops TUEV kappa to 0.3968. The pre-trained features need adapting.

5. Spatial embeddings (Table 11). Without them AUROC falls from 0.9022 to 0.8888 and kappa from 0.6637 to 0.6069. During pre-training, the loss could not converge at all without them.

One more result sits in the text. Training LaBraM from scratch, with no pre-training, falls steeply on both tasks (Figure 8). Pre-training matters more than any single tweak above.

What to take into your own design

Four habits follow from the tables. First, write down the spread before you run the comparison, so a lucky seed cannot fool you. Second, test on more than one exam, since the target ablation shows that two exams can disagree. Third, test a frozen version and a fine-tuned version, to learn how far the features travel alone.

Fourth, keep the cheap defaults unless a table beats them by several spreads. The mask ratio of 0.5 and the symmetric mask cost nothing to keep. A new tokenizer costs weeks. The tables tell you which piece deserves the weeks: here, the tokenizer codes and the spatial tags move the six-class score the most.

In Table 10 a linear probe keeps TUAB AUROC near 0.88 but TUEV kappa falls to 0.3968. What does that suggest?

Chapter 8

Let Rows and Columns Talk Separately: CBraMod

Follow the ICLR 2025 successor: criss-cross attention, a position code computed from neighbors, and a plain rebuild loss, with EEGPT as context

Look at a spreadsheet of sales: stores down the side, months across the top. A cell relates most to its own row (the same store, other months) and its own column (the same month, other stores). It relates little to a random cell far away.

EEG patches have the same grid: channels down, seconds across. LaBraM lets every patch attend to every other. A group at Zhejiang University argued that this wastes effort and blurs two different kinds of relation.

Two complaints about full attention

CBraMod (Wang, Zhao and colleagues, ICLR 2025) names two problems with models like LaBraM. First, full EEG modeling treats all patches alike. But space and time are different: patches in one channel or in one time slot may relate more closely than patches far apart in both.

Second, absolute position codes tied to electrode numbers assume a fixed link between a channel and a place. In real data a channel also depends on the reference, the electrode or average that voltages are measured against (earlobe, average, REST or bipolar). The same electrode name can mean different things across datasets.

The method, piece by piece

  1. Patch and mask. A sample S has C channels and T time points. Cut it into patches of t = 200 samples: X has shape (C, n, 200) with n = ⌊T / t⌋. Hide 50% at random, replacing hidden patches with a zero vector.
  2. Time-frequency patch encoder. Two branches add up. The time branch is three 1-D convolution blocks (kernel 49, stride 25: ⌊(200 + 48 − 49) / 25⌋ + 1 = 8 steps, times 25 channels = 200 numbers). The frequency branch takes the FFT energy of the patch (101 values) through one linear layer to 200. Output: (C, n, 200).
  3. Asymmetric conditional positional encoding (ACPE). A depthwise 2-D convolution slides over the (channel, time) grid of patch vectors with kernel (19, 7): wide across channels, short across time. Its output is added to the vectors. Position is computed from neighbors, not looked up by electrode ID.
  4. Criss-cross transformer. Twelve blocks. In each, 8 heads split in two. Four spatial heads attend only within one time slot, across channels. Four temporal heads attend only within one channel, across time. Outputs are concatenated.
  5. Reconstruct. A linear head predicts the raw patch. The loss is mean squared error on hidden patches only.

Notice the contrast with LaBraM. There is no tokenizer and no codebook. The target is the raw patch itself. The encoder gets frequency help directly at the input instead.

Fkj
The output of spatial head k on time slot j. Ẽj is the (C × d) table of normalized patch vectors in that slot. WQ, WK, WV are the head’s learned projections. Temporal heads do the same on one channel’s n slots.
headk
With K = 8 heads, the first half are spatial and the second half temporal. All K outputs are concatenated, as in a normal Transformer.

Now see the pattern. Choose how heads attend, and move the query patch. The counts at the bottom use CBraMod’s real pre-training shape: 19 channels and 30 seconds, which is 570 patches.

See who each patch listens to

Rows are channels and columns are seconds. The teal ring is the query patch. Purple tiles are the patches it attends to. Move the query with the sliders.

3
5

The grid is a toy of 6 channels by 8 seconds. The pair counts use the real shape from the CBraMod paper (19 channels, 30 s, 570 patches, 8 heads: 4 spatial and 4 temporal). Pair count means query-key pairs in one head of one layer. The paper’s measured FLOPs (Table 22) are the cost figure that counts.

The saving is large in pairs and smaller in total cost. One full head compares 570 × 570 = 324,900 pairs. A spatial head compares 30 slots of 19 × 19, which is 10,830. A temporal head compares 19 channels of 30 × 30, which is 17,100. Per layer: 8 × 324,900 = 2,599,200 pairs against 4 × 10,830 + 4 × 17,100 = 111,720. But the measured cost drops less, from 469.4M to 318.9M FLOPs (Table 22, measured on CHB-MIT with 16 channels and 10 seconds), about 32%. We expect the feed-forward layers, which attention changes do not touch, to explain the gap; the paper does not say.

What the paper reports

CBraMod pre-trains on TUEG, 27,062 hours in 69,652 recordings. After cutting recordings to 19 channels, 30 second samples, and dropping any sample over 100 µV, 1,109,545 samples remain, over 9,000 hours. That is about 3.5 times LaBraM’s 2,534.78 hours. The model has 4.0M parameters (LaBraM-Base has 5.8M) and trained for about 5 days on four RTX A5000 GPUs.

It is tested on up to 10 task types with 12 public datasets. Selected comparisons, all from the CBraMod paper’s own runs:

Task and metricCBraModBest earlier
FACED emotion, kappa0.50410.4698 (LaBraM-Base)
SEED-V emotion, kappa0.25690.2386 (LaBraM-Base)
PhysioNet-MI motor imagery, kappa0.52220.4912 (LaBraM-Base)
SHU-MI motor imagery, AUROC0.69880.6609 (BIOT)
TUEV events, balanced accuracy0.66710.6616 (LaBraM-Huge)
TUAB abnormal, AUROC0.92270.9162 (LaBraM-Huge)

The ablations tell a clear story. Full attention scores worst of the four attention types tried (Figure 4). A 4:4 split of spatial and temporal heads beats 2:6 and 6:2 (FACED kappa 0.5041 against 0.4921 and 0.4910). Time and frequency together beat either alone (0.5041 against 0.4956 and 0.4652). Cleaning the pre-training data matters: kappa 0.5041 with clean data, 0.4768 with dirty data, 0.4615 with none.

Two checks echo LaBraM. A frozen CBraMod fails as a fixed feature extractor (FACED kappa 0.2579 against 0.5041 fine-tuned). And with only 30% of the labels, CBraMod keeps 0.3239 on FACED against 0.2672 for LaBraM and 0.2573 for BIOT.

Where EEGPT sits

EEGPT (Wang and colleagues, NeurIPS 2024) is a third design. Its abstract describes about 10 million parameters, a mask-based dual self-supervised method that aligns spatial and temporal representations instead of raw signals, and a hierarchical design that handles space and time separately. It reports strong linear-probing results. We read only its abstract and the way CBraMod describes it, so we state no numbers for it here.

Read with care

CBraMod re-ran most baselines itself, using the BIOT code. Only LaBraM-Base weights are public, so on new tasks it compares against Base, and on TUEV and TUAB it compares against LaBraM’s published Huge numbers. Its own Appendix G tests the architecture on the same data and finds the architecture matters more than the larger data. Its stated limits: crude cleaning of TUEG loses data, it still needs more compute than small task models, and it has not been scaled toward billions of parameters.

In one CBraMod block, what do the spatial heads and the temporal heads each attend to?

Chapter 9

Field Guide: The Numbers, the Checklist, the Code

Keep the whole paper on one page: what to remember, what to check, and where to read next

LaBraM makes one model out of many EEG datasets. It cuts each recording into one-second channel patches, tags them with time and electrode, learns an 8,192 word vocabulary by rebuilding Fourier spectra, and pre-trains by naming the codes of hidden patches. After full fine-tuning it beats earlier models on TUAB, TUEV and SEED-V, and its Large and Huge sizes lead on MoBI. CBraMod then drops the tokenizer, splits attention into spatial and temporal heads, and reaches higher scores with fewer parameters.

The numbers to remember

QuantityValueWhere
LaBraM pre-training data2,534.78 hours, about 20 datasetsLaBraM, Appendix D
Patch and budget200 samples (1 s at 200 Hz), at most 256 patchesLaBraM, Section 3.2
Model sizes5.8M, 46M, 369M parametersLaBraM, Section 3.2
Layers, width, heads12, 200, 10 · 24, 400, 16 · 48, 800, 16LaBraM, Table 4
Codebook8,192 codes of width 64, L2 normalizedLaBraM, Table 3 and Eq. 4
Maskr = 0.5, plus the inverse maskLaBraM, Table 4, Eq. 12
TUEV, Basebalanced accuracy 0.6409, kappa 0.6637, weighted F1 0.8312LaBraM, Table 2
TUAB, Basebalanced accuracy 0.8140, AUROC 0.9022LaBraM, Table 1
CBraMod size and data4.0M parameters, 1,109,545 clean samples (over 9,000 hours)CBraMod, Section 3.1
CBraMod heads and cost4 spatial + 4 temporal; 318.9M FLOPs against 469.4M for full attentionCBraMod, Table 22

The terms, in plain words

TermPlain meaning
Channel patchOne electrode for one second: 200 numbers. The unit every model here reads.
Spatial / temporal embeddingLearned tags for the electrode name and the time slot, added to each patch.
Neural tokenizerA network that maps each patch to one of 8,192 codes, trained by rebuilding the spectrum.
Vector quantizationReplace a vector by the nearest entry of a fixed list of vectors.
Masked EEG modelingHide patches, then predict the tokenizer’s code for each hidden one.
Symmetric maskingRun the inverse mask on the same sample and add the two losses.
Criss-cross attentionHalf the heads attend across channels in one slot, half across time in one channel.
ACPEA position code made by a convolution over neighboring patches, not a lookup by electrode ID.
Linear probeTrain only a new output layer on frozen features.
Balanced accuracy, kappaAverage per-class recall; agreement beyond chance.

A checklist to re-implement LaBraM

The core in code

This sketch is ours, not the authors’ code. It shows the data flow with shapes. The authors’ implementation is at github.com/935963004/LaBraM.

python
import torch, torch.nn as nn, torch.nn.functional as F

W = 200                                   # samples per patch: 1 s at 200 Hz

def patchify(x):                          # x: (C, T), resampled to 200 Hz, scaled by 0.1 mV
    C, T = x.shape
    n = T // W
    return x[:, : n * W].reshape(C, n, W)   # (C, n, 200)

class PatchEncoder(nn.Module):               # Table 3, Base: channels 8, kernels 15/3/3, strides 8/1/1
    def __init__(self, co=8):
        super().__init__()
        blk = lambda i, o, k, s, p: nn.Sequential(nn.Conv1d(i, o, k, s, p), nn.GroupNorm(4, o), nn.GELU())  # 4 groups: our choice
        self.net = nn.Sequential(blk(1, co, 15, 8, 7), blk(co, co, 3, 1, 1), blk(co, co, 3, 1, 1))
    def forward(self, p):                 # p: (N, 200)
        return self.net(p[:, None, :]).flatten(1)   # (N, 8 * 25) = (N, 200)

def spectrum_target(p):                   # p: (N, 200) patches of ONE sample
    X = torch.fft.rfft(p, dim=-1)[:, 1:101]    # bins 1..100 Hz
    z = lambda v: (v - v.mean()) / v.std()   # z-score within the sample
    return z(X.abs()), z(torch.angle(X))   # amplitude, phase: each (N, 100)

def tokenize(p, codebook):                # p: (N, 64), codebook: (8192, 64)
    sim = F.normalize(p, dim=-1) @ F.normalize(codebook, dim=-1).T   # cosine, (N, 8192)
    return sim.argmax(-1)                  # (N,) ids in [0, 8191]

def masked_loss(model, head, e, tags, ids, mask_token, r=0.5):
    # e: (N, d) patch vectors; tags: (N, d) = time tag + electrode tag; ids: (N,) from the frozen tokenizer
    N = e.shape[0]
    hide = torch.zeros(N, dtype=torch.bool)
    hide[torch.randperm(N)[: int(r * N)]] = True
    loss = 0.0
    for m in (hide, ~hide):                 # symmetric masking: the mask, then its inverse
        x = torch.where(m[:, None], mask_token.expand_as(e), e) + tags
        logits = head(model(x))               # (N, 8192)
        loss = loss + F.cross_entropy(logits[m], ids[m])   # hidden patches only; mean, not the paper's sum
    return loss

Limits, and where the field stands

The LaBraM authors list three limits. It is still far below the scale of vision and language models. It needs full fine-tuning, which is costly in compute and memory. And it uses only EEG, with no other signal. They point to adapters, prompt tuning and LoRA, and to pairing EEG with images, speech or text, as next steps.

CBraMod adds its own: crude data cleaning, higher cost than small task models, and no scaling study past 9,000 hours and 4M parameters. Both papers test on offline datasets. Neither shows a closed-loop device. Treat the scores as a step in representation learning, not as a ready product.

Keep going

Credit. LaBraM: Wei-Bang Jiang, Li-Ming Zhao and Bao-Liang Lu (Shanghai Jiao Tong University), ICLR 2024, arXiv 2405.18765. CBraMod: Jiquan Wang, Sha Zhao, Zhiling Luo, Yangxuan Zhou, Haiteng Jiang, Shijian Li, Tao Li and Gang Pan (Zhejiang University and partners), ICLR 2025, arXiv 2412.07236. EEGPT: Wang et al., NeurIPS 2024, abstract page. All figures here come from these papers, except toy numbers marked as illustrative.

The takeaway. One recipe makes EEG models portable across caps: cut to one-second channel patches, tag with place and time, predict something stable (a spectrum code, or the raw patch with the right attention), and fine-tune. Each design choice here earned its place in a table you can now read.

Now press Present or Teach and explain the pipeline from memory: how a 62 channel file becomes 248 patches, why the tokenizer rebuilds spectra, how the inverse mask reuses the codes, and why CBraMod splits its heads. Then go back to the cap picker at the top, choose 23 channels and 64 channels, and say why both fit one model.

EEG Foundation Models: LaBraM and CBraMod
LaBraM paper · CBraMod paper · The lecture this paper belongs to · Back to microLearning