EEG Preprocessing

A raw EEG is mostly not brain. Blinks, jaw muscle, mains hum and a loose electrode bury the signal. Switch each fix on and watch the leftover junk fall.

Learn how a recording is cleaned before any decoder sees it: filters for hum and drift, a check for dead electrodes, a choice of reference, and ICA to lift out blinks.

Press the five fixes one at a time and watch the four traces and the percentage. Then we build, piece by piece, why each fix works, what it costs, and the order that keeps them from hurting each other.

You need a feel for a wave (a signal that goes up and down some number of times each second) and a little Python for the code. We build the rest from zero.

Built from primary sources: the MNE-Python ICA tutorial and its filtering, reference and bad channel pages, Muthukumaraswamy (2013) on muscle, and Jung et al. (2000) on ICA. Lecture 11 of the BCI series.

Clean a recording

Illustrative: a made-up two second recording sampled at 400 Hz, with microvolt sizes chosen to be easy to see. Each fix is the ideal version of the real one, except that the low-pass also removes a real 22 Hz rhythm and Fix T8 averages its two neighbours.

Chapter 0

Why Raw EEG Is Mostly Not Brain

See how small the brain’s voice is on the scalp, and meet the six kinds of junk that drown it

Picture a person in a lab chair. Dozens of small metal disks sit on the scalp, each wired to an amplifier. The person is asked to imagine moving a hand. A program must read that intention from the wires.

Now watch the first ten seconds of what the wires record. A big slow hill rises on the forehead: the person blinked. A fuzzy band thickens over the ears: the person clenched the jaw. A thin, constant buzz sits on every wire. One wire shows nothing at all. The hand movement, the thing you came for, is hidden somewhere under all of this.

What an electrode measures

An electrode is a small conductor that touches the skin. EEG (electroencephalography) records the tiny voltages that thousands of neurons make when they act together. Those voltages have to cross the brain, the skull and the scalp before they reach the disk. On the way they get weaker and blurrier.

The disk does not measure a voltage alone. A voltage is always a difference between two points, so each channel is the difference between its electrode and a shared reference electrode. A channel is one such recorded wire, a list of numbers over time. Chapter 3 returns to this.

The brain’s signal is small. Anything else that makes a voltage near the head can swamp it. Muthukumaraswamy (2013) gives a clear pair of numbers: muscle activity over the temporal electrodes, the ones near the ears, can reach about 100 microvolts, while fast activity from the brain is less than 1 microvolt at the scalp. A microvolt (µV) is one millionth of a volt.

Muscle against brain

The blue bar is fast brain activity at the scalp. The pink bar is muscle. The scale is logarithmic: each gridline is ten times the one before. Drag the clench from a relaxed jaw to a hard one.

30 µV

The 1 µV brain level (the review gives it as an upper bound) and the 100 µV muscle ceiling are from Muthukumaraswamy (2013, Frontiers in Human Neuroscience 7:138). The positions in between are for you to try.

At least a hundred to one is a hard ratio. The bar for muscle runs off the left edge of a plain scale while the brain bar is a sliver. That is why cleaning is not polish. It decides whether a decoder sees the brain at all.

Six kinds of junk

An artifact is any part of the recording that did not come from the brain activity you want. This lesson covers six. Each has its own cause, its own look and its own fix.

Line noise is a steady hum at the frequency of the wall power, 50 or 60 cycles each second. Drift is a slow wander of the baseline, over many seconds, from sweat and cable movement. A bad channel is an electrode that has lost contact or has gone noisy. A bad reference spreads its own noise into every channel. An eye blink makes a large slow bump on the forehead. Muscle makes broad, fast, fuzzy noise.

Here is the rule for the whole lesson. Every fix removes something, and every fix can remove something real. We will always ask two questions. What does this fix take away? What does it cost the brain signal?

Why a BCI cares more than a lab does

A lab can throw away a bad ten seconds and run the experiment again. A brain-computer interface (BCI, a system that turns brain signals into commands) cannot. It runs live, on a person who blinks, talks and shifts in the chair. Its cleaning must be fast, must never look at the future, and must not fail silently. We come back to this in Chapter 8.

Also, a decoder trained on dirty data learns the dirt. A model that finds blinks easy to detect will happily call a blink “an intention” if blinks and intentions happen together. Cleaning is how you make sure the model learns the brain.

The road

The syllabus line for this session is: eye blinks, muscle, line noise, bad channels, re-referencing, ICA artifact removal. We take them in the order a pipeline meets them, with the shortest fixes first.

  1. Hum and drift: a notch filter and a high-pass filter.
  2. Bad channels: detect by variance, repair by interpolation.
  3. Re-referencing: what you subtract from every channel, and the average reference.
  4. Eye blinks: how a blink spreads and why regression harms brain.
  5. Muscle: where it sits in frequency and why filters fail on it.
  6. The ICA idea: unmixing sources by rotation.
  7. ICA in practice: the MNE recipe, step by step.
  8. Order of steps: what must come first, and what a live decoder cannot use.
  9. Field guide: numbers, checklist and code.
Why do the first chapters treat cleaning as more than polish?

Chapter 1

Hum and Drift: Filters That Cut One Frequency

Remove the mains hum and the slow wander with two filters, and see what each one costs the real signal

You plug in the amplifier and look at the live trace. It is thick, as if drawn with a fat pen. You unplug the lamp on the desk and the trace gets thinner. Plug the lamp back in and it thickens again. Something in the wall is talking into your wires.

That something is line noise: a pure up-and-down wiggle at the frequency of the mains power. In most of the Americas that is 60 cycles each second. In much of Europe and Asia it is 50. The wiggle is far from random. It is a clean sine wave, and that makes it the easiest junk to remove.

Frequency, and the picture that shows it

A frequency is how many times a wave repeats each second, counted in hertz (Hz). A slow ocean swell is a low frequency. A mosquito whine is a high one. Any recording is a mix of waves at many frequencies.

A spectrum (or power spectrum) draws that mix. The horizontal axis is frequency. The vertical axis is how much power the recording has at that frequency. The method that computes it is the Fourier transform, taught in the DFT lesson. Here you only need to read the picture.

Brain activity makes a smooth, falling spectrum with small bumps. Most power sits at low frequencies. A bump near 10 Hz is the alpha rhythm, a wave that grows when a person relaxes with closed eyes. The hum shows up as a tall, thin spike at exactly 50 or 60 Hz, often with smaller copies at 100 or 120 Hz, called harmonics.

The notch: cut one thin slice

A filter multiplies each frequency in the spectrum by a number called its gain. Gain 1 keeps the frequency. Gain 0 deletes it. A notch filter has gain 0 at one frequency and gain near 1 everywhere else. Its shape is a narrow trench.

The width of the trench is a choice, and it has a cost. A thin trench removes the spike but misses the hum if the mains frequency wobbles a little. A fat trench catches the wobble but also deletes real brain power close to the hum. Gamma activity, the fast rhythms linked to attention, lives in the same neighbourhood.

The MNE-Python filtering tutorial shows one such filter. It uses 60 Hz and also its harmonics at 120, 180 and 240 Hz, with a transition width of 0.50 Hz on each side. That filter is 3,965 samples long, which is 6.6 seconds at the tutorial’s sampling rate. The tutorial also notes that its spectrum-fitting method does not find the line frequency reliably by itself, so you must tell it which frequencies to remove.

The high-pass: cut the slow wander

Now the opposite end of the spectrum. Sweat changes how well the skin conducts. A cable swings. The baseline of each channel then wanders slowly, over ten or twenty seconds. In the tutorial’s example, half a period of the drift lasts about 10 seconds, so one full period is about 20 seconds, which is 0.05 Hz.

A high-pass filter keeps everything above a cutoff frequency and fades everything below it. The tutorial tried cutoffs of 0.1 and 0.2 Hz. Of 0.1 Hz it says that it “was not quite high enough to fully remove the slow drifts.” Chapter 7 will show that ICA wants a higher cutoff still, 1 Hz.

The cost is on the low end. A high-pass also thins out real slow brain activity, such as the delta rhythm between about 1 and 4 Hz, and it distorts slow evoked responses. The higher the cutoff, the larger the bite.

Carve the spectrum

Warm is the recording. Green is the result after your filters. Blue is the brain alone. Choose the mains frequency, widen the notch, and raise the high-pass cutoff. Watch the spike, the drift and the bumps of real brain.

1.0 Hz
0.5 Hz

Illustrative spectra built from a falling brain curve, an alpha bump at 10 Hz, a hum spike and a drift rise. Filter shapes are idealised: a Gaussian trench for the notch at the mains frequency and its double, and a second-order Butterworth-style rise for the high-pass.

The worked numbers

Take the device at its start: 60 Hz, a notch width of 1 Hz, a high-pass at 0.5 Hz. Follow three steps. First, the spike at 60 Hz sits far above the brain curve, so the notch gain of nearly 0 there cuts it by many decibels. A decibel (dB) is ten times the base-10 logarithm of a power ratio, so a cut of 30 dB means the power drops to a thousandth.

Second, the drift sits below 0.5 Hz. The high-pass at 0.5 Hz has gain one half in amplitude at its cutoff and falls fast below it, so most drift power vanishes. Third, look at the cost: raise the cutoff to 2 Hz and the 1 to 4 Hz band loses a large share of its power. You paid brain for a cleaner baseline.

The filter you cannot use live

MNE’s default filters are FIR (finite impulse response, a weighted average of nearby samples), zero-phase and non-causal. Zero-phase means the filter does not shift the waveform in time. The price is that each output sample uses samples from the future.

That works on a saved file. It cannot work on a live stream, where the future has not happened yet. A real-time BCI must use a causal filter, which only looks backward. A causal filter delays the signal, and a sharper one delays it more. In the tutorial a filter with a 0.20 Hz transition band is 9,911 samples long, 16.5 seconds at 600 Hz. Chapter 8 returns to this trade.

Symptom, measurement, fix

The symptom is a thick trace and a decoder that works in one room and fails in another. The measurement is the spectrum: a tall thin spike at the mains frequency, and copies at its multiples. The fix is a notch at each of them, as narrow as the line allows. Check the result by plotting the spectrum again. The spike should be gone and the alpha bump should not have changed.

The spectrum shows a tall thin spike at 60 Hz and a smaller one at 120 Hz. What is the best first fix, and what does a wider trench cost?

Chapter 2

Bad Channels: Find the Dead Wire and Rebuild It

Flag the electrodes that lie, then replace each one with a guess made from its neighbours

The lab assistant sets the cap on a volunteer’s head and presses the electrodes into place. Sixty-four lights turn green. Twenty minutes later one channel is a perfectly flat line, and another has grown a huge, jagged scribble. Nobody touched them. The gel dried under one disk and a wire came loose from the other.

A bad channel is an electrode whose recording no longer tells the truth about the brain beneath it. It is not noisy in a clever way. It lies in one of two simple ways: it records almost nothing, or it records mostly junk. Both are easy to spot if you know what number to look at.

The number: variance

Variance measures how much a channel wiggles. Subtract the channel’s average, square each sample, then average the squares. A flat line has a variance near zero. A scribble has a variance far above the rest. Healthy channels, covering similar patches of scalp, have similar variances.

The MNE-Python bad channel tutorial describes both cases. One EEG channel in its sample data “is not picking up scalp potentials at all.” It also warns that a flat channel “will have zero channel variance,” which makes noise estimates unrealistically low and gives that channel too much weight in later calculations. Very noisy channels cause the opposite trouble: they trigger excessive rejection of good data, and the tutorial says ICA is corrupted by them.

A fair threshold: median and MAD

How far from normal is too far? The usual tool is a z-score: how many standard deviations a value sits from the average. But the average and the standard deviation are themselves pulled by the bad channels, as the worked example shows.

The fix is to use robust partners. The median is the middle value when you sort the list, so one wild value cannot move it. The MAD (median absolute deviation) is the median of how far each value sits from the median. Multiply it by 1.4826 and it matches the standard deviation for bell-shaped data. A robust z-score is then (value minus median) divided by 1.4826 times MAD. We apply it to the logarithm of variance, because variances span many orders of ten.

Worked example with five channels. Their variances are 1.1, 0.9, 1.0, 0.02 and 9.0. Take base-10 logarithms: 0.04, minus 0.05, 0.00, minus 1.70, 0.95. Sorted, they are minus 1.70, minus 0.05, 0.00, 0.04, 0.95, so the median is 0.00. The distances from it are 1.70, 0.05, 0.00, 0.04, 0.95. Sorted, the middle one is 0.05, so MAD is 0.05 and the robust spread is 1.4826 × 0.05 = 0.074. The flat channel has robust z of minus 23 and the noisy one of plus 13. Every healthy channel has |z| below 1.

Now use the plain average instead. The mean of the five logs is minus 0.15 and the standard deviation is 0.86. The noisy channel scores only 1.3 and the flat one minus 1.8. Neither crosses a threshold of 3. The bad channels hid themselves by inflating the yardstick.

Flag and rebuild

Each bar is the log variance of one of sixteen channels. Three channels are truly bad: one flat, two noisy. One healthy channel has strong alpha rhythm and is only a decoy. Move the threshold, then press Interpolate flagged.

3.5

Toy data: sixteen channels on a 4 by 4 grid, made from three smooth hidden sources plus own noise, seeded so it is the same on every visit. The rebuild here averages the three nearest good channels. MNE uses a spherical spline for EEG, which does the same job on a sphere.

The repair: interpolation

Once a channel is flagged, you have two choices. You can drop it and work with fewer channels. Or you can interpolate it: build a stand-in from its good neighbours. Brain activity changes smoothly over the scalp, so a spot’s value is close to the values around it.

MNE does this for EEG with the spherical spline method, which places the sensors on a unit sphere and blends the nearby good ones. By default, interpolate_bads() then clears the bad list so later steps treat the channel as normal.

Be honest about what you did. The new channel carries no new information. It is a weighted sum of channels you already have. Counting channels now overstates what you recorded, and the data matrix has one fewer independent direction than it has rows. That is called a rank deficit, and it matters for ICA and for the average reference, as the MNE reference page notes when it cites work on rank problems from interpolation.

Symptom, measurement, fix

The symptom is a decoder that is oddly bad on one session, or one channel that dominates every plot. The measurement is the log variance against the median of its peers, with a robust threshold. The fix is to mark the channel bad first (MNE stores the list in raw.info['bads']), keep it out of later steps, and interpolate it late. In a live BCI the same test runs on a sliding window, so a wire that fails mid-session is caught within seconds.

Why use the median and MAD instead of the mean and standard deviation to flag bad channels?

Chapter 3

Re-referencing: Every Voltage Is a Difference

Choose what to subtract from every channel, and see how one noisy reference leaks into all of them

You are on a hike and your watch says “2,000.” Two thousand what? Metres above the sea, or metres above the car park where you started? A height means nothing until you say what it is measured from. If the car park were on a sliding platform, every height you read would slide with it.

EEG has exactly this problem. A voltage is the difference in electrical level between two points. Each channel records the difference between its own electrode and one shared reference electrode. The reference is the car park. Every channel is a height above it.

One bad reference spoils every channel

Write the potential at electrode i as ui(t), and the reference as electrode R. The channel you save is yi = ui − uR. Notice that uR is inside every channel. If the reference electrode moves, or picks up muscle, or loses contact, that same junk is subtracted from all channels. It appears in every one of them, upside down.

This is why people say a bad reference “leaks.” You cannot find the junk by looking at one channel. You see the same bump, at the same moment, in all of them. That sameness is the clue.

Changing the reference after the fact

You may re-reference the saved data, because the reference is only a subtraction. If you want a new reference that is a weighted mix of electrodes with weights wj that add to 1, compute y′i = yi − Σj wj yj. The old reference cancels out. Nothing is lost, as long as the original reference electrode is counted too. MNE warns that if the original reference is missing, the average uses n channels instead of n+1, which is wrong.

The popular choices are these. A single electrode, such as Cz. Linked mastoids: the average of two electrodes on the bony bumps behind the ears. And the average reference, where each channel gets the mean of all channels subtracted. MNE sets it with set_eeg_reference(ref_channels="average"). It can also store the average as a projector, so it is recomputed when you mark channels bad and you can switch it on and off when plotting.

y′
The re-referenced data: n channels, each a list of numbers over time.
H
An n by n matrix that subtracts the mean of all channels from each channel at every time point.
y
The data as recorded, with shape (n channels, T samples).
1 1T
An n by n matrix of ones. Dividing by n and subtracting from the identity I gives H.

y = [3, −1, 5, −5] → mean = 0.5 → y′ = [2.5, −1.5, 4.5, −5.5] (sum = 0, illustrative numbers)

Look at the worked line. Four channels at one moment have a mean of 0.5. Subtract it from each. The result adds to exactly zero, at every moment, for ever. One consequence is the rank deficit from Chapter 2: the n new channels are not independent, because the last equals minus the sum of the others. Methods that need independent channels, such as ICA, must allow for that.

Choose what to subtract

Electrode E1 has a loose contact and makes a brief pop. Choose the reference and watch which of the six traces carry a copy of the pop. The pop is pink; brain is the rest.

Toy potentials: six electrodes with a 10 Hz rhythm of different size, and one 40 µV pop on E1 only. The traces are recomputed live as y′ = y − (weighted mean of the reference electrodes). Sizes are illustrative.

What the device shows

With E1 as the reference, E1 itself becomes a flat zero line. The pop shows in the other five channels, upside down. With the clean E3 as reference, the pop appears only on E1, where it belongs, and you can see which electrode is guilty. With linked mastoids it is the same. With the average, the pop is shared out: E1 shows five sixths of it, and each of the other five shows one sixth, upside down.

That last case is a warning. The average reference smears one bad electrode over every channel, just a little in each. This is why MNE leaves bad channels out of the average and why Chapter 8 puts “mark bad channels” before “set the reference.” The average also assumes that the electrodes cover the whole head evenly. With a handful of channels that is a weak assumption, so treat the average reference as less trustworthy on small headsets. That last point is our reasoning, not a measured result.

Why a decoder cares

A decoder learns patterns across channels. Change the reference and every pattern changes shape, though the brain has not moved. Always train and run with the same reference, and write it in the settings file next to the model. In MNE’s sample data there are 18 good EEG channels and one bad (EEG 053), and the average projector covers only the 18.

Symptom, measurement, fix

The symptom is the same bump, at the same instant, in many channels. The measurement is the correlation between channels at the moment of the bump. The fix is to find the electrode that carries the pop alone under a clean reference, mark it bad, and then compute the new reference without it.

The same bump appears upside down in every channel at the same instant. What is the most likely cause?

Chapter 4

Eye Blinks: A Battery on the Front of Your Head

Trace how a blink spreads across the scalp, and see why subtracting an eye channel also subtracts brain

Ask a volunteer to stare at a dot and not blink for as long as they can. Their forehead channels stay calm. Now ask them to blink once. A tall smooth hill rises on the front channels, many times larger than anything the brain was doing a moment ago. Channels near the back barely twitch.

The hill is not from the brain. Textbooks explain it with the eye itself. The front of each eyeball is electrically positive compared with the back, a little like a battery. When the eye moves or the lid slides over it, that battery changes the voltage in the skin around it, and the change spreads over the forehead.

How a blink spreads

The spread follows distance. The voltage from the eyes is strong at the electrodes nearest to them (the ones labelled Fp for frontal pole) and fades toward the back of the head. We can write it as a weight for each channel: how much of the eye’s voltage reaches this electrode. Our toy uses 1.0 at Fp1, 0.3 at Cz and 0.05 at O1. These weights are illustrative. Real ones depend on the head.

The weights are fixed for a given head and cap. That fact is the key to all three fixes. Whatever a blink does, its shape on the scalp stays the same. Only its size changes from blink to blink.

A blink is also slow. The MNE ICA tutorial, to find blinks, filters its EOG signal to between 1 and 10 Hz. That range overlaps the slow brain rhythms, so a filter cannot cut blinks without cutting some of the brain. EOG stands for electro-oculogram, a recording of the eye’s own voltage, usually from electrodes placed around the eyes.

Fix 1: throw away the blink

The simplest fix is to mark every stretch that holds a blink and drop it. It is safe: nothing is guessed. It is wasteful. Suppose, as an illustration, a person blinks every 4 seconds and each blink spoils a 1 second window. Then a quarter of the recording is gone. For a live BCI, dropping data is worse: the decoder must still answer during the blink.

Fix 2: subtract a scaled copy of the EOG

Regression removes the blink by subtraction. Record the eye channel too. For each EEG channel, find the number that says how much of the eye signal it contains. Then subtract that much of the eye signal from the channel. This was the standard method for years.

It has a hidden flaw. The EOG electrode sits on the forehead, so it also picks up real brain signal. Subtract the EOG and you subtract that brain signal too, and you add it into every other channel as well. Jung and colleagues (2000) make this point: regression against EOG channels also removes some genuine EEG, and PCA cannot cleanly separate eye artifacts from brain signals when their amplitudes are similar. They proposed ICA instead.

Follow the arithmetic in our toy. Let the EOG read e = blink + 0.25 × (frontal brain). At the blink peak, Fp1 holds brain 5 plus blink 90, so 95. The EOG reads 90 + 0.25 × 5 = 91.25. Subtract it all and Fp1 becomes 95 − 91.25 = 3.75. The true brain value is 5. The blink is gone, but 1.25 of the brain went too.

Check a moment without a blink. Fp1 holds brain 8. The EOG reads 0.25 × 8 = 2. Fp1 becomes 8 − 2 = 6. A quarter of the brain signal was removed, at every moment, not only during blinks. That is the cost of regression.

Subtract the eye channel

Blue is the true brain. Warm is the channel after you subtract a share of the EOG. Slide the strength to 100%, then raise how much brain leaks into the EOG electrode.

50%
25%

Illustrative: blink weights 1.0, 0.3 and 0.05 for Fp1, Cz and O1; a 90 µV peak at Fp1. The EOG reads the blink plus the chosen share of Fp1’s brain signal. Strength 100% means the ideal regression coefficient.

Fix 3: find the blink as a source

The third fix does not subtract a measurement. It asks: is there one pattern, with fixed weights across channels, that grows and shrinks over time like a blink? If so, take that pattern out as a whole. That is ICA, and the next two chapters build it. It needs no clean eye channel, though an EOG channel can help point to the right component.

Symptom, measurement, fix

The symptom is a large, slow hill on the front channels, with smaller copies fading toward the back, all rising and falling together. The measurement is the correlation of each channel with an EOG channel or with a frontal proxy such as Fp1 alone (MNE accepts a channel such as Fpz as a stand-in). The fix, if you have a clean eye channel and little brain in it, is regression. Otherwise use ICA.

Why does subtracting a scaled EOG signal remove some real brain activity?

Chapter 5

Muscle: Noise That Looks Like Fast Brain

Find where jaw and neck muscle sit in frequency, and see why a low-pass filter cannot separate them from brain

A volunteer is asked to imagine closing their right hand, and to keep the rest of the body still. They do their best. But thinking hard makes people frown, press their tongue to the roof of the mouth and tighten their jaw. The EEG fills with a fuzzy burst at the moment of effort. A decoder sees the burst, which lines up perfectly with the task, and learns it with joy.

That decoder has learned a clenched jaw. It will fail the day the volunteer relaxes, and it would never work for someone who cannot move their jaw. This is the most dangerous artifact for a BCI, because it tracks the task. Blinks and hum are random. Muscle is correlated with intent.

What muscle looks like

Muscles make their own electrical signal when they contract, called EMG (electromyography). The scalp is wrapped in muscle: the temples, the forehead, the jaw, the neck. Their signal is broad. It is not one tone but a wide smear across frequency.

Muthukumaraswamy (2013) reviews this in detail. Scalp muscle spreads across roughly 20 to 300 Hz, with most power at the low end. The peak depends on the muscle: about 50 to 60 Hz for the masseter (the main jaw muscle), 30 to 40 Hz for the frontalis (the forehead), and about 100 Hz for the muscles at the back of the neck. Over the temporal electrodes, muscle can reach about 100 µV, against less than 1 µV for fast brain activity.

The review reports a striking finding too. In studies of people paralysed by a drug, the review reports that the junk starts near 20 Hz: at 40 Hz the unparalysed state had about 5 times more power, and at 80 Hz about 10 times more. A large part of what we call fast brain activity in ordinary recordings is muscle. This is also why the habit of a low-pass near 30 Hz exists: it was partly a way to hide muscle.

Why a filter cannot fix it

A low-pass filter keeps frequencies below a cutoff and fades those above. It would be perfect if muscle and brain lived in different places. They do not. The fast brain rhythms we care about, the beta band near 13 to 30 Hz and gamma above it, sit in the same range as the muscle. So a cutoff has two bad outcomes. Set it low and you cut the real brain you wanted. Set it high and the muscle passes through.

Take the worked example. A fourth-order low-pass at 30 Hz has power gain 1 / (1 + (f / 30)8). At f = 55 Hz, the ratio 55/30 is 1.833. Raise it to the eighth power by squaring three times: 1.833 squared is 3.36, then 11.3, then 127.7. The gain is 1/128.7 = 0.0078, a cut of about 21 dB. Good. At 22 Hz, the ratio is 0.733 and its eighth power is 0.084, so the gain is 1/1.084 = 0.92. The beta rhythm keeps 92% of its power. So far the filter looks fine.

Now look at the part the filter misses. Muscle is also strong at 20 to 30 Hz, where the gain is near one. That part passes. And all brain activity above 40 Hz is gone with the muscle. A 30 Hz cutoff does not separate them. It decides to give up the whole upper range.

Where muscle hides

Blue is the brain alone. Pink is the muscle. The green line is what a low-pass leaves of both. Choose a muscle, set how hard it works, and slide the cutoff.

30 µV
30 Hz

Illustrative spectra: muscle is a broad hill whose centre is the jaw 55 Hz, forehead 35 Hz or neck 100 Hz peak from Muthukumaraswamy (2013); its power relative to brain is set so that a size of 1 µV equals the brain level. The low-pass is a fourth-order Butterworth shape.

What you can do

Four options, each with a cost. Prevent it: ask the person to relax the face, and give breaks. It costs nothing but needs a cooperative person. Low-pass: fine if you only need slow rhythms, as above. Reject: measure the power above 20 Hz in each short window, and drop windows that exceed a threshold set from clean data. ICA: lift muscle out as components. This works less cleanly than for blinks, because muscle comes from many separate fibres and not one tidy source.

Whichever you choose, check the decoder. Train it on clean-looking data, then test it with the jaw deliberately clenched and with all channels above 20 Hz removed. If the score collapses, it had been decoding the jaw.

Symptom, measurement, fix

The symptom is a fuzzy band in the traces, strongest at the temples, jaw side and neck, in bursts that follow effort. The measurement is band power from 20 Hz upward per window, per channel. The fix is a mix of prevention, rejection and, when the band you need is low, a low-pass. Be wary of any result whose accuracy rises with a faster rhythm only.

Why is muscle the most dangerous artifact for a BCI decoder?

Chapter 6

ICA: Unmix the Voices in the Room

Learn the idea that separates blinks from brain without a filter: rotate until one source stands alone

Two people talk at once in a room. You put two microphones in different corners. Each microphone records a blend of both voices, with a different mix of each. Can you get the two voices back from the two blends, knowing nothing about either person?

It sounds impossible, yet it works, and it is called the cocktail party problem. EEG is the same trouble. The electrodes are the microphones. The brain rhythms, the blinks, the muscle and the hum are the voices. Each electrode records a blend. ICA is the method that un-blends them.

The model: a blend with fixed weights

Say there are k hidden sources, s1 to sk: each is one process, like the blink. At each moment, source j has a value. Each electrode records a weighted sum of them. In matrix form this is x = A s.

x
The recording. Shape (n channels, T samples).
x
The same recording, now the input to the unmixing step.
A
The mixing matrix. Shape (n channels, k sources). Column j is the scalp pattern of source j: how strongly it reaches each electrode.
s
The source values over time. Shape (k sources, T samples).
s
The sources, recovered.
W
The unmixing matrix, the inverse of A when A is square. ICA must find it from x alone.

The fixed weights are the point. In Chapter 4 a blink had the same scalp pattern every time. That is one column of A. The same holds for the alpha rhythm and for the hum. Only the values in s change.

A = rotation by 35° → s = (4, 0.5) gives x = (0.819 × 4 − 0.574 × 0.5, 0.574 × 4 + 0.819 × 0.5) = (2.99, 2.71) (illustrative)

Unmix with W, the rotation back by 35°. The first output is 0.819 × 2.99 + 0.574 × 2.71 = 4.00. The second is minus 0.574 × 2.99 + 0.819 × 2.71 = 0.50. We recover (4, 0.5) exactly.

What makes unmixing possible

We do not know A. We see only x. Why can it be found? Two ideas.

The first idea: independent sources. A blink does not tell the alpha rhythm what to do. Two sources are independent when knowing one tells you nothing about the other. This is much stronger than being uncorrelated. PCA finds uncorrelated directions, and with two equally strong sources any rotation is as good as any other, so PCA cannot pick the true one. (See the PCA lesson.)

The second idea: non-Gaussian sources. A Gaussian is the bell-shaped distribution. A blink is mostly quiet with rare tall spikes: far from bell-shaped. The central limit theorem says that the sum of independent things looks more bell-shaped than the things themselves. So a blend of a spiky source and a flat one is rounder than either. To unmix, rotate until the output is as un-round as it can be.

A number measures roundness: excess kurtosis, the average of u4 divided by the square of the average of u2, minus 3, for a zero-mean signal u. A bell curve scores 0. A spiky source (a Laplace curve) scores 3. A flat-topped wave such as a sine scores minus 1.5. Rotate the output direction, keep the one with the largest absolute kurtosis, and you have found a source. This is the idea behind FastICA. Infomax, from the same family, reaches the same goal by making the outputs carry as much information as possible. Jung and colleagues (2000) applied this family of methods to EEG artifacts.

Rotate until one source stands alone

Each dot is one moment: channel 1 across, channel 2 up. The warm line is the direction you project onto. Slide the angle and watch the curve of kurtosis on the right and the output trace below. Find the spikiest output.

10°

Toy data: 400 moments. The sources are one spiky (Laplace) source and one flat (sine) source with equal variance, mixed by a rotation of 35°. Real mixing matrices are not rotations; this one is so that two channels are enough to see the idea.

What ICA cannot tell you

Three things stay unknown. The order of the components is arbitrary: the blink may be component 0 on Monday and component 7 on Tuesday. The sign is arbitrary: a source and its mirror image give the same blend. And the scale is arbitrary: doubling a source while halving its column of A changes nothing. So a person (or a rule) must look at each component and decide what it is. Chapter 7 shows how.

ICA also needs enough channels: you cannot find more sources than electrodes. It assumes the sources do not move, so one fixed A. That fits a blink well, since the eye stays put in the head. It fits muscle less well, which is why Chapter 5 said ICA is less clean there. And truly Gaussian sources cannot be separated at all.

Symptom, measurement, fix

The symptom is an artifact that overlaps the brain in time, frequency and space, so no filter and no channel can isolate it. The measurement is the scalp pattern: a component whose column of A is strong in front and fades toward the back, with a spiky time course. The fix is to unmix, drop that component and mix back. For a deeper treatment of the method, see the ICA lesson.

Why does ICA look for the direction whose output is least bell-shaped?

Chapter 7

ICA in Practice: Fit, Pick, Exclude, Rebuild

Run the four steps of the MNE recipe on a small recording and see what leaves with each component

You have a 20 minute recording from 64 channels. Blinks mark it every few seconds. You do not want to drop the blinks, and you do not own a good eye channel. You run ICA and get 64 components, each a scalp pattern and a time course. Which of them is the blink? And once you know, how do you get the cleaned recording back?

MNE-Python’s tutorial “Repairing artifacts with ICA” answers with four steps. We take each in order and follow the shapes of the data.

Step 1: filter a copy, then fit

The tutorial says: “A high-pass filter with 1 Hz cutoff frequency is recommended.” Slow drifts make the sources less independent, and that makes it harder for the algorithm to find a good answer. So it fits ICA on a copy of the data filtered with l_freq=1.0, and then applies the result to the original, unfiltered recording.

Why can you do that? The unmixing matrix W works across channels, at each time point separately. A filter works across time, in each channel separately. The two act on different axes, so a W learned on filtered data is just as valid on unfiltered data of the same channels.

Fitting needs choices. The tutorial starts with n_components=15, which it calls “a very small number,” to keep the run fast. A number between 0 and 1 means “enough components to explain that share of the variance.” MNE offers four algorithms: fastica (the default), picard, infomax and jamica. Picard is expected to converge faster than FastICA and Infomax, and to be more robust when the sources are not fully independent.

Data flow: the filtered copy has shape (n channels, T). MNE first reduces it with PCA to 15 directions, then ICA finds W with shape (15, 15) on those. In the tutorial, the 15 components explain 78.5% of the EEG variance and 93.4% of the magnetometer variance. The rest is the PCA residual, which we keep for rebuilding.

Step 2: pick the bad component

The components come out in arbitrary order. You identify an artifact by its look. A blink component has a scalp pattern strong at the front and fading backward, and a time course of rare tall spikes. A hum component has a flat pattern and a pure oscillation. A muscle component is a fuzzy burst, strongest near the temples.

MNE can help. ica.find_bads_eog() correlates each component with an EOG signal, band-filtered to 1 to 10 Hz, and returns the ones that match. The EOG can be a real channel or a stand-in such as Fpz. Notice the difference from regression: here the EOG only picks which component to drop. It is never subtracted from the data, so its small share of brain does no harm.

Step 3: exclude

Put the chosen indices in ica.exclude. Nothing changes yet. This is just a list. You can add one more component and look again.

Step 4: rebuild

ica.apply(raw) rebuilds the sensor signals from the components you kept, plus the PCA residual. In symbols, the cleaned data is Akept times skept: the mixing matrix with the excluded columns removed, times the sources with the excluded rows removed. It is applied to the unfiltered Raw object.

Worked numbers, with the toy patterns below. At the blink peak the sources are blink 1.0 and alpha 0.5. Fp1 is 90 × 1.0 + 3 × 0.5 = 91.5. O1 is 4.5 × 1.0 + 16 × 0.5 = 12.5. Exclude the blink and Fp1 becomes 3 × 0.5 = 1.5, while O1 becomes 16 × 0.5 = 8.0. The blink is gone, and the alpha rhythm is untouched in both channels.

Exclude a component, rebuild the channels

Each row is a component: four bars show its scalp pattern (Fp1, Cz, O1, T8) and the line shows its time course. Decide which are artifacts. Press a button to exclude it. Below, warm is the recording and green is the rebuilt channel.

Illustrative: four sources with exact, known patterns, so the unmixing is perfect. A real fit estimates the patterns from the data and never separates this cleanly, which is why you check the components by eye. Sizes in µV are made up. Order is shuffled on purpose.

The recipe as code

pythonimport mne
raw.info['bads'] = ['EEG 053']                          # mark bad channels first
filt = raw.copy().filter(l_freq=1.0, h_freq=None)    # (n_ch, T) high-passed copy
ica = mne.preprocessing.ICA(n_components=15, random_state=97)
ica.fit(filt)                                          # learns W on the copy
eog_idx, scores = ica.find_bads_eog(raw, ch_name='Fpz')
ica.exclude = eog_idx                                  # a list, nothing changes yet
clean = ica.apply(raw.copy())                          # A_kept s_kept + PCA residual

Symptom, measurement, fix

The symptom is a blink that survives, or a decoder that has lost a rhythm it should have. The measurement is the component’s scalp pattern and spectrum, and how much variance it explains. The fix is to exclude the component that fits the artifact and nothing else. Over-excluding is the failure to watch: drop the alpha component by mistake and the device shows that the cleaned channels lose their rhythm.

One more point for a live system. After the fit, cleaning is a single matrix: P = Akept Wkept, with shape (n channels, n channels). Apply P to each new sample and you have removed the blink. That needs no future samples, so a live BCI can use a fixed, pre-fitted ICA.

Why does the MNE recipe fit ICA on a 1 Hz high-passed copy and then apply it to the unfiltered data?

Chapter 8

The Order of Steps, and the Online Problem

Put six steps in an order that lets each help the next, then ask which of them a live decoder can use

A new teammate reads this lesson and writes the pipeline in the order the chapters came. It runs. The plots look fine. A month later someone finds that the ICA was fitted on data that still held a dead channel, and the average reference included that same dead channel. Nothing crashed. The results were just a little worse than they should have been, for a month.

Order is a quiet source of errors in preprocessing, because every step accepts whatever the last one gave it. The rules below each come from a stated reason. Learn the reasons, and the order follows.

Six rules, each with a reason

Mark bad channels before ICA. The MNE bad channel page says that ICA is corrupted by noisy channels, the same way a projection method (SSP) is. Mark bad channels before the average reference. MNE leaves bad channels out when it computes the average. A bad channel left in would smear its noise over every other channel, as Chapter 3 showed.

Filter before fitting ICA. Slow drift weakens the independence of the sources, and the tutorial recommends a 1 Hz high-pass for the fit. Fit before you apply. You cannot exclude a component that does not exist yet.

Interpolate after ICA. An interpolated channel is a weighted sum of its neighbours. It carries no new direction in the data, so the data have fewer independent directions than they have channels. ICA does not expect that. Do the repair after the components are removed. And mark before you interpolate, because the bad list is what tells the repair which channels to rebuild.

One thing these rules leave open is where the reference goes relative to ICA. Pipelines differ. Pick one, and write it down next to the model.

Here is the cost of one slip, in numbers. Suppose you set the average reference over 64 channels while a dead channel is still in the list. The dead channel reads close to zero, so it adds almost nothing to the sum, but it still counts in the division. The mean is then about 1/64 too small, and every channel keeps a small error. Now suppose the dead channel is a noisy one with 100 µV of junk. The average carries 100/64, about 1.6 µV of that junk, upside down, into all 64 channels. That is more than the 1 µV of fast brain activity from Chapter 0.

Put the steps in order

The list starts in a poor order. Move steps up and down. The rules under the list turn red when you break them, with the reason.

      The rules come from the MNE-Python bad channel, reference and ICA tutorials, and from the rank argument above. They are rules of thumb, not law; they leave several valid orders.

      What a live decoder cannot use

      Everything so far assumed a finished file. A live BCI sees one sample at a time, and the future does not exist. Walk down the pipeline and ask of each step: does it need future samples?

      StepOffline fileLive stream
      Notch, high-passZero-phase FIR looks ahead; no delayNeeds a causal filter; adds delay
      Bad channel checkVariance over the whole fileVariance over a sliding window
      InterpolationMatrix from sensor positionsThe same matrix, once the bad list is set
      ReferenceSubtraction per sampleThe same; no future needed
      ICA fitFit on the whole recordingFit earlier, e.g. a calibration block
      ICA applyMatrix productThe same fixed matrix per sample

      The reference, the interpolation and the ICA clean-up are each one fixed matrix applied to the channels of the current sample. They can run live with no delay. The filters are the problem. A symmetric (zero-phase) filter of length L uses (L − 1) / 2 samples of the future. Make the 9,911-sample filter from Chapter 1 causal and it lags by (9,911 − 1) / 2 = 4,955 samples. At the tutorial’s 600.6 Hz, that is 8.25 seconds.

      A live design therefore uses short causal filters, accepts a gentler cutoff, or uses a different filter family. It pays in delay or in sharpness. That is the trade a control loop has to make on purpose.

      Train and serve with the same pipeline

      The failure that matters most for an ML engineer is train-serve skew: a model is trained on features made one way and run on features made another way. A decoder trained on zero-phase, whole-file-cleaned data meets causal, windowed, delayed data online. Its inputs now differ from its training inputs, by a filter delay and a different noise floor.

      The symptom is a decoder that scores well offline and poorly live. The measurement is simple: send one recording through both paths and compare the features sample by sample. The fix is to train on the data produced by the live path, or to replay it offline with the same causal code.

      Which sequence follows the rules of this chapter?

      Chapter 9

      Field Guide: The Whole Cleaning Pass on One Page

      Keep the numbers, the checklist and the code, and see where to go next

      The one-paragraph summary

      Raw EEG is a blend of brain and junk, and the junk is often larger than the brain. A notch removes mains hum and a high-pass removes drift. A variance test flags bad channels, and interpolation rebuilds them. A reference is subtracted from every channel, so one bad reference spoils all of them, and the average reference shares a bad channel out. Blinks have a fixed scalp pattern; regression against an eye channel removes some brain too. Muscle overlaps brain in frequency, so a filter cannot split them. ICA unmixes sources by looking for the least bell-shaped outputs, and you exclude the artifact components and rebuild. The order matters, and a live system can use only the steps that need no future.

      The numbers that matter

      WhatNumberSource
      Muscle against fast brain at the scalpabout 100 µV against less than 1 µVMuthukumaraswamy 2013
      Scalp muscle bandroughly 20 to 300 Hz; jaw 50 to 60, forehead 30 to 40, neck about 100 HzMuthukumaraswamy 2013
      Mains hum50 or 60 Hz, plus harmonics (60, 120, 180, 240 in the MNE example)MNE filtering tutorial
      MNE notch filter0.50 Hz transition, 3,965 samples, 6.6 sMNE filtering tutorial
      Slow drift in the exampleperiod about 20 s; a 0.1 Hz high-pass was not enoughMNE filtering tutorial
      High-pass before ICA1 HzMNE ICA tutorial
      ICA components in the tutorial15; explain 78.5% of EEG and 93.4% of magnetometer varianceMNE ICA tutorial
      EOG band used to find blinks1 to 10 HzMNE ICA tutorial
      Robust spread from the MAD1.4826 × MADStandard statistics
      Average referencey′ = (I − 1 1T/n) y; channels sum to 0; rank n − 1Derived in Chapter 3
      Zero-phase FIR with 0.20 Hz transition9,911 samples, 16.5 s; causal lag 8.25 sMNE tutorial; the lag is derived here

      The checklist

      The core as code

      pythonimport numpy as np
      
      def bad_channels(x, k=3.5):                    # x: (n_ch, T)
          lv  = np.log10(x.var(axis=1) + 1e-12)         # (n_ch,) log variance
          med = np.median(lv)
          mad = np.median(np.abs(lv - med))
          z   = (lv - med) / (1.4826 * mad + 1e-9)       # robust z-score
          return np.where(np.abs(z) > k)[0]
      
      def average_reference(x, bad=()):
          good = np.setdiff1d(np.arange(len(x)), bad)       # leave bad channels out of the mean
          return x - x[good].mean(axis=0, keepdims=True)
      
      def ica_clean_matrix(A, W, drop):                 # A: (n, k) mixing, W: (k, n) unmixing
          keep = np.setdiff1d(np.arange(A.shape[1]), drop)
          return A[:, keep] @ W[keep]                     # (n, n); clean = P @ x, one fixed matrix

      Where it sits in the field

      These steps are the quiet first stage of every non-invasive pipeline. The recording hardware in Lecture 4 sets how much junk arrives, and the recording map shows where EEG sits among the ways to listen. Cleaned epochs feed event-related potential work in the ERP and P300 lesson, and the cleaned signal feeds a decoder like the Kalman decoder. Implanted electrodes sit far from the eyes and jaw, so blinks and muscle matter much less there; spike work has its own cleaning, in spike detection.

      Companies building non-invasive systems face this at scale. Sabi, for example, describes on its site brain imaging with sensors built on custom ASICs and a large decoding dataset; that is the company’s own claim. How much cleaning a learned model needs is a question for each model’s paper, so read what preprocessing it assumes before you reuse it.

      Keep going

      Sources used

      The takeaway. Every cleaning step trades something away. A good pipeline names the artifact each step removes, measures that it is gone, checks what real signal left with it, and keeps an order and a design that a live decoder can repeat.
      Your decoder scores 90% offline and 60% live, with the same people and tasks. What do you check first?

      Now press Teach and draw the whole cleaning pass from memory: the six artifacts, the fix for each, what each fix costs, and the order. Then go back to the four traces at the top, switch the fixes on one at a time, and say aloud which junk each one removed and which real signal it took along.

      BCI 11: EEG Preprocessing
      MNE-Python ICA tutorial · Jung et al. 2000 · Muthukumaraswamy 2013 · Back to Gleams