The Recording Map

Every brain sensor listens from a different distance. Move from one neuron out to the scalp and watch two nearby signals merge into one: the same brain, a blurrier map.

Pick a sensor, then drag the gap between two active spots. Find the gap where each sensor stops telling them apart. Then we build, piece by piece, what each sensor measures, how invasive it is, why the skull blurs EEG, and where new sensors try to land.

You need no neuroscience and a little arithmetic. We define every term, from neuron to skull conductivity, as it appears.

Lesson 3 of the series Brain-Computer Interfaces, from the spike to the foundation model. Built from primary sources (listed in the Field Guide), led by the review of Saha et al. 2021, and grounded on the lecture outlines of Princeton NEU 502B and UIUC ECE 421. We used no slides; every idea is told in our own words and devices.

Listen to two neighbours

6 mm

Toy model: the brain surface is one line, the two sources are equal points, and each sensor blurs every point into a bell curve whose full width at half height is that sensor’s spatial resolution. The resolutions are the review’s Table 1 (Saha et al. 2021): single neurons about 0.05 mm, ECoG about 1 mm, MEG about 5 mm, EEG about 10 mm. Two equal bell curves show a dip only when the gap is wider than 0.85 of that width. The line is drawn no thinner than 0.35 mm so you can see it.

Chapter 0

Why a Map of Sensors

See why one brain, listened to from five places, gives five different pictures

Picture a football stadium on match night. You want to know what one fan in row 40 is shouting. You could stand next to her. You could sit in the same stand. You could stand outside the stadium wall. Or you could fly a drone overhead and measure the heat.

Each seat gives you a different recording of the same night. Next to her, you hear her words. Outside the wall, you hear a roar, and you cannot tell one voice from another. The drone sees the crowd move, but only seconds later.

A brain-computer interface (BCI) is a system that reads activity from the brain and turns it into a command or a message. Every BCI starts with the same choice you just made in the stadium: where do you put the microphone? The answer sets what the rest of the system can ever know.

The brain in one paragraph

The brain holds billions of cells called neurons. A neuron sends a brief electrical pulse, a spike, that lasts about a millisecond. Groups of neurons fire together, and their combined activity makes electric fields, magnetic fields and changes in blood flow. A recording modality is one way to sense those effects. Each modality is a microphone in a different seat.

Two numbers that rank a sensor

The first is spatial resolution: how close two sources can sit and still look like two. The hero at the top of this page measures it. A sensor with 10 mm resolution cannot tell two neighbours 5 mm apart.

The second is temporal resolution: how short an event can be and still look like an event. A sensor with 1 s resolution turns a 200 ms burst into a faint bump. Try it below.

One burst, four traces

The orange bar is a real burst of brain activity. Each blue trace is what one kind of sensor reports. Drag the burst length and watch which sensors still show it at full height.

200 ms

Toy: each trace is the burst smoothed by a bell curve whose full width at half height is the sensor’s temporal resolution in Saha et al. 2021, Table 1: about 3 ms for intracortical and ECoG, 50 ms for EEG, 1 s for fMRI and fNIRS. Real blood-flow signals also lag the burst by seconds; the toy leaves the lag out.

The orange burst never changes. Only the seat does. A sensor with fast timing keeps the burst sharp. A sensor with slow timing spreads it out and lowers it, so a short burst can almost vanish.

A worked example: one 200 ms burst

Take the default burst, 200 ms long. Step 1: list each sensor’s timing from the table in Chapter 2. ECoG and needles, 3 ms. EEG, 50 ms. fMRI and fNIRS, 1 s. Step 2: divide the burst by the timing. 200 / 3 is about 67, so ECoG sees the burst as a long flat block with sharp edges. 200 / 50 is 4, so EEG still shows a clear block with rounded edges. 200 / 1000 is 0.2, so fMRI sees a bump a fifth as long as its own blur, and its height falls well below the true burst. Step 3: read the rule. A sensor keeps a burst at full height only when the burst lasts several times the sensor’s timing.

The failure mode: a team records a 50 ms burst with fNIRS and sees almost nothing. The sensor did not fail. Its own timing, about 1 s, is twenty times longer than the burst, so the burst is averaged away before any decoder sees it.

What this lesson does

We will not rank sensors as good and bad. Each sensor trades something for something else. Your job as an engineer is to read the trade. Later lessons in this series build decoders (the code that turns recordings into commands). A decoder can never recover what the sensor threw away.

Two sources frame the work. Saha et al. 2021 is a review that tabulates the resolutions we use. The 2006 review by Lebedev and Nicolelis gives the history of why the field moved from the scalp to the cortex. The lecture outlines of Princeton NEU 502B and UIUC ECE 421 both put these sensors in the same order: the physics first, then the signals.

The road

  1. What each sensor feels: electric, magnetic, blood and sound.
  2. The resolution map: every modality placed by space and time.
  3. The invasiveness ladder: from the scalp to inside the cortex.
  4. Why the scalp blurs the brain: volume conduction and the skull.
  5. Dense EEG: when more sensors help, and when they stop.
  6. New corners: ultrasound and molecules.
  7. From voltages to tensors: shapes and data rates.
  8. Matching the sensor to the job: reading the trade.
  9. Field Guide: numbers, terms, checklist, code, sources.
Two active brain spots sit 4 mm apart. The review lists EEG at about 10 mm spatial resolution and ECoG at about 1 mm. What do the two sensors report?

Chapter 1

What Each Sensor Actually Feels

Follow one burst of neural activity into electric fields, magnetic fields, blood flow and sound

Think of a crowd in a stadium again. When the crowd cheers, three things happen at once. The air shakes, so you can hear it. The stand heats up, so a thermal camera sees it. And the ticket gates log a surge a few minutes later. One event, three kinds of trace.

A burst of brain activity works the same way. It leaves electric, magnetic and chemical traces. Each recording modality picks up one kind of trace. Before we compare the sensors, we must know which trace each one reads.

Electric: the current that leaks out of a neuron

A neuron is a cell with a thin skin, its membrane. Charged particles called ions cross the membrane through tiny gates. Each crossing is a small electric current. Many crossings at once make a current large enough to spread through the fluid around the cell.

That current makes a small voltage difference in the tissue. An electrode is a metal contact that measures this voltage. Buzsáki, Anastassiou and Koch (2012) explain why the voltage tells us about neurons: extracellular recordings reflect transmembrane currents. The biggest contributor is the synaptic current, the current at the contacts where one neuron talks to the next. Spikes and other membrane events also shape the field.

So one physical quantity, the voltage in the tissue, feeds four modalities. They differ only in where the electrode sits.

Magnetic: the field around the same current

A moving electric current always makes a magnetic field. Magnetoencephalography (MEG) measures that field outside the head with very sensitive detectors. Saha et al. note one gain: the skull, skin and fluid around the brain pass magnetic fields in nearly the same way, so MEG “can capture signal with less distortion than EEG”. They list about 5 mm resolution. They also note the cost: MEG needs expensive, fixed equipment. You cannot wear it.

Blood: an indirect trace

Active neurons use energy. The body answers by sending more oxygen-rich blood to that region. The change in blood oxygen is called the BOLD signal (blood-oxygen-level dependent). Two tools read it. fMRI uses a large magnet and radio pulses. Saha et al. judge it “not feasible for most BCI applications, due to unmanageable size and cost”. fNIRS (functional near-infrared spectroscopy) shines near-infrared light through the scalp and measures how much blood absorbs. It is, in their words, “safe, non-invasive, relatively inexpensive and portable”. Both list about 1 s timing, because blood is slow.

Compare how far each trace sits from the neurons. A current is one step from the cell. A magnetic field is also one step: the same current. Blood oxygen is three steps: the cell fires, it uses energy, blood flow changes, and oxygen levels change. Choose a sensor below to see its path.

Follow the trace to the sensor

Pick a sensor. The left box is the neurons. The middle box is the trace they leave. The right box is the sensor. The bar below puts the sensor’s timing on a scale from 1 ms to 10 s.

Timing is Saha et al. 2021, Table 1 (about 3 ms intracortical and ECoG, 50 ms EEG and MEG, 1 s fMRI and fNIRS). The step count is our own simple tally: current or field, one step; blood, three steps (firing, energy use and blood flow, oxygen level).

Worked example: which sensor can follow a 100 ms event?

A person decides to squeeze a hand. The motor cortex bursts for about 100 ms. Step 1: list each sensor’s timing. ECoG, 3 ms. EEG and MEG, 50 ms. fNIRS and fMRI, 1 s. Step 2: compare with the burst. 3 ms is much shorter than 100 ms, so ECoG sees the shape. 50 ms is half the burst, so EEG sees it, but blurred. 1 s is ten times the burst, so fNIRS spreads it into a slow bump. Step 3: decide. For a hand that must open and close in real time, use an electric or magnetic sensor.

The failure mode. The symptom: a cursor driven by fNIRS moves about a second after the user decides. The measurement: the sensor’s own timing is about 1 s, so no decoder can make it quicker. The fix: use fNIRS for slow states, such as workload, and use an electric sensor for fast control.

Why can fNIRS not follow a 100 ms brain event, even with a perfect decoder?

Chapter 2

The Resolution Map

Place every modality by how finely it sees in space and in time

You are planning a BCI that lets someone type by thinking of a letter. The system must tell apart two small spots of cortex a few millimetres apart. And it must do so within a tenth of a second, or typing feels broken. Which sensors qualify?

You can answer that with a single picture: a map with space on one axis and time on the other. Every sensor is a point. A sensor near the top-left corner sees fine detail and sees it quickly.

The numbers behind the map

Saha et al. (2021) give one table for the whole field. We use it as our map. Every value in it is approximate (they write “~”), and other papers quote somewhat different values. Treat the table as a map of orders of magnitude, not as a spec sheet.

ModalityTimeSpaceInvasive?
EEG~0.05 s~10 mmNo
MEG~0.05 s~5 mmNo
ECoG~0.003 s~1 mmYes
Intracortical~0.003 s~0.5 mm (LFP), ~0.1 mm (MUA), ~0.05 mm (SUA)Yes
fMRI~1 s~1 mmNo
fNIRS~1 s~5 mmNo
PET1 to 2 min~4 mmListed as yes

Read the table by columns. Time spans from 3 milliseconds to 2 minutes: a factor of 40,000. Space spans from 0.05 mm to 10 mm: a factor of 200. Because the ranges are so wide, we plot them on a logarithmic scale, where each step to the right multiplies the value by 10. Equal steps then mean equal ratios, and all seven modalities fit on one page.

One note on the table. It lists PET (positron emission tomography, which tracks an injected radioactive tracer) as invasive without explaining why. We keep the label as published and do not use PET again.

How to read the map

Look at where the points cluster. The three electric tools that sit close to the neurons share one column on the left, at 3 ms. The two blood tools share a column on the right, at 1 s. EEG and MEG sit in the middle at 50 ms. Space spreads the points vertically: the finest tools sit at the top, the coarsest at the bottom. Notice that no point lies in the top-left corner without a yellow ring, the mark for surgery. That pattern is the main finding of the table.

Draw what you need, and see who qualifies

Now set your own need. The green box in the device below is the region of the map that meets your two limits. A sensor qualifies only if its point is inside the box. The yellow ring marks a sensor that needs surgery.

Set the need, find the sensors

Up means finer detail in space. Left means faster in time. Drag the two sliders to move the corner of the green box. Sensors inside the box turn green.

3 mm
0.1 s

Points are Saha et al. 2021, Table 1, with the three intracortical spatial values shown as separate points. A sensor qualifies when its spatial value is at or below your detail limit and its timing is at or below your speed limit. Approximate values, one review.

Start with the defaults: 3 mm and 0.1 s. Two groups qualify, ECoG and the intracortical probes, and both need surgery. Now slide the detail limit to 10 mm and the speed limit to 0.1 s. EEG and MEG join. Slide the speed limit to 3 s and fMRI and fNIRS join. The upper-left corner of the map, fine and fast without surgery, stays empty in this table. That empty corner is the grail of non-invasive BCI, and Chapter 6 returns to it.

Worked example: the typing BCI

The need: two spots 3 mm apart, within 100 ms. Step 1: filter by space. Keep every sensor with spatial value at or below 3 mm. That keeps ECoG, all three intracortical levels and fMRI. Step 2: filter by time. Keep timing at or below 0.1 s. fMRI at 1 s drops out. Step 3: what is left is ECoG and intracortical recording. Step 4: check invasiveness. Both need surgery. The map shows the price of the need.

The failure mode. The symptom: a decoder trained on scalp EEG cannot separate two classes that come from neighbouring spots. Accuracy stays near chance, however big the model. The measurement: the gap between the spots is smaller than EEG’s roughly 10 mm resolution, so the two classes look the same on the scalp. The fix: change the task to classes that sit far apart, such as left hand and right hand, which lie in different halves of the brain, or change the sensor.

You need detail of 3 mm and timing within 0.1 s, and you may not use surgery. What does the Saha et al. table offer?

Chapter 3

The Invasiveness Ladder

Climb from the scalp to the cortex and count what you gain and what it costs

A person lies in a hospital bed, paralysed from the neck down. She would like to move a cursor with her mind. A surgeon asks one question before anything else: how close to the brain must the sensor sit? Every centimetre closer buys a sharper signal and costs a larger operation.

An invasive sensor goes inside the body. A non-invasive sensor stays outside. In between sit sensors that need only a small cut or a catheter. We can line the options up as rungs on a ladder, from the safest to the most demanding.

Four rungs

Rung 0: the scalp. EEG caps, MEG helmets and fNIRS headbands sit outside. No surgery. Between the cortex and the sensor lie the skin, the skull and a layer of fluid and membranes. Saha et al. say it plainly: “Non-invasive BCI exploiting EEG are most common.”

Rung 1: a vein. A stentrode is a stent, a small mesh tube that props a blood vessel open, with electrodes woven in. A surgeon threads it through a catheter into a vein that runs over the motor cortex. Oxley and colleagues (2016) implanted one in sheep and recorded for up to 190 days while the vein stayed open. Its spectral content and bandwidth were comparable to recordings from epidural surface arrays, which sit under the skull on the outer membrane of the brain. Saha et al. list this approach as minimally invasive. A later study in four patients (SWITCH, 2023) reported no serious device-related adverse events over 12 months.

Rung 2: the cortex surface. ECoG puts a flat grid of electrodes directly on the brain, under the skull. The surgeon must open the skull. The grid sits close to the neurons and has no bone between, so its resolution is about 1 mm.

Rung 3: inside the cortex. Penetrating arrays push electrode needles into the tissue. The Utah array has 100 needle sites in a 10 by 10 grid, spaced 400 micrometres apart, each 1.5 mm long (Campbell and colleagues, 1990; Normann and colleagues, 1999). Neuropixels probes are thin silicon shanks. One 10 mm shank carries 960 recording sites, and 384 of them record at once (Jun and colleagues, 2017). With two probes, that team recorded more than 700 separate neurons at once in an awake mouse. These probes hear single neurons, at 0.05 to 0.5 mm.

Try the four rungs in the device below. It counts the tissue layers between the sensor and the cortex surface.

Move the sensor inward

The orange dot is a group of active neurons. Pick a rung. The teal marker is the sensor. The line shows what lies between sensor and source. The yellow bar shows the cost of getting there.

A schematic, not to scale. Layer count: skin, skull, and fluid with membranes lie between the scalp and the cortex surface. Resolutions are Saha et al. 2021, Table 1; the stentrode has no table row, and Oxley et al. 2016 found its recordings similar in bandwidth to surface arrays in sheep.

What closeness buys, and what it costs

Saha et al. give the benefit in one line: invasive electrodes offer “a superior signal-to-noise ratio and better localization of brain activity”. Signal-to-noise ratio (SNR) is how big the brain signal is compared with everything else the electrode picks up. They also give the cost: invasive BCI “should generally not be considered for neurologically intact people due to risks associated with surgery”.

The risk is measured, not guessed. In the BrainGate feasibility study (Rubin and colleagues, 2023), 14 adults received Utah arrays between 2004 and 2021, for an average of 872 days each, 12,203 days in all. The most common device-related event was skin irritation around the connector where the wire leaves the head.

Notice a second cost that the ladder hides. A closer sensor sees a smaller patch. A Utah array covers a square of about 4 by 4 mm. One EEG cap covers the whole scalp. Detail and coverage trade against each other.

Worked example: the layers between

Count the barriers. The scalp sensor has three layers between it and the cortex: skin, skull, fluid. The vein sensor has none of those, only the wall of the vessel. The ECoG grid and the needles have none. The signal from the orange dot must cross three layers to reach the cap and none to reach the grid. The next chapter shows what those three layers do to the signal.

The failure mode. The symptom: a team proposes a penetrating array for healthy volunteers to record faster. The measurement: Saha et al. advise against invasive BCI for neurologically intact people because of surgical risk. The fix: for healthy users, work to make the scalp sensor better, which is the aim of the dense-EEG efforts in Chapter 5.

Which statement about the ladder matches the sources?

Chapter 4

Why the Scalp Blurs the Brain

Watch current spread through the head until two sources look like one

Put your ear against a thick wall while a party goes on next door. You hear that people are talking. You cannot tell who. The wall has not removed the sound. It has spread it, and mixed every voice into one low murmur.

The skull does that to brain signals. A cluster of neurons makes a small current. The current has to travel through bone, fluid and skin before an EEG electrode feels it. By the time it arrives, it is wide, weak and mixed with every other source.

Current looks for any way to flow

Electricity needs a closed loop. A source in the cortex pushes current out into the tissue, and the same current must find its way back. It does not follow a wire. It spreads through every conductive material around it, like water finding every crack. This spreading through a conductive block is called volume conduction.

Materials differ in conductivity: how easily they pass current. Brain, fluid and skin conduct well. Bone conducts badly. In a head model used to study this, the scalp and brain are both given 0.33 siemens per metre, and the skull is set much lower. Only the ratio matters, the brain-to-skull conductivity ratio (BSCR).

Why does a poorly conducting skull blur the picture? Current cannot easily climb through the bone straight above the source. So it spreads sideways, in the better-conducting layers under and over the bone, before it reaches the scalp. The voltage that an electrode on the scalp feels is spread over a wide patch. Move the source a little, and the pattern on the scalp barely changes.

How large is the ratio? The textbook value is 80. Rush and Driscoll measured it in 1968 with a skull in an electrolyte tank, and Cohen and Cuffin supported it with combined EEG and MEG data. Later work disagrees: one study of five epilepsy patients estimated 24.8 with a spread of 6.6. The two values differ by a factor of three. That much doubt sits inside the most basic number in EEG.

MEG avoids part of this. Saha et al. explain that skull, skin and fluid pass magnetic fields in much the same way as air, so MEG “can capture signal with less distortion than EEG”.

The forward model: from sources to scalp

We can write the whole story as one line of linear algebra. Let each of K brain locations carry a source strength. Let M scalp electrodes each report one voltage. Each source spreads into a fixed pattern across the electrodes. Stack those patterns as the columns of a table.

v
The M voltages on the scalp: one number per electrode. A 64-electrode cap gives 64 numbers at each instant.
L
The lead field, a table with M rows and K columns. Column j is the smear that source j makes across the electrodes. The skull sets how wide each smear is.
s
The K source strengths. This is the brain activity we want. K runs to thousands, far more than M.
n
Noise: muscle, wiring, the electrode skin contact, and every part of the brain we do not care about.

Going from s to v is the forward problem, and it is easy: multiply. Going back from v to s is the inverse problem, and it is hard for two reasons. First, there are more unknowns (K) than equations (M), so many different s fit the same v. Second, each smear is wide, so neighbouring columns of L look almost alike, and a little noise makes them swap places. A common fix picks, of all the s that fit, the one with the least energy: ŝ = LT(LLT + λI)−1v, where λ is a small number that tells the method how much to distrust the noise.

Blur the sources, then try to undo it

Top: three true sources on a 60 mm line, at 18, 26 and 43 mm. Middle: the 24 scalp voltages after the smear. Bottom: the least-energy reconstruction. Widen the smear, raise the noise, or make the method guess the skull wrongly, and watch peaks merge or appear from nowhere.

10 mm
3%

Toy, not a head model: one dimension, a bell-curve smear whose full width at half height you set, 24 evenly spaced electrodes, noise as a percentage of the largest voltage (a fixed random pattern), and the least-energy method above with λ matched to the noise. The “guess” buttons make the method assume a smear half or double the true one. A real head is three-dimensional and the smear depends on depth and skull. A peak counts when it passes 35% of the tallest.

Start with the exact guess and 3% noise. At a 6 mm and a 10 mm smear, all three sources come back in place, even though the two sources 8 mm apart sit inside the 10 mm smear. At 18 mm they merge into two. So a stated resolution is a rule of thumb, not a wall. A clean signal and a known skull let the inverse step beat it. Now raise the noise to 10% and set the smear to 14 mm: two peaks. Now make the method guess too sharp, at a 10 mm smear: it returns six peaks, three of them ghosts. Guess too blurry, and the near pair merges. The smear erases information, and noise plus a wrong skull model decide how much you can win back.

Worked example: the cost of a wrong head model

A research team studied this with a three-layer head model: brain, skull and scalp (the study is listed in the Field Guide). They placed a known source, simulated the scalp voltages, then solved the inverse problem with a different skull ratio than the one used to simulate. They tried ratios of 15, 20, 25 and 80. Step 1: with no noise and the right ratio, the error was about zero. Step 2: with no noise and a wrong ratio, the error rose to 2 to 16 mm. Step 3: with noise at 10 dB signal-to-noise and the right ratio, the error was about 9 mm from noise alone. So both the uncertain skull and the noise cost millimetres, and those are the same millimetres we hoped to resolve.

The failure mode. The symptom: a localization map shifts by a centimetre when you swap one head model for another. The measurement: the assumed skull ratio. The same data under ratios 15 to 80 gives errors from 2 to 16 mm. The fix: estimate the ratio for each person, or use a decoder trained end to end on the voltages, which never needs a map of sources.

Why does the inverse problem for EEG have many equally good answers?

Chapter 5

Dense EEG: More Sensors, Sharper Picture?

Test how many scalp sensors help, and where more sensors stop helping

You are in a dark room, listening for a whisper. One microphone hears a murmur. Ten microphones, spread around the room, hear which corner it comes from. A thousand microphones would hear more, but only up to a point. Once the microphones sit closer together than the whisper can be told apart, more of them add little.

Scalp EEG faces the same question. A common research cap has 64 electrodes. A high-density cap has 128 or 256. Some labs have built arrays of several hundred, and some companies talk about far more. Does density fix the blur of Chapter 4?

What more sensors can and cannot buy

Two effects pull in different directions.

Sampling. The skull smears every source into a wide bump on the scalp. Think of the scalp voltage as a smooth picture. To record a smooth picture, you need to sample it at least twice across its narrowest feature. This is the sampling rule from signal processing (our DFT lesson shows it for time). Here it works over space. If electrodes sit closer than half the smear width, the picture is already fully sampled. Extra electrodes then add no new shape.

Averaging. Each electrode adds its own noise. If that noise is independent from one electrode to the next, then averaging N electrodes cuts it by the square root of N. Four times as many electrodes halve the noise. This effect does not stop when the picture is fully sampled. But it fails when the noise is shared, such as muscle activity that reaches every electrode, or a common reference.

So density helps in two stages. At first it fills in missing shape. Later it only averages noise.

What the sources report

Two numbers anchor the idea. The head-model study from Chapter 4 ran the same inverse problem with 31 and with 128 electrodes, noise at 10 dB signal-to-noise. With 31 electrodes the mean localization error ran from about 9.0 to 17.3 mm, depending on source depth and skull ratio. With 128 electrodes it fell to about 2.0 to 11.8 mm. Saha et al. report a second number from the literature: an ultra-dense array of 700 to 800 electrodes reached a signal-to-noise ratio twice as high as high-density EEG with up to 256 gold-coated electrodes. They also warn that dry electrodes, which need no gel, “are assumed to provide lower signal-to-noise ratio compared to conventional wet electrodes”, and that high-density mapping raises computing cost and the effort to keep quality across channels.

Now the company side, with a clear label: these are claims. The start-up Sabi describes on its website a pipeline of three steps: brain imaging with novel sensors on custom chips, a large recorded dataset, and a “Brain Foundation Model” that maps signals to thoughts. The site mentions sensors with a 1 mm footprint. Press coverage adds figures we could not check: up to about 100,000 non-contact, chip-scale sensors of 1 to 5 mm, worn in a cap, and a custom amplifier chip said to be more power-efficient than a standard part. The idea fits what you just learned: when each sensor is cheap and small, you trade per-sensor quality for count, then lean on averaging and on a learned model to make up the gap. Whether it works is an open question for measurement, not for slides.

Add electrodes, watch the error

The curve shows the position error of the reconstruction for every electrode count, with the smear and noise you set. The teal dot marks your count. Find the elbow where more electrodes stop helping.

12
10 mm
5%

The same toy as Chapter 4 (three sources at 18, 26 and 43 mm on a 60 mm line, least-energy reconstruction, exact skull guess). Error is the average distance from each true source to the nearest reconstructed peak, with a lost source counted as 12 mm. The noise is independent from electrode to electrode, which real muscle noise is not. Not a measurement of any product.

Worked example: the elbow

Take a 10 mm smear. Step 1: the sampling rule says the spacing should be at most half the smear, 5 mm. Over a 60 mm line, that needs 60 / 5 + 1 = 13 electrodes. Step 2: check the device at 5% noise. With 4 to 8 electrodes the average error is 2.7 to 4.3 mm, because gaps between electrodes miss the shape. With 12 electrodes it falls to 1.0 mm. Step 3: from 16 to 96 electrodes the curve wanders between 0.3 and 1.7 mm and does not fall steadily, because the grid has 1 mm steps and the noise pattern is fixed. Step 4: what is left is noise. At 5% noise per electrode, 64 electrodes average it to 5% / 8 = 0.6%, so the remaining error comes from the smear and the skull model, which no electrode count fixes.

The failure mode. The symptom: a team multiplies the electrode count by ten and the decoder barely improves. The measurement: the spacing is already below half the smear width, and the dominant noise, such as muscle activity and the common reference, is the same on every channel, so averaging does not remove it. The fix: raise the quality per sensor, remove shared artefacts (for example with independent component analysis), and model the skull better.

A 10 mm smear is fully sampled by electrodes 4 mm apart. A lab adds ten times more electrodes. What should it expect?

Chapter 6

New Corners of the Map: Ultrasound and Molecules

Find the empty places on the map where new sensors are trying to land

Look again at the map from Chapter 2. One corner is empty: fine detail, fast timing, no surgery. Everything above the line of today’s table is either coarse (the scalp) or needs a surgeon. A new sensor that landed in the empty corner would change what a BCI can be.

Why is it empty? Because each non-invasive tool fights a different physical wall. Electric fields get smeared by the skull (Chapter 4). Blood signals are slow (Chapter 1). To fill the corner, a sensor needs a physical route around both walls.

Why people look at sound

Ultrasound is sound above the range of human hearing. A probe sends short pulses into tissue and listens for the echoes. Echoes come back from boundaries, such as the wall of a blood vessel. Medical scanners use it every day. Unlike light, ultrasound passes through soft tissue and can reach deep. Unlike an electric field, it is not smeared sideways by the bone in the same way (bone does bend and weaken sound, so research systems often work through a thinned skull or a window).

The best-known neuroscience use is functional ultrasound imaging. Macé and colleagues (2011) built a version in rats. It sent many unfocused sound waves per second and combined the echoes to see blood moving in small vessels. The method, called ultrafast Doppler, mapped changes in cerebral blood volume across a slice of the rat brain, with a probe placed over the head. We do not quote its frame rate here, because we could not open the methods section. Notice what it measures. It sees blood, so it inherits the slow blood trade of Chapter 1. It sees very fine detail in blood vessels. It does not see spikes.

A second route is to put tiny devices in the brain that talk by sound. Saha et al. mention “neural dust”, a wireless ultrasonic system that records muscle and nerve signals at the millimetre scale. The sensors are implants, so this route is invasive, but it needs no wires through the skull.

The molecular route

A third route changes the cells instead of the sensor. In 2018, Bourdeau and colleagues made bacteria produce gas vesicles, tiny air-filled protein shells that bounce sound strongly, so ultrasound could see the microbes inside a mammal. In 2019, Farhadi and colleagues, from the same laboratory (led by Mikhail Shapiro), built a genetic program that made mammalian cells produce them as well. A cell wearing a gas-vesicle gene becomes visible to ultrasound. A cell could in principle also be changed so that its activity changes how it sounds. That is a research goal, not a finished tool.

Merge Labs states a goal in this direction. On its website, dated 15 January 2026, the company says it aims to build BCIs that “connect with neurons using molecules instead of electrodes”, to use deep-penetrating signal methods such as ultrasound to send and receive information, and to avoid placing implants in brain tissue. It says it wants to raise BCI bandwidth and brain coverage “by several orders of magnitude”, and that this takes “thinking in decades rather than years”. The page gives no resolution or timing figure. So we cannot place Merge on the map. The next device lets you place any such claim yourself.

Land a new sensor on the map

Set a sensor’s detail, speed and surgery rung. A sensor beats another when it is at least as fine, at least as fast and no more invasive, and strictly better in one. Find a landing spot that nothing beats.

3 mm
0.1 s

The seven reference points are Saha et al. 2021, Table 1 (PET left out; intracortical shown by its MUA value, 0.1 mm), with rungs from Chapter 3: EEG, MEG, fMRI, fNIRS rung 0; ECoG rung 2; intracortical rung 3. Your sensor is hypothetical: no real product is placed here.

Worked example: three millimetres, a tenth of a second, no surgery

Place a sensor at 3 mm, 0.1 s, rung 0. Step 1: compare with EEG (10 mm, 0.05 s, rung 0). Yours is finer but slower, so neither beats the other. Step 2: MEG (5 mm, 0.05 s, rung 0). The same: yours is finer, slower. Step 3: fNIRS (5 mm, 1 s, rung 0). Yours is finer and faster, so it beats fNIRS. Step 4: fMRI (1 mm, 1 s, rung 0). Yours is coarser and faster: no winner. Step 5: ECoG (1 mm, 3 ms, rung 2). It is finer and faster but needs surgery: no winner. Result: your sensor beats one point and nothing beats it. It sits on a new front.

The failure mode. The symptom: a pitch promises “surgery-free, single-neuron quality” and shows no numbers. The measurement: ask for the three numbers, detail, speed and rung, and try to place the claim on the map. If you cannot, it is not yet a claim you can test. The fix: wait for published detail and timing, from an independent group where possible.

Functional ultrasound imaging maps cerebral blood volume very quickly. Why does it not fill the empty corner of the map?

Chapter 7

From Voltages to Tensors

Turn each modality into the array and the data rate a model receives

A sensor does not hand a model a thought. It hands over a stream of numbers: one voltage per channel, thousands of times each second. Everything a decoder will ever do starts from that stream. Its shape, its rate and its size decide what a model can afford to be.

Imagine you are building the software behind a cap with 64 electrodes. The chip measures each electrode 250 times per second and writes each measurement as a 24-bit number. How much data is that? Does it fit through a Bluetooth link? And how do you turn it into the array a neural network expects?

Three numbers set the size

A channel is one electrode’s stream of voltages. The sampling rate is how many measurements per second each channel makes, in hertz (Hz). The bit depth is how many binary digits store each measurement, which sets how finely the voltage is rounded. Our quantization lesson shows the rounding in detail.

Multiply the three. Channels times rate times bits gives bits per second. Divide by 8 for bytes. For the cap: 64 channels × 250 Hz × 24 bits = 384,000 bits per second, which is 48,000 bytes, or 48 kB, every second. That is 172.8 MB per hour, per person. A probe with 384 channels sampled at 30 kHz and 10 bits (a typical research setting, an illustrative figure here) produces 384 × 30,000 × 10 / 8 = 14.4 MB every second, which is 51.8 GB per hour. Closer sensors pay for their detail with data.

Size the stream

Pick a preset or set the three numbers. The block is one window of data: rows are channels, columns are time samples. The bar shows data per hour on a log scale.

64
250 Hz
2 s

Illustrative settings, not datasheet values for any named product. The arithmetic is exact: bytes per second = channels × rate × bits / 8. The presets set the bit depth (24, 16 and 10); the sliders change channels, rate and window.

From raw samples to a model input

Raw samples are integers from the chip. A decoding pipeline follows five steps. Each step changes the shape.

pythonraw = read_chip()                  # (64, 250*60) int32 counts, one minute
uv  = raw * LSB_UV                   # (64, 15000) microvolts: counts x volts per count
x   = bandpass(uv, 1, 40, fs=250)    # (64, 15000) keep the band you need
x   = x - x.mean(axis=0)             # (64, 15000) re-reference: remove what all channels share
win = windows(x, 500, step=125)      # (116, 64, 500) 2 s windows, 0.5 s apart
y   = model(win)                     # (116, n_classes) one prediction per window

The last two shapes are what a model sees. Different sensors give different shapes. EEG gives (batch, channels, time). A spike probe gives (batch, time bins, units) after you count spikes per bin, for instance per 20 ms. fMRI gives (x, y, z, time), a whole 3D volume every second or so. The sensor’s place on the map fixes the model’s input.

The rate must fit the band

How fast must you sample? The Nyquist rule says: at least twice the highest frequency you want to keep. Suppose a design keeps activity up to 200 Hz (an illustrative choice, since fast activity is often used in ECoG work). That needs at least 400 samples per second. To record spike waveforms, which last about a millisecond, a probe samples tens of thousands of times per second. Frequencies above half the rate do not vanish. They fold back as false slow waves.

The failure mode. The symptom: a 10 Hz rhythm appears in recordings from a device in a room with mains power. The measurement: the mains hum is 60 Hz. Sample it at 70 Hz and it folds to |60 − 70| = 10 Hz, right inside the brain band. The fix: an analog filter before the converter that removes everything above half the sampling rate, a higher sampling rate, and a notch filter for the hum. After sampling, no software can tell the false wave from a real one.

A 64-channel cap samples each channel at 250 Hz with 24 bits. How much data is that per second?

Chapter 8

Matching the Sensor to the Job

Pick a sensor for a task by the trade it forces

Four people walk into a lab. One has lost the use of her hands and wants to move a cursor. One has lost her speech and wants to talk again. One is a healthy gamer who wants to type with a headset. One is a researcher who wants a map of the whole brain. They all say “I need a brain sensor.” No single sensor serves all four.

This chapter turns the map into a habit. For any job, ask four questions, in this order, and the sensor mostly picks itself.

Four questions

  1. How fine must it see? The smallest gap between two things the decoder must tell apart. Compare it with the spatial column.
  2. How fast must it see? The shortest event that matters. Compare it with the time column.
  3. What will the person accept? Surgery or not. Saha et al. say invasive BCI should generally not be considered for neurologically intact people.
  4. Where will it be used? The same table marks which modalities are portable: EEG, ECoG, intracortical and fNIRS are; MEG, fMRI and PET are not.

The device below runs those checks for five jobs. The thresholds for each job are our own round numbers, for illustration. You can override the surgery and wearable answers and watch the list change.

Run the four checks

Pick a job. Each row is a sensor, each column a check. A green dot means it passes. A sensor fits only if all four dots are green. Toggle the two rules to change what the person accepts.

Sensor numbers are Saha et al. 2021, Table 1 (intracortical shown at its MUA value, 0.1 mm). The thresholds per job are illustrative round numbers of ours: cursor and speech 3 mm and 0.1 s with surgery allowed; headset typing 10 mm and 0.1 s without surgery; focus state 20 mm and 5 s; whole-brain map 5 mm and 2 s, wearable not needed.

A sensor is an observation model

For an engineer who builds decoders, the best way to hold all of this is a state-space view. Let x be what the brain is doing, such as the intended direction of a hand. The sensor gives a measurement y.

yt
The data at time t: a column of channel voltages, or a vector of spike counts. The sensor produces it.
xt
The hidden brain state that you want. It does not depend on which sensor you choose.
H
How the state shows up in the data. For EEG it is the smearing table L from Chapter 4. For a needle it picks out one neuron.
vt
Noise, with a spread you can write as a covariance. A bigger spread means a less trusted sensor.

This is the same form as the Kalman filter measurement equation. Choosing a sensor means choosing H and the noise spread. Everything else in the series, filters, smoothers, spike-count models, learned encoders, works on top of that choice. If the series later trains a model on one sensor and applies it to another, the observation model has changed. The decoder must be re-fit, or trained to work across sensors.

The tools line up with the sensors too. For slow oscillations in EEG, the time-frequency tools of the STFT and wavelet lessons apply. For separating muscle and eye artefacts from brain signals, ICA applies. For many correlated channels, PCA finds the few directions that carry most of the variance. For smoothing a noisy track in time, RTS smoothing and adaptive Kalman filtering apply.

Worked example: headset typing

Job: type with a headset. Step 1, detail: letters can be chosen using signals from brain regions centimetres apart, so 10 mm is enough. EEG passes at 10 mm; MEG passes; ECoG and probes pass but fail later. Step 2, speed: 0.1 s. EEG at 0.05 s passes. fMRI and fNIRS at 1 s fail. Step 3, surgery: none. ECoG and probes fail. Step 4, wearable: MEG fails. One sensor passes all four: EEG. Then Chapter 5 applies, because EEG is the weakest sensor in the detail check, so you want the densest, cleanest cap you can build.

The failure mode. The symptom: a decoder that worked with gel electrodes in the lab loses accuracy with dry electrodes at home. The measurement: Saha et al. note that dry electrodes are assumed to give a lower signal-to-noise ratio than wet ones, so the noise term v is larger and H is less stable. The fix: re-estimate the noise level and re-calibrate for the new sensor, rather than reuse the lab settings.

A job needs 10 mm detail, 0.1 s timing, no surgery and a wearable device. Which sensor in the table passes all four checks?

Chapter 9

Field Guide

Keep the numbers, the terms, the checklist, the code and the sources

You now hold the whole map. Brain activity leaves electric, magnetic and blood traces. Each modality reads one trace from one seat. The seat sets how finely the sensor sees in space, how quickly it sees in time, how much surgery it needs, and how big the data is. The skull blurs the electric trace; the blur and the noise set a floor under every EEG decoder. Dense caps, ultrasound and molecular tools are attempts to move the sensor into the empty corner of the map.

The numbers that matter

WhatNumberSource and use
EEG~10 mm, ~0.05 s, non-invasiveSaha 2021, Table 1. Coarse in space, fast in time.
MEG~5 mm, ~0.05 s, not portableSaha 2021, Table 1. Less distorted than EEG, costly.
ECoG~1 mm, ~0.003 sSaha 2021, Table 1. Needs the skull opened.
Intracortical~0.5 mm LFP, ~0.1 mm MUA, ~0.05 mm SUA, ~0.003 sSaha 2021, Table 1. Smallest patch, finest detail.
fMRI and fNIRS~1 s; ~1 mm and ~5 mmSaha 2021, Table 1. Blood is slow.
Skull conductivitybrain to skull 80 (classic); 24.8 ± 6.6 in vivo (5 patients)Head-model study (PMC3652134). Sets the EEG blur.
Wrong skull ratio2 to 16 mm error, noise-freeSame study. Ratios 15, 20, 25 and 80 tested.
31 vs 128 electrodes9.0 to 17.3 mm vs 2.0 to 11.8 mmSame study, 10 dB noise. Density helps.
Utah array10 × 10 needles, 400 µm apart, 1.5 mm longCampbell 1990; Normann 1999.
Neuropixels960 sites, 384 recorded at once, 10 mm shankJun 2017. Over 700 neurons from two probes in a mouse.
Stentrode190 days in sheep; 4 patients, 12 monthsOxley 2016; SWITCH 2023.
BrainGate safety14 adults, mean 872 days, 12,203 days in allRubin 2023.
Data ratechannels × Hz × bits / 864 × 250 × 24 / 8 = 48 kB per second.

Terms in one line each

TermMeaning
SpikeA pulse of about 1 ms that a neuron sends.
SUA, MUA, LFPOne neuron’s spikes; a few neurons’ spikes; the slow voltage of a patch.
ECoG, EEG, MEGVoltage on the cortex; voltage on the scalp; magnetic field outside the head.
BOLD, fMRI, fNIRSBlood-oxygen signal; read by magnet; read by near-infrared light.
InvasiveNeeds a cut or an implant. Rungs: scalp, vein, cortex surface, in cortex.
Volume conductionCurrent spreading through every conductive tissue.
Lead field LThe table of smears from sources to electrodes.
Forward, inverseSources to sensors (easy); sensors back to sources (many answers).
Sampling rule, aliasingSample at over twice the top frequency, or fast waves fold into false slow ones.

The checklist

The core as code

Four small functions carry the whole lesson: the blur that merges two sources, the lead field that smears sources into electrodes, the least-energy inverse, and the data rate. The last lines print the shapes you saw in Chapters 4 and 7.

pythonimport numpy as np

FW = {"neuron": 0.05, "ecog": 1.0, "meg": 5.0, "eeg": 10.0}   # mm, Saha 2021 Table 1

def report(x, gap, fw, c0=0.0):
    """Two equal point sources gap mm apart, seen through a bell-curve blur of width fw."""
    s = fw / 2.3548
    return np.exp(-(x - (c0 - gap / 2)) ** 2 / (2 * s * s)) + np.exp(-(x - (c0 + gap / 2)) ** 2 / (2 * s * s))

def lead_field(electrodes, grid, fw):
    """v = L @ s : column j is the smear that source j makes across the electrodes."""
    s = fw / 2.3548
    return np.exp(-(electrodes[:, None] - grid[None, :]) ** 2 / (2 * s * s))   # (M, K)

def min_norm(L, v, lam):
    """Least-energy sources that explain v : s_hat = L^T (L L^T + lam I)^-1 v."""
    return L.T @ np.linalg.solve(L @ L.T + lam * np.eye(len(v)), v)            # (K,)

def bytes_per_second(channels, hz, bits):
    return channels * hz * bits / 8

grid = np.arange(61.0); s = np.zeros(61); s[[18, 26, 43]] = 1
el = np.linspace(0, 60, 24)
L = lead_field(el, grid, fw=10.0)            # (24, 61)
v = L @ s + 0.03 * np.abs(L @ s).max() * np.random.default_rng(0).normal(size=24)
s_hat = min_norm(L, v, lam=1e-4 * (L @ L.T)[0, 0])
print(L.shape, v.shape, s_hat.shape, bytes_per_second(64, 250, 24))
x = np.linspace(-15, 15, 3001)
for name, fw in FW.items():
    y = report(x, 6.0, fw); print(name, "peaks" if y[1500] < y.max() * 0.98 else "one hump")

Where it sits in the field

This lesson is the middle of the series’ foundations. The lessons before it explain what a BCI is and what the neurons do. The lessons after it work on the signals: filtering, spikes, decoding, and then learned models of neural data. Reading the map first keeps you honest about what each later model is allowed to know. The Merge Labs and Sabi statements in this lesson are company claims, as of the dates cited. They belong to the open edge of the map, not to its measured core.

Keep going

Sources

We opened each source below ourselves. We used no slides. Company statements are labelled as claims in the chapters.

The takeaway. A brain sensor is a choice of seat. The seat fixes how finely, how fast, and at what cost the brain can be seen, and no model downstream can add what the seat threw away. Put the job in numbers, place it on the map, and read the trade before you choose a model.
A pitch promises single-neuron detail from a headband, with no surgery and no published numbers. What is the best first response?

Now press Teach and explain the map from memory: draw the space and time axes, place each sensor, say why the skull blurs EEG, and say what a dense cap can and cannot fix. Then go back to the two neighbours at the top, pick EEG, and find the gap where it stops telling them apart.

Brain-Computer Interfaces L03: The Recording Map
Saha et al. 2021 · Princeton NEU 502B · UIUC ECE 421 · Back to Gleams