A sound, in a
handful of numbers
You can hear that a flute is brighter than a tuba, that noise is harsher than a sine, that a drum hits and a pad swells. This lesson teaches a machine to measure exactly those things — to crush a whole spectrum down to a few honest numbers it can compare, search, classify, and one day generate.
A spectrum is too much. Squeeze it.
By now you can take any sound and, with the DFT, see its spectrum — hundreds or thousands of magnitude numbers, one per frequency bin, refreshed many times a second. That picture is gorgeous. It is also far too much for a machine to reason about directly.
Suppose you want a computer to answer: "is this a string or a drum?", "find me sounds like this one," or "is a new note starting right now?" Handing it ten thousand raw bins per frame is like answering "is this person tall?" by mailing every atom in their body. You need a summary — a few numbers that capture what matters and throw away what doesn't.
Those numbers are called audio features, or descriptors. Each one is a single recipe that boils a whole spectrum down to one value. Get a good handful of them and a sound becomes a short list — a point in a small space — that machines can store, compare, and search at scale.
The big four spectral features — centroid (brightness), spread, flux (change), and rolloff — plus the time-domain zero-crossing rate, a one-paragraph honest intuition for MFCCs, a centroid worked out by hand, the code in JavaScript and Python, a live meter-lab you drive with synthesized sounds, and the bridge from describing sound to generating it.
Brightness is a center of mass.
Start with the one feature everybody reaches for first, because it maps so cleanly onto something your ear already knows: brightness. A piccolo is bright; a bass drum is dark. What is "bright," in numbers?
Here's the idea, and it's a borrowing from physics. Imagine the spectrum is a thin rod, and at each frequency you hang a little weight equal to that frequency's magnitude — how much energy lives there. Now ask: where does the rod balance? That balance point is the spectral centroid — the spectrum's center of mass along the frequency axis.
If most of the weight sits low (a sine, a tuba), the rod balances on the left: low centroid, dark sound. If energy is shoved up into the highs (a cymbal, a hiss), it balances on the right: high centroid, bright sound. The centroid is brightness, made of arithmetic.
Read it slowly and it's just a weighted average. The numerator is "frequency times how much of it there is," summed over every bin. The denominator is the total amount of stuff. Divide, and you get the average frequency, weighted by energy. That's exactly the balance point of the loaded rod.
Two details that matter in practice. First, fk is the real frequency of bin k, which from the DFT lesson is fk = k · fs / N. Second, we use magnitude |Xk| as the weight (sometimes magnitude-squared, the power) — never the raw complex value, because phase shouldn't move the balance point.
Mathematician: the centroid is the first moment of the magnitude distribution — E[f] if you treat |Xk| as an (unnormalized) probability over frequency. Musician: it's brightness — the single knob that separates a muted trumpet from an open one, a dull cymbal from a sizzling one. Artist: it's a marker that slides left and right along the spectrum as the timbre opens and closes — and in the lab below, it's the warm needle you'll watch chase the sound.
The centroid of a toy spectrum.
A formula stays abstract until you push real numbers through it. So take the smallest spectrum that's still interesting: just three bins. Say a tiny DFT gave us magnitudes at three frequencies —
| Bin | Frequency fk | Magnitude |Xk| | fk · |Xk| |
|---|---|---|---|
| 0 | 100 Hz | 3 | 300 |
| 1 | 200 Hz | 1 | 200 |
| 2 | 400 Hz | 1 | 400 |
Most of the energy (magnitude 3) sits at the lowest frequency, with smaller dribbles up higher. Intuition says the balance point should land low — closer to 100 than to 400. Let's confirm with the formula.
(100+200+400)/3 ≈ 233 Hz). The heavy low bin pulled the balance down — exactly as a center of mass should.Now imagine the same three bins but with the weights flipped — magnitude 1 at 100 Hz, 1 at 200 Hz, 3 at 400 Hz. The numerator becomes 100 + 200 + 1200 = 1500, divided by 5, giving a centroid of 300 Hz. Same frequencies, but the energy moved up, so the centroid climbed and the sound got brighter. You just computed brightness by hand, twice.
How wide, and how fast it changes.
The centroid tells you where the energy balances, but not how spread out it is. A pure sine and a bright-but-narrow band could share a centroid yet feel totally different — one is a laser, the other a smear. So the natural second feature is the spectral spread: the standard deviation of frequency about the centroid. It's the spectrum's variance, then square-rooted back into hertz.
If all the energy huddles at one frequency, every (fk − centroid) is near zero and the spread collapses toward 0 — a pure sine. If energy is smeared across the whole band, the distances are large and the spread balloons — that's white noise, the widest possible sound. Spread is, almost literally, "how noisy versus how tonal."
The third feature asks a completely different question: not "what does this frame look like?" but "how much did it just change?" That's spectral flux — the frame-to-frame difference of the spectrum. You take the spectrum now, subtract the spectrum a moment ago, and add up the changes.
Why keep only the increases (the half-wave rectification, written (·)+)? Because the thing we usually care about is onsets — moments when new energy bursts in: a drum hit, a plucked string, a consonant. When a note starts, lots of bins jump up at once and the flux spikes. When a note simply decays, energy is leaving, not arriving, and we don't want that to register as an event. Flux, rectified this way, is the classic onset detector — the heartbeat reader of a track.
Steady tones — a held sine, a sustained pad — have flux near zero: nothing is changing frame to frame. A skittering hi-hat pattern has flux that pulses with every hit. In the lab, flux is the blue needle that twitches when you switch sounds and pulses while the sweep moves.
Centroid → brightness
Energy balance point. Low = dark (sine, tuba). High = bright (cymbal, hiss).
Spread → width
Std-dev about the centroid. Near 0 = tonal (sine). Huge = noisy (white noise).
Flux → change
Frame-to-frame spectral jump. Spikes at onsets — the cue for "a new sound just started."
Rolloff → tilt
The frequency below which 85% of the energy lives. Another brightness-ish proxy (next chapter).
Two features you can read without a DFT.
Not every descriptor needs the Fourier transform. Two cheap, beloved ones come almost free.
The first is the zero-crossing rate (ZCR): how often the raw waveform crosses zero per second — how many times it flips sign from + to − and back. A low, smooth sine crosses zero only a handful of times per cycle; harsh, high-frequency, or noisy signals cross constantly. So ZCR is a rough, DFT-free brightness/noisiness proxy — and it's so cheap it shows up in voice-activity detectors and speech-vs-music splitters.
x = [+1, −1, +1, −1, +1].|sign(x[n]) − sign(x[n−1])| = 2 at each of the 4 steps → sum = 8.8 / (2·5) = 0.8 crossings per sample — extremely high. A buzzy, high-pitched sound. A slow sine over the same window would crash this number toward zero.The second is the spectral rolloff: the frequency below which some fixed fraction (commonly 85%) of the total spectral energy lives. Walk up the spectrum accumulating magnitude until you've collected 85% of the total — the frequency where you stop is the rolloff. A dark sound hits 85% early (low rolloff); a bright, hissy sound has energy strung out high, so you must climb far before you've collected 85% (high rolloff). It's another tilt-of-the-spectrum measure, complementary to the centroid.
Centroid, rolloff, and ZCR all sniff at "how high does this sound sit?" — but they fail differently. ZCR gets fooled by DC offset and low-frequency rumble; centroid gets pulled by a single loud high partial; rolloff ignores where the energy is, only how much is below a line. A classifier fed all three can triangulate what no single one nails. That redundancy-with-different-failure-modes is the whole game of feature engineering.
MFCCs, without the heavy math.
If you read one acronym in audio machine learning, it's MFCC — Mel-Frequency Cepstral Coefficients. For decades they were the features behind speech recognition and music tagging. Here's the honest one-paragraph intuition, no matrices required.
Start with the spectrum. Do three things to it. One: rebucket the bins onto the mel scale — a warped frequency axis that mirrors the ear, with fine resolution down low (where we hear pitch precisely) and coarse resolution up high (where we mostly hear "bright-ish"). Two: take the logarithm of each mel band's energy, because loudness is perceived in ratios, in decibels — the same log you met for pitch and amplitude. Now you have a log-mel spectrum: a perceptually honest picture of the spectral shape.
Three: run a DCT (a cousin of the DFT, a Fourier transform of the log-mel curve) and keep only the first ~13 numbers. This last step is the clever one: it captures the broad shape of the envelope — the smooth contour of which bands are loud — in a few coefficients, while discarding the fine pitch ripple. The first coefficient is overall loudness; the next few sketch the spectral tilt and bumps that distinguish an "ah" from an "ee," a violin from a clarinet. Thirteen numbers per frame, and you've described the timbre well enough to recognize a phoneme or a guitar.
MFCCs are just our descriptor philosophy taken to its logical end: spectrum → perceptual warp → compress to a few numbers. The centroid is one number that summarizes balance; MFCCs are a dozen that summarize shape. Both throw away most of the spectrum on purpose, keeping only what an ear — or a classifier — actually uses. And note the recursion: a DCT inside the feature is the DFT machine from Module 2, pointed at the log-mel curve instead of the waveform. The same idea, all the way down.
What do you do with a list of features once you have it? Three classic jobs. Classification: feed the feature vector to a model and ask "drum, string, or voice?" — it's learning regions in feature space. Similarity / search: two sounds are "alike" if their feature vectors are close (small distance), which is exactly how Freesound finds sounds like the one you uploaded, with descriptors computed by libraries such as Essentia (from the same Stanford/UPF group behind this course). Retrieval & tagging: the backbone of music-information-retrieval — auto-genre, mood, tempo, instrument.
Centroid, spread, and flux in code.
Every formula above is a couple of loops over the magnitude spectrum. Here's the whole battery — the exact math the lab below runs 60 times a second:
// mags[k] = |X[k]|, sr = sample rate, N = fft size function spectralFeatures(mags, prevMags, sr, N) { let sum = 0, sumF = 0; for (let k = 0; k < mags.length; k++) { const f = k * sr / N; // real frequency of bin k sum += mags[k]; sumF += f * mags[k]; // f · |X| — the weighted numerator } const centroid = sum > 0 ? sumF / sum : 0; // center of mass = brightness let sumVar = 0; for (let k = 0; k < mags.length; k++) { const f = k * sr / N, d = f - centroid; sumVar += d * d * mags[k]; // (f − centroid)² · |X| } const spread = sum > 0 ? Math.sqrt(sumVar / sum) : 0; let flux = 0; for (let k = 0; k < mags.length; k++) { const diff = mags[k] - prevMags[k]; if (diff > 0) flux += diff; // keep only positive change → onsets } return { centroid, spread, flux }; }
import numpy as np def spectral_features(mags, prev, sr, N): f = np.arange(len(mags)) * sr / N # bin → Hz s = mags.sum() centroid = float((f * mags).sum() / s) if s > 0 else 0.0 spread = float(np.sqrt(((f - centroid)**2 * mags).sum() / s)) if s > 0 else 0.0 flux = float(np.maximum(mags - prev, 0).sum()) # rectified change return centroid, spread, flux # zero-crossing rate, straight off the waveform — no DFT needed def zcr(x): return float(np.mean(np.abs(np.diff(np.sign(x)))) / 2) # in practice you'd lean on a library: Essentia / librosa # import librosa # librosa.feature.spectral_centroid(y=x, sr=sr) # librosa.feature.mfcc(y=x, sr=sr, n_mfcc=13)
Watch the numbers chase your ears.
Now make the abstractions move. Press hear it, then switch between a pure sine, a buzzy saw, white noise, and a filtered sweep — all built live in your browser, no files. The spectrum draws on top; the warm centroid marker slides where the energy balances; the three meters below report centroid, spread, and flux in real time.
Listen and read at the same time. The sine pins the centroid far left and the spread near zero — dark and pure. The noise shoves both to the right — harsh and wide. The saw lands in between, brighter than the sine because its harmonics carry energy upward. And the sweep walks its centroid back and forth across the dial, with the flux needle twitching as it moves. The math you derived is doing exactly what your ear says.
You can't easily store or compare a thousand spectrum bins, but you can store {centroid, spread, flux} — three numbers. Sine, saw, noise, and sweep each carve out a distinct corner of that little 3-D space. A classifier doesn't need to "hear"; it just checks which corner a sound falls in. That tiny space — built from the very features you just watched — is what makes audio searchable.
From describing sound to making it.
This is where the Signal track closes, and it closes by turning around. Every lesson so far has gone one way: sound → numbers. The synth lesson built waves from recipes; the DFT pulled recipes back out of waves; this lesson crushed those spectra into a handful of honest descriptors. Analysis, all the way down.
But descriptors are exactly the vocabulary of generation. A modern generative-audio model — a Lyria-class music model — doesn't dream in raw samples. It learns and works in a compressed, spectral-flavored representation: mel-spectrograms, learned audio tokens, latent codes. Those are sophisticated, learned descendants of the same move you made by hand here — perceptual spectrum → a small set of numbers. To generate, the model predicts those numbers and a decoder turns them back into sound. To condition ("make it brighter," "more percussive"), it nudges features that are kin to centroid and flux.
So the centroid you computed on a toy 3-bin spectrum and the mel-tokens inside a billion-parameter music model are the same idea at different scales: reduce a sound to a description a machine can manipulate. Learn to describe sound and you've learned the language generation speaks.
Reverbs is built toward one finale: a generative visual conducted by AI-generated music, over an ambient bed — a Lyria-class stream pouring through the same live analysis you've been running, blooming into art on screen. Trace the path you walked. Synths made the wave. The DFT made the spectrum. This lesson made the descriptors — the spectral language. The next step isn't a new tool; it's swapping the oscillator for a music model. The meters don't change. Only the source of the signal does.
That's the Signal track. You can analyze any sound — see its spectrum, summarize its timbre, detect its onsets — and you understand that those summaries are precisely what a generator builds from. Sound, math, and art were never three subjects. They were one signal, read three ways. Now you can read it.
Back to the Reverbs north star →A whole sound, honest, in three numbers — and those three numbers are the first words of the language that makes sound.
Serra et al., Audio Signal Processing for Music Applications — Module 9, Sound & Music Description (Stanford / UPF). · Essentia — the UPF feature-extraction library behind much of Freesound. · Freesound — descriptor-based audio search. · Peeters, A large set of audio features for sound description (CUIDADO/IRCAM, 2004). · MDN, AnalyserNode.getFloatFrequencyData — the live spectrum the meters read.