Two tedious jobs — designing a network and choosing how many bits each layer should use — folded into one evolutionary search that hunts the accuracy-vs-size frontier on a single GPU.
You trained a beautiful image classifier. It hits 76% on ImageNet. Now your boss says: "Ship it on the phone." Suddenly two completely separate headaches land on your desk at once.
Headache one — the architecture. Which network shape is best? How many layers, which operations in each block, how wide? Hand-designing this is the job that Neural Architecture Search (NAS) automates — let an algorithm try thousands of architectures and keep the winners.
Headache two — the bits. A 32-bit float weight costs 4 bytes. A ResNet-50 stores ~100 MB of them. To fit on a phone you must quantize: store each weight in fewer bits (8, 4, even 2). But how many bits? Use too few and accuracy collapses; use too many and the model stays fat.
The standard quantization recipe of 2018 was blunt: pick one precision — say 8 bits — and apply it to every layer. But layers are not equally fragile. An early convolution feeding the whole network might be sensitive; a fat, redundant late layer might shrug off aggressive compression. Treating them identically leaves megabytes (and accuracy) on the table.
Below is a toy network. Each bar is a layer; its height is how many parameters it holds. Drag the global bit-width slider and watch the total model size. The point: one knob for everyone is a coarse instrument.
Each bar = a layer's parameter count. The dashed line is a 4 MB phone budget. Move the slider: every layer drops to the same precision at once.
Notice you cannot get under budget without either crushing the sensitive first layer or leaving a redundant one over-precise. You need a per-layer knob — and a way to discover the right setting automatically. That is JASQ.
JASQ's move is almost embarrassingly simple to state. The authors noticed that choosing a quantization bit for a layer looks exactly like choosing an operation (kernel size) for that layer. Both are discrete picks from a small menu, made per layer. So why run two searches?
In a normal NAS, a candidate is described by a string specifying the architecture A. JASQ extends that string with a quantization policy P — one bit-width per cell. Together they form a full model Θ = {A, P}.
Maintain a population of such models. Repeatedly: sample a few, keep the best one as a parent, mutate it (flip one operation or one bit-width), train the child briefly, measure its accuracy and size, score it with a multi-objective fitness that rewards accuracy but punishes blowing the size budget, and let it compete its way into the population while the worst sampled model is evicted. Run this on one GPU for three days. Out comes a small, accurate, mixed-precision network.
Two special cases fall out for free. Freeze A to a hand-designed network (ResNet, MobileNet) and you have an automatic mixed-precision quantizer for existing models. Search both and you get JASQNet — a from-scratch tiny network. We will build each piece in the chapters ahead.
Before we search over bit-widths, we need to know exactly what "quantize to b bits" means at the level of a single weight. Manufacture the need first: a float32 weight can be any of ~4 billion values. With b bits you get only 2b allowed values. Quantization is snapping each weight to the nearest allowed value.
Weights live in some arbitrary range. The paper first applies a linear scaling function that maps them into the unit interval (μ is the mean of the chunk, ν its range):
Here w is one weight, μ (mu) is the average weight in its chunk — think of it as the "center" — and ν (nu) is the spread, the largest minus the smallest. Subtracting the center and dividing by the spread squeezes everything into a clean [0, 1] ruler. We remember μ and ν so we can undo this later.
Now divide [0, 1] into 2b evenly spaced points and round each normalized weight to the closest one. The quantizer is:
Read it slowly. wi·2b stretches the [0,1] ruler so the grid points become integers. ⌊·⌋ drops to the integer below. ξi (xi) is the rounding bit — it equals 1 if the leftover fraction is above 0.5 (round up) and 0 otherwise (round down). Dividing by 2b puts us back on the [0,1] ruler, now snapped to a grid point.
Finally apply the inverse scaling L−1 (multiply by ν, add μ) to return to the original range. The complete round trip the paper writes as one line:
where ŵ (w-hat) is the quantized weight. The gap between w and ŵ is the quantization error — the price you pay for fewer bits.
Below is one real weight distribution (a bell curve). The horizontal ticks are the 2b allowed values. Drag the bits slider down and watch the grid get coarse — weights snap to fewer and fewer rungs, and the red error bars grow.
Blue curve = the weight distribution. Vertical ticks = allowed quantized values. Red segments = how far sample weights must jump to reach the nearest tick (the error).
Take one weight w = 0.62 in a bucket with μ = 0.5, ν = 0.4, quantizing to b = 2 bits (4 levels).
# Step 1: normalize L = (0.62 - 0.5) / 0.4 # = 0.30 # Step 2: snap to nearest of 2^2 = 4 levels scaled = 0.30 * 4 # = 1.20 floor = 1 # drop to integer below xi = 1 if (1.20 - 1) > 0.5 else 0 # 0.20 < 0.5 -> xi = 0 Q = (floor + xi) / 4 # = 1/4 = 0.25 (snapped on [0,1] ruler) # Step 3: un-normalize w_hat = 0.25 * 0.4 + 0.5 # = 0.60 # original 0.62 -> quantized 0.60. Error = 0.02.
With only 4 levels, 0.62 became 0.60. Coarse — but the storage dropped from 32 bits to 2.
Now combine the per-weight quantizer of Chapter 2 with the per-layer idea of Chapter 0. Instead of one global b, give each cell its own bit-width. A network with k cells gets a policy P = {b1, b2, ..., bk}.
For one weight vector of N weights quantized to b bits with bucket size k, full precision needs f·N bits (with f = 32). The quantized version needs b·N bits for the weights plus 2·(f·N)/k bits for the two scaling numbers per bucket. The compression ratio is:
Every symbol: k = bucket size (how many weights share one μ,ν pair), f = 32 (the float bit-width), b = the chosen bits. The 2·f term is the bucketing overhead — two 32-bit scalars per bucket. Big buckets (large k) amortize that overhead but risk outlier damage; small buckets are safe but cost more. Yet another knob the paper tunes.
Here is the same toy network from Chapter 0, but now every layer has its own bit slider. Your goal: get total size under the 4 MB dashed line while keeping the "accuracy estimate" high. Notice you can leave the fragile first layer at 16 bits and crush the fat redundant layers to 4 — something a single global knob could never do.
Each slider sets one layer's bits. The orange marker on each bar is its current precision. Sensitivity (the small label) tells you which layers punish low bits. Beat the budget without tanking accuracy.
We have candidates. We can quantize them. Now we need a single number that says "this candidate is good" — one that balances two things we both want but that fight each other: high accuracy and small size. This is a multi-objective problem, and the paper solves it with a clever penalty function.
If fitness were "accuracy alone," the search would always pick the biggest, fattest, full-precision model — defeating the entire point. If fitness were "smallest size alone," it would pick a 1-bit network that classifies nothing. You need a function that says: shrink as much as you like until you hit the target size; past that, accuracy is worthless if you exceed budget.
JASQ maximizes:
Symbols: α(Θ) is the model's validation accuracy (0–1, higher better). S(Θ) is its size in MB. TS (the "target size," e.g. 4 MB) is your budget. γ (gamma) is an exponent that flips based on whether you are under or over budget:
If the model is within budget, γ = 0, so (S/TS)0 = 1 and F = α. The size term vanishes — fitness is just accuracy. Below budget, the search only cares about being accurate.
If the model is over budget, γ = −1, so F = α · (TS/S). Since S > TS, the ratio is less than 1 and shrinks as the model gets fatter. Past budget, fitness is sharply penalized — the bigger the overrun, the harder the punishment.
Below is the fitness surface. Move the budget TS and an example model's size. Watch fitness sit flat at "just accuracy" while under budget, then fall off a cliff the instant size crosses the line.
Curve = F vs model size for a fixed accuracy. Left of the budget line, F is flat (=accuracy). Cross the line and F plunges. The orange dot is the current model.
How big is the haystack JASQ searches? Knowing this makes the efficiency claim ("1 GPU, 3 days") land properly. The space splits in two: S = {SA, SP} — architectures and quantization policies.
JASQ reuses the well-known NASNet cell-wise search space. Instead of designing a whole network, you design two small reusable modules and stack them:
Each cell is a small directed graph of combinations. One combination picks two inputs and applies one operation to each, then adds the results: Cj = {i1, i2, o1, o2}. The operation menu:
On CIFAR-10 the stacked network has k = 3N + 2 cells (with N = 6 that is 20 cells). Each cell gets one bit-width from a menu of three: 4, 8, or 16 bits. So the quantization policy is a length-20 string over a 3-symbol alphabet.
That is 3.5 billion quantization policies — for a single fixed architecture. Multiply by the (already astronomical) architecture space and the joint space is 3.5 × 109 times larger than searching architectures alone.
Below: build a candidate string yourself. Click cells to cycle their bit-width and watch the policy string and resulting model size update. This is the object the evolutionary search mutates.
Each block is one cell. Click to cycle 4→8→16 bits. The encoded policy string and total size update live. This is exactly what one "individual" in the population looks like.
A 3.5-billion-times-bigger haystack rules out brute force. JASQ uses an evolutionary algorithm — specifically tournament selection — the same family AmoebaNet showed could match RL-based NAS with no learned controller at all.
Picture a population of, say, 64 candidate models. Each iteration is a small tournament:
A child differs from its parent by exactly one gene. Change one operation, or reset one cell's bits. This makes the search a careful walk over the frontier rather than a wild jump — small steps let the population accumulate good combinations gradually, the way real evolution tinkers rather than redesigns.
import random def tournament_evolve(pop, n_epochs, sample_size): # pop: list of (model, fitness) individuals, already evaluated for _ in range(n_epochs): sample = random.sample(pop, sample_size) # step 1: tournament best = max(sample, key=lambda ind: ind.fitness) # step 2 worst = min(sample, key=lambda ind: ind.fitness) child = mutate(best) # step 3: flip ONE gene train(child, D_train) # step 4: evaluate quantize(child, child.policy) child.acc = test(child, D_val) child.size = model_size_mb(child) child.fitness = F(child.acc, child.size, T_S) # Ch.4 fitness pop.append(child) # step 5: push pop.remove(worst) # step 5: pop the loser return pop def mutate(parent): child = parent.copy() if random.random() < 0.5: # mutate architecture... slot = random.choice(child.arch_slots) slot.op = random.choice(OPS) # {sep3,sep5,avg,max,id,zero} else: # ...or quantization policy cell = random.randrange(len(child.policy)) child.policy[cell] = random.choice([4, 8, 16]) return child
Time to put every piece together. Below is a live JASQ evolutionary search over a fixed 8-cell network's quantization policy — the "freeze the architecture" mode from the paper. The population evolves bit-width strings, scored by the Chapter 4 fitness against a size budget you control.
Set the budget, then Run. The population migrates left (smaller) and up (more accurate) until it presses against the budget line — the Pareto frontier the paper hunts.
No quiz here — the simulation is the test. If you can predict which way the cloud moves when you change the budget, you understand JASQ.
Does it actually work? Two experiments. First, freeze the architecture and just search bit-widths for existing networks. Second, search both from scratch.
For ResNets, DenseNets and mobile nets on ImageNet, JASQ's searched mixed-precision policy was both smaller and more accurate than the uniform 8-bit version — and sometimes more accurate than the original float model. The surprising part: compression acting as mild regularization can raise accuracy.
| Network | JASQ Acc / Size | 8-bit Acc / Size | Float Acc / Size |
|---|---|---|---|
| ResNet-18 | 70.02% / 7.2 MB | 69.64% / 11.5 MB | 69.76% / 46.8 MB |
| ResNet-50 | 76.39% / 14.9 MB | 76.15% / 24.7 MB | 76.13% / 102 MB |
| MobileNet-v1 | 70.59% / 4.1 MB | 68.77% / 4.1 MB | 69.57% / 16.9 MB |
| MobileNet-v2 | 72.19% / 4.3 MB | 68.06% / 3.5 MB | 71.81% / 14.0 MB |
Searching architecture and bits jointly on CIFAR-10 (1 GPU, 3 days) produced two models by sliding the budget TS:
| Model | CIFAR-10 Error | Size | Search cost |
|---|---|---|---|
| NASNet-A | 2.65% | 13.2 MB | ~1800 GPU-days |
| AmoebaNet-B | 2.55% | 11.2 MB | ~3150 GPU-days |
| JASQNet (TS=3MB) | 2.90% | 2.5 MB | 3 GPU-days |
| JASQNet-Small (TS=1MB) | 2.97% | 0.9 MB | 3 GPU-days |
JASQNet is within ~0.3% error of NASNet-A while being 5× smaller and using ~600× less search compute. JASQNet-Small matches SqueezeNet's footprint but crushes its accuracy.
Most NAS searches on a small, cheap proxy network then scales up. JASQ found this hurt: searching at the real width/depth (F=36, N=6) converged faster and to higher fitness than the small proxy (F=16). For joint search, the proxy's behavior apparently diverges enough from the full model that its rankings mislead.
JASQ sits at the crossroads of two big movements: automated architecture design and model compression. Here is how it relates to its neighbors and what came after.
| Method | Searches | How |
|---|---|---|
| NASNet / AmoebaNet | Architecture only | RL controller / evolution, ~1000s GPU-days |
| DARTS | Architecture only | Differentiable (gradient) relaxation of discrete choices |
| Fixed-bit quantization | Nothing — uniform b | Apply one precision to all layers |
| JASQ | Architecture + per-layer bits | Multi-objective evolution, ~3 GPU-days |
| HAQ (later) | Per-layer bits | RL with hardware-in-the-loop latency feedback |
| Symbol / Eq. | Meaning |
|---|---|
| Θ = {A, P} | A candidate model: architecture A + quantization policy P |
| P = {b1...bk} | One bit-width per cell; each b ∈ {4, 8, 16} |
| ŵ = L−1(Q(L(w),b)) | Quantize: normalize → snap to 2b levels → un-normalize |
| L(w) = (w−μ)/ν | Linear scaling into [0,1]; per-bucket μ,ν (bucket size k) |
| ratio = kf / (kb + 2f) | Compression ratio; 2f = bucketing overhead, f=32 |
| F = α·(S/TS)γ | Fitness; γ=0 if S≤TS (F=α), else γ=−1 (penalty) |
| k = 3N+2 | Cell count; N=6 → 20 cells on CIFAR-10 |
| #SP = 320 | 3.5×109 quantization policies per architecture |
If you remember one thing: a layer's bit-width is just another architectural hyperparameter, no different from its kernel size — so search it the same way, and let a single size-aware fitness function dial you anywhere on the accuracy-vs-megabytes frontier.
Build the surrounding intuition with these Gleams:
"What I cannot create, I do not understand." — Richard Feynman