HAQ: Hardware-Aware
Automated Quantization with Mixed Precision

Chapter 1: The Key Insight

Picking bitwidths layer-by-layer under a hardware budget is a sequential decision problem with a delayed reward. That is exactly the shape of a reinforcement-learning problem. HAQ's move is to treat the quantizer as an agent walking down the network, one layer at a time, each step choosing that layer's bitwidth.

The loop in one breath

Why this is different from everything before it

Two design decisions carry the whole paper. First, the action is the bitwidth itself — the agent doesn't tweak training, it directly authors the quantization policy. Second, the environment is the hardware. Most quantization papers stop at "fewer bits = smaller model." HAQ closes the loop with the chip, so the policy it learns is co-designed with the silicon. The paper shows the learned policies for edge vs cloud accelerators are drastically different — and explains why using the roofline model (Chapter 8).

Everything that follows builds this loop piece by piece: how a weight actually becomes 4 bits (Ch 2), what the agent sees and does (Ch 3), how the chip talks back (Ch 4), the agent in motion (Ch 5–6), how budgets are enforced (Ch 7), and what the learned policies reveal (Ch 8).

Chapter 2: Background — How a Weight Becomes b Bits

Before the agent can choose a bitwidth, we need to know exactly what "quantize this layer to b bits" does to a number. HAQ uses plain linear (uniform) quantization because linearly-quantized models only need fixed-point arithmetic — the cheap thing hardware is built for.

Take a weight w in layer k. Two steps:

Reading every symbol: clamp(w, c) truncates w into the range [−c, c] (a hard ceiling and floor). s is the step size — the gap between adjacent representable values. With b bits you get 2^b levels; for signed values that's 2^b−1−1 positive steps on each side. Dividing by s, rounding, and multiplying back snaps w onto the nearest grid point.

Choosing the clip value c — the KL-divergence trick

Where do you put the ruler's ends? Too wide and most of your few ticks land in empty space; too narrow and you clip away large weights. HAQ picks c to make the quantized weight distribution look as much like the original as possible, by minimizing KL-divergence:

In words: D_KL measures how much information is lost when you replace the true weight histogram W_k with its quantized version. Sweep candidate clip values x, quantize, and keep the x that distorts the distribution least. Activations are handled identically except the range is [0, c] not [−c, c], because they're ReLU outputs and never negative.

Worked example: one weight, two bitwidths

Let w = 0.62 and clip c = 1.0.

The 2-bit ruler has one tick per side, so 0.62 rounds all the way to 1.0 — a 0.38 error. The 8-bit ruler nails it to 0.002. This is the lever the agent pulls: more bits buys precision but costs memory and compute. Use the widget below to feel it.

Chapter 3: What the Agent Sees and Does

The agent processes the network layer by layer, and for each layer it takes two steps: one to set the weight bitwidth, one to set the activation bitwidth. At each step it observes a 10-dimensional feature vector describing the current layer.

The observation O_k (10 numbers)

For a convolution layer:

Every symbol, plainly: k = layer index (where am I in the stack); c_in, c_out = input/output channel counts; s_kernel = kernel size; s_stride = stride; s_feat = input feature-map size; n_params = parameter count; i_dw = is-this-depthwise flag (1/0); i_w/a = is-this-the-weight-step-or-activation-step flag; a_k−1 = the action the agent took on the previous step (so it has memory of context). Fully-connected layers use the same vector with kernel→1, stride→0, and the depthwise flag→0. Every dimension is normalized to [0,1] so no single feature dominates.

The action: continuous, then rounded

A natural instinct is a discrete action — pick directly from {2,3,…,8}. HAQ deliberately avoids this. The reason is subtle and worth internalizing: discrete categories lose order. To a softmax, "2 bits" and "8 bits" are just two unrelated labels; it can't feel that 2 is "more aggressive" than 4 which is "more aggressive" than 8. So HAQ uses a continuous action a_k ∈ [0,1] and rounds it to a bitwidth:

With b_min=2, b_max=8 this maps the continuous knob onto 7 equal bins. Worked example: a_k=0.0 → round(1.5)=2; a_k=0.5 → round(2−0.5+0.5×7)=round(5.0)=5; a_k=1.0 → round(8.5)=8 (clamped). The continuous knob means a tiny nudge in policy is a tiny nudge in bitwidth — gradients flow smoothly, which is exactly what an actor-critic agent needs.

Chapter 4: The Chip Talks Back — Direct Feedback

This is the chapter that makes HAQ "hardware-aware." The agent could be rewarded with FLOPs or model size — both cheap to compute. HAQ refuses, and the reason is the heart of the paper.

The roofline model — why a layer's "shape" decides its bits

The key lens is operation intensity: arithmetic operations performed per byte of memory accessed (OPs/byte). The roofline model says a layer is either compute-bound (lots of math per byte — the chip's ALUs are the bottleneck) or memory-bound (little math per byte — moving data is the bottleneck).

This single number explains HAQ's most famous result. In MobileNet, depthwise convolutions have low operation intensity — they reuse data poorly, so they're memory-bound; their cost is dominated by moving activations. Pointwise (1×1) convolutions have high operation intensity — they're compute-bound. So if a chip is starved for memory bandwidth, cutting activation bits on the depthwise layers (less data to move) buys the most speed for the least accuracy loss. The agent discovers this on its own.

Chapter 5: SHOWCASE — The Bit-Allocation Explorer

This is HAQ's core figure (the per-layer policy bars of Figures 3–4) made interactive — and turned into the actual optimization the agent solves. You're looking at a small MobileNet-style stack: alternating depthwise (memory-bound, low OPs/byte) and pointwise (compute-bound) layers, plus a sensitive first and last layer.

Each layer has its own bitwidth slider. As you change bits, watch three things update live: the model's size, its latency on the selected chip, and the predicted accuracy drop. The "Run HAQ agent" button performs a greedy version of HAQ's search — it spends the budget where it's cheapest in accuracy, exactly the trade-off the RL agent learns.

Notice the policy the agent lands on is not "everyone to 4 bits." It's lumpy — fat tolerant layers go low, sensitive landmark layers stay high, and the memory-bound layers on the edge chip get squeezed on activations. That lumpiness is the value of mixed precision, and it's invisible to any uniform-bitwidth scheme.

Chapter 6: The Reward and the DDPG Agent

We have the loop's shape. Now the engine. Two questions remain: what reward drives the agent, and what algorithm trains it?

A reward that's only about accuracy

You might expect the reward to balance accuracy against latency. It doesn't — and the reason is elegant. HAQ already enforces the resource budget by clipping the action space (Chapter 7), so every policy the agent is allowed to propose already fits the budget. With the constraint handled structurally, the reward is freed to care about one thing:

Here acc_orig is the full-precision model's top-1 accuracy (on the training set), acc_quant is the quantized model's accuracy after a quick one-epoch finetune, and λ=0.1 scales it. The reward is the accuracy you gave up — almost always negative — and the agent's job is to make it as close to zero as possible while living inside the budget.

DDPG: actor-critic for a continuous knob

Because the action is continuous, HAQ uses Deep Deterministic Policy Gradient (DDPG) — an off-policy actor-critic method. The actor maps an observation to an action (the bitwidth knob, squashed by a sigmoid into [0,1]). The critic estimates the value of a state-action pair so the actor knows which direction to move. One step = one layer's weight-or-activation decision; one episode = a full pass over all layers.

HAQ tweaks the Bellman target slightly. The discount γ=1 (every layer's decision matters equally toward the final accuracy), and a baseline B — an exponential moving average of past rewards — is subtracted to cut variance:

Exploration is driven by adding truncated Gaussian noise to the actor's output, with noise std starting at 0.5 and decaying ×0.99 per episode — anneal from "try wild bitwidths" to "commit to the good policy."

# HAQ's core loop, distilled to the essential moves (pseudocode)
for episode in range(N_EPISODES):
    bits = []
    for k, layer in enumerate(net.layers):       # one episode = walk all layers
        for kind in ['weight', 'act']:           # two steps per layer
            O = observe(layer, kind, last_action)  # 10-dim state, normalized [0,1]
            a = actor(O) + noise()                 # continuous action in [0,1]
            b = round(B_MIN - 0.5 + a*(B_MAX - B_MIN + 1))  # -> bitwidth 2..8
            bits.append(b); last_action = a
    bits = clip_to_budget(bits, net, hw_sim, budget) # Ch.7: shrink until it fits
    qnet = quantize(net, bits)                       # linear quant, KL-chosen clip c
    acc  = finetune_one_epoch(qnet, subset=100)      # recover, fast
    R = 0.1 * (acc - acc_origin)                     # reward = accuracy delta
    ddpg.update(transitions, R, baseline)            # actor + critic step
    noise.decay(0.99)

Chapter 7: Living Within the Budget

How does HAQ guarantee the final model actually fits the latency/energy/size limit? Not through the reward — through the action space itself. This is the trick that lets the reward stay pure.

After the agent has proposed a bitwidth for every layer, HAQ measures the policy's total resource cost on the hardware simulator. If it exceeds the budget, HAQ sequentially decreases the bitwidth of each layer until the constraint is finally satisfied. The agent's proposal sets the relative priorities (which layers it wanted to protect); the clipping step enforces the absolute budget.

What this means for the data flow

Concretely: a policy is a length-2N vector of bitwidths (weights + activations per layer). The hardware simulator maps that vector → (latency, energy, size). If size > budget, decrement the layer that loses the least accuracy per bit saved, re-measure, repeat. The agent then sees, via the reward, that aggressive proposals get clipped into mediocre accuracy — so it learns to pre-allocate bits in a way that survives clipping. The constraint shapes the policy indirectly, through experience.

Chapter 8: Results — and Reading the Policy

HAQ was evaluated on ImageNet with MobileNet-V1/V2 against PACT (a strong uniform-bitwidth baseline) across three constraint types: latency, energy, and model size, on edge (Xilinx Zynq-7020) and cloud (Xilinx VU9P) accelerators.

The headline numbers

Concretely on the edge accelerator at the 4-bit-equivalent budget, MobileNet-V1 PACT scored 62.44% top-1; HAQ's flexible policy hit 67.40% at the same latency — a ~5-point gain for free, just by spending bits more wisely.

Reading the learned policy — the paper's real prize

The most cited part of HAQ isn't the accuracy table — it's the interpretation. The agent rediscovered hardware physics on its own:

Same network, opposite policies — driven entirely by the chip's roofline. The agent also learned the human heuristic for free: keep more bits in the first and last layers. No one told it; the reward did. Toggle the chip below and watch the policy flip.

Chapter 9: Connections & Cheat Sheet

HAQ sits at the intersection of three lineages, and pulls from each.

• AutoML / RL-for-design. HAQ is the quantization sibling of AMC (RL for channel pruning, same lab) and NAS (RL for architecture). The recurring idea: hand a chip's feedback to an RL agent and let it author a design choice humans used to hand-tune.
• Quantization. Builds directly on uniform linear quantization (PACT, Jacob et al.'s integer-arithmetic). HAQ's contribution is not how to quantize a layer but how many bits each layer gets.
• Hardware-software co-design. By putting BISMO/BitFusion simulators in the loop, HAQ is an early, clean demonstration that the optimal model depends on the silicon — a theme that now runs through all of efficient ML.

Cheat sheet — every equation, every symbol

The one-sentence takeaway

Where to go next

• TinyML: Pruning II & AMC — RL for channel pruning, HAQ's direct sibling.
• Efficient Architectures — MobileNet depthwise/pointwise blocks, the model HAQ quantizes.
• Off-Policy Actor-Critic / DDPG — the RL engine under the hood.
• LLM Inference — where mixed-precision quantization lives today (INT4/INT8 weights).