Quantization-aware training

Also known as: QAT

On this page 7

Quantization-aware training (QAT) is a model compression method that simulates low-precision arithmetic during training or fine-tuning, so that a neural network learns weights and activations that survive conversion to integer formats such as INT8 or INT4. In Edge AI, where models must run on phones, microcontrollers and neural processing units with tight memory and power budgets, QAT is the standard technique for recovering accuracy that simpler quantization loses.

The core idea is that a network trained in 32-bit floating point has never experienced rounding error. QAT exposes the model to that error while gradients can still adapt the parameters, which typically keeps an INT8 model within a fraction of a percentage point of its floating-point baseline, and makes 4-bit and lower precision practical for many architectures.

How quantization-aware training simulates integer inference

QAT works by inserting fake quantization nodes into the computational graph. Each node takes a floating-point tensor, maps it to the nearest representable integer level using a scale and zero point, clamps it to the integer range, and immediately maps it back to floating point. The forward pass therefore produces exactly the rounding and clipping errors that the deployed integer kernel will produce, while all arithmetic still runs in floating point on the training hardware.

Rounding has a gradient of zero almost everywhere, so ordinary backpropagation would stall. QAT solves this with the straight-through estimator (STE), which treats the rounding step as an identity function in the backward pass and passes gradients through unchanged, usually masked to zero outside the clipping range. The optimizer then nudges latent full-precision weights so that their quantized versions perform well, rather than optimizing weights that will later be distorted.

The following table lists the main components that a QAT pipeline adds to a standard training loop.

ComponentRole during trainingEffect at deployment
Fake quantizer on weightsRounds weights to the target bit widthWeights match the integer kernel exactly
Fake quantizer on activationsRounds layer outputs using observed rangesActivation ranges are fixed and calibrated
Range observersTrack minimum, maximum or moving averagesProvide final scale and zero point values
Straight-through estimatorApproximates the rounding gradientNot present; only the trained weights remain
Batch normalization foldingMerges normalization into convolution weightsRemoves a separate operator from the graph
Building blocks of a quantization-aware training pipeline

Precision targets and quantization granularity

QAT is configured by the bit width, the granularity of the scale factors and the choice between symmetric and asymmetric mapping. Lower bit widths shrink the model footprint and energy per operation, but every halving of precision makes the training problem harder and increases the accuracy gap that QAT must close.

The table below summarizes typical targets and what QAT usually achieves at each level; exact results depend heavily on architecture and task.

Target precisionSize versus FP32Typical accuracy outcomeCommon hardware
INT8About 4x smallerNear-lossless for most convolutional networksMobile NPUs, DSPs, microcontrollers
INT4 weights, INT8 activationsAbout 6-8x smallerSmall loss, recoverable with QATRecent NPUs and GPU kernels
INT4 throughoutAbout 8x smallerNoticeable loss without careful tuningSpecialized accelerators
Ternary or binary16-32x smallerLarge loss on complex tasksResearch and ultra-low-power chips
Typical precision targets for quantization-aware training

Per-channel quantization assigns a separate scale to each output channel of a weight tensor and is the default for convolution weights, because channel ranges can differ by orders of magnitude. Per-tensor scales remain common for activations because most integer kernels require them. Some frameworks also learn the scales themselves, as in learned step size quantization, which often improves results at 4 bits and below.

Quantization-aware training compared with post-training quantization

The main alternative is post-training quantization (PTQ), which converts an already trained model using a small calibration set and no gradient updates. PTQ is fast and needs no training pipeline, while QAT costs extra compute but adapts the weights to the rounding error. Related methods such as GPTQ, AWQ and SmoothQuant are advanced PTQ techniques that narrow this gap, particularly for large language models.

This comparison shows where each approach fits.

CriterionQuantization-aware trainingPost-training quantizationDynamic quantization
Training data neededLabeled training set or large subsetA few hundred calibration samplesNone
Compute costFine-tuning for several epochsMinutesNone before deployment
Accuracy at INT8Best, usually near baselineGood for many modelsGood for linear-heavy models
Accuracy at 4 bitsStrongest option for small modelsOften degrades without special methodsRarely used
Activation rangesLearned and fixedCalibrated and fixedComputed at run time
QAT versus post-training approaches

QAT also interacts with other compression steps. It is often combined with structured pruning or weight clustering, and the usual order is to prune first, then quantize with QAT so that the network recovers from both distortions in a single fine-tuning stage.

Applying QAT in an edge deployment workflow

In practice, QAT almost always starts from a converged floating-point model rather than from random initialization. Engineers first measure PTQ accuracy; only if the drop exceeds the product tolerance do they invest in QAT. A typical fine-tuning run lasts about 5-15 percent of the original training schedule with a learning rate one or two orders of magnitude lower.

  1. Train or obtain the floating-point baseline and record its accuracy.
  2. Fuse convolution, batch normalization and activation layers to match the target runtime.
  3. Insert fake quantizers configured for the target bit widths and granularity.
  4. Fine-tune with a reduced learning rate, then freeze observers and batch statistics for the final epochs.
  5. Convert to a true integer model and verify accuracy with the edge inference runtime itself.

The final step matters because the simulated graph must match the deployed kernels. If the hardware delegate fuses operators differently or uses another rounding mode, the trained model can lose accuracy that the training graph never showed. Matching the training simulation to the runtime is the most important configuration decision in QAT.

Limitations and common mistakes

QAT is effective but not free. It requires access to training data and a working training pipeline, which is often unavailable for third-party or very large models, and it adds memory overhead because fake quantizers store extra tensors. For billion-parameter language models, full QAT is usually too expensive, so practitioners prefer weight-only PTQ methods.

The table below lists frequent problems and how experienced teams address them.

SymptomLikely causeRemedy
Accuracy collapses earlyQuantizers enabled on an untrained modelStart from a converged FP32 checkpoint
Unstable loss late in trainingBatch statistics still updatingFreeze batch normalization near the end
Good simulated, poor deployed accuracyGraph does not match runtime fusionAlign fusion and rounding with the target kernel
Large error in one layerOutliers stretch the activation rangeUse per-channel scales or keep that layer in higher precision
Common QAT failure modes

A frequent misconception is that QAT always beats PTQ. For large, overparameterized models at INT8, a well-calibrated PTQ model often matches QAT within noise, so the extra training cost brings little benefit. QAT pays off mainly for compact architectures such as MobileNet, for aggressive bit widths, and for models where the first and last layers are sensitive.

Put simply

Imagine training a musician on a full concert piano and then handing them a small keyboard with fewer keys. Their first performance would sound off, because some notes no longer exist. Quantization-aware training lets the musician rehearse on the small keyboard before the concert, so they learn to play well with the notes that are actually available.

For AI models, the small keyboard is low-precision integer math, which phones and small chips handle quickly and cheaply. By practicing with that limitation during training, the model ends up small and fast without losing much accuracy.

In practice

A team building a smart doorbell needs a person detector that runs on a low-power microcontroller with 512 KB of RAM and an integer-only accelerator. Their floating-point model, a compact MobileNet variant, reaches 91.8 percent accuracy on their validation set. After post-training quantization to INT8, accuracy falls to 87.9 percent, which produces too many missed visitors at night.

The engineers fine-tune the model with QAT for eight epochs, using per-channel weight scales, frozen batch normalization for the last two epochs and the same operator fusion as the device runtime. The converted INT8 model reaches 91.2 percent accuracy, occupies about one quarter of the original storage, and runs within the device power budget. QAT recovered most of the lost accuracy without changing the architecture or the hardware.