Per-channel quantization is a quantization scheme in which each output channel of a weight tensor receives its own scale factor (and optionally its own zero point), instead of one shared scale for the whole tensor. In Edge AI, it is the default way to compress convolutional and linear layers to 8-bit or 4-bit integers while keeping accuracy close to the floating-point original.
The idea is simple but decisive: neural network weights vary widely in magnitude from one filter to the next, so a single scale wastes precision on small channels. By fitting the integer grid to each channel’s actual range, per-channel quantization removes most of the accuracy loss caused by outlier channels at almost no runtime cost.
How per-channel quantization maps weights to integers
Per-channel quantization applies the standard affine mapping q = round(w / s) + z separately to every slice of a weight tensor along its output channel axis. For a convolution kernel with shape [out, in, h, w], there is one scale s per output filter; for a fully connected layer, one scale per output neuron (each row of the weight matrix).
Most deployments use symmetric quantization for weights, which fixes the zero point at 0 and sets s = max(|w|) / 127 for INT8. Symmetric weights simplify the integer kernel because the cross terms that involve a weight zero point disappear. Activations, by contrast, usually stay per-tensor with an asymmetric zero point, because their ranges change with every input.
The table below shows how the granularity choice changes the number of stored parameters and the precision available for each weight.
| Granularity | Scales stored | Precision per weight | Typical use |
|---|---|---|---|
| Per-tensor | One per tensor | Lowest when ranges differ | Activations, simple accelerators |
| Per-channel | One per output channel | High for most layers | INT8 weights in CNNs and transformers |
| Per-group | One per block of 32-128 weights | Highest of the three | 4-bit weight-only LLM quantization |
Why channel-wise scales preserve accuracy
The core problem that per-channel quantization solves is range mismatch between channels. In a trained network, one filter may hold weights within ±0.02 while another reaches ±1.5. With a single per-tensor scale set by the largest value, the small filter is mapped onto only two or three integer levels, which effectively erases its information.
Range mismatch is especially severe in depthwise separable convolutions, used heavily in MobileNet. Each depthwise filter sees only one input channel, so filter magnitudes can differ by two orders of magnitude. Per-tensor INT8 quantization of MobileNetV2 can lose several percentage points of top-1 accuracy or collapse entirely without corrective steps, while per-channel quantization typically keeps the drop below about one point.
Batch normalization folding makes the problem worse and the solution more valuable. When batch normalization parameters are merged into the preceding convolution before export, each output channel is multiplied by a different factor, which widens the spread of channel ranges. Per-channel scales absorb that spread naturally.
The comparison below summarizes the typical outcomes reported for common model families; exact numbers depend on the calibration data, the export toolchain and the target runtime.
| Model family | Per-tensor weights | Per-channel weights | Main reason |
|---|---|---|---|
| Standard ResNet | Small loss | Negligible loss | Moderate channel spread |
| MobileNet with depthwise layers | Large or severe loss | Small loss | Extreme filter range differences |
| Transformer linear layers | Moderate loss | Small loss | Uneven output neuron norms |
Runtime and hardware support
Per-channel weight scales add very little inference cost because they can be applied after the integer accumulation. An INT8 matrix multiply accumulates products in 32-bit integers, and the requantization step then multiplies each output channel by its combined scale, which is a vector instead of a scalar. The extra memory is one 32-bit value per output channel, usually far below 0.1% of the layer size.
Support is broad but not universal. The TensorFlow Lite integer specification requires per-axis symmetric weights for convolutions, ONNX expresses the scheme through the axis attribute of QuantizeLinear and DequantizeLinear, and PyTorch provides per-channel observers in its quantization workflows. Some fixed-function NPUs and older DSPs accept only per-tensor scales, so the edge inference runtime may fall back to slower paths or reject the model.
The table below lists the practical checks to perform before relying on per-channel weights on a given device.
| Check | What to verify | Consequence if missing |
|---|---|---|
| Operator coverage | Every quantized layer accepts per-axis scales | Layers fall back to CPU or float |
| Hardware delegate | NPU or DSP driver supports vector scales | Model partitioned across processors |
| Requantization path | Fixed-point multipliers per channel | Rounding errors or failed conversion |
| Model conversion tool | Exporter keeps the axis attribute | Silent conversion to per-tensor |
How it relates to other quantization techniques
Per-channel quantization is a granularity choice, not a full workflow, so it combines with other Edge AI compression methods rather than competing with them. It is used inside post-training quantization, inside quantization-aware training and inside most INT8 quantization pipelines for weights.
- Post-training quantization uses per-channel weights to reach acceptable INT8 accuracy without retraining, often after a short quantization calibration pass for activations.
- Quantization-aware training simulates per-channel rounding during fine-tuning, which recovers most of the remaining error at the cost of training time.
- Per-group quantization splits each channel further into blocks, which 4-bit quantization methods such as GPTQ and AWQ need because a single channel scale is too coarse at 16 integer levels.
- SmoothQuant addresses the opposite problem, outliers in activation channels, by moving scale from activations into weights, which then rely on per-channel scales to absorb the change.
Limitations and common mistakes
Per-channel quantization does not fix outliers within a single channel. When one weight in a row is far larger than the rest, the whole row still loses precision, and clipping, per-group scales or weight equalization are needed. The scheme also does nothing for activations, which often dominate the error in transformer models.
The most frequent practical failure is choosing the wrong axis. Scales must follow the output channel dimension; quantizing along the input dimension cannot be folded into the requantization step and produces either wrong results or an unsupported graph.
| Symptom | Likely cause | Fix |
|---|---|---|
| Accuracy still drops sharply | Activation outliers dominate | Better calibration or SmoothQuant |
| Model runs slower on the NPU | Delegate lacks per-axis support | Check operator support or use per-tensor |
| Converted model gives wrong outputs | Scales applied on the input axis | Quantize along the output channel axis |
| Tiny channels quantize to zero | Dead or nearly dead filters | Prune them or keep a minimum scale |
A final caution concerns reporting. Accuracy figures for quantized models are only comparable when the granularity is stated, because a per-channel result and a per-tensor result for the same bit width can differ substantially.
A non-technical view
Shrinking an AI model means storing its numbers with fewer digits. The trick is choosing the measuring scale. If one ruler must measure both an ant and a building, the ant ends up as zero. Per-channel quantization gives every group of numbers its own ruler, so small details are not rounded away.
Because each ruler is just one extra number, the model stays almost as small and fast. The result is a compact model that behaves almost like the original, which is exactly what phones and small devices need.
A concrete example
An engineering team building a smart doorbell wants to run a MobileNetV2-based person detector on a low-power microcontroller with an integer-only accelerator. The float model reaches about 92% detection accuracy on their test set, but it needs roughly 14 MB, while the device has about 4 MB of usable flash.
Their first INT8 conversion with per-tensor weights shrinks the model by roughly four times but drops accuracy to around 80%, because several depthwise filters collapse to a few integer values. Switching the exporter to symmetric per-channel weights and calibrating activations on 500 typical doorway images brings accuracy back to about 91%.
The extra scales add only a few kilobytes. The team confirms that the accelerator driver supports per-axis scales, so inference stays at under 60 milliseconds per frame, and the doorbell ships with on-device detection and no cloud dependency.