Skip to main content

calibrate_kl_divergence

Function calibrate_kl_divergence 

Source
pub fn calibrate_kl_divergence(hist: &TensorHistogram) -> QuantizationParams
Expand description

Compute quantization parameters using KL-divergence (entropy) calibration.

Finds the optimal symmetric clipping threshold that minimizes the KL divergence between the original float distribution and the quantized distribution. This is the standard TensorRT-style calibration approach.

The search expands outward from the histogram center (value 0), trying symmetric clip ranges [-T, T] and selecting the T that minimizes KL divergence.