Skip to main content

calibrate_minmax

Function calibrate_minmax 

Source
pub fn calibrate_minmax(
    hist: &TensorHistogram,
    symmetric: bool,
) -> QuantizationParams
Expand description

Compute quantization parameters using MinMax calibration.

For asymmetric (uint8): scale = (max - min) / 255, zero_point = round(-min / scale) For symmetric (int8): scale = max(|min|, |max|) / 127, zero_point = 0