Skip to main content

calibrate_percentile

Function calibrate_percentile 

Source
pub fn calibrate_percentile(
    hist: &TensorHistogram,
    percentile: f32,
    symmetric: bool,
) -> QuantizationParams
Expand description

Compute quantization parameters using Percentile calibration.

Clips to the given percentile (e.g. 99.99) instead of absolute min/max, reducing the impact of outliers.