> For the complete documentation index, see [llms.txt](https://boinc-ai.gitbook.io/optimum/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://boinc-ai.gitbook.io/optimum/furiosa/reference/quantization.md).

# Quantization

## Quantization

### FuriosaAIQuantizer

#### class optimum.furiosa.FuriosaAIQuantizer

[\<source>](https://github.com/huggingface/optimum.furiosa/blob/vr_/optimum/furiosa/quantization.py#L91)

( model\_path: Path config: Optional = None )

Handles the FuriosaAI quantization process for models shared on huggingface.co/models.

**compute\_ranges**

[\<source>](https://github.com/huggingface/optimum.furiosa/blob/vr_/optimum/furiosa/quantization.py#L246)

( )

Computes the quantization ranges.

**fit**

[\<source>](https://github.com/huggingface/optimum.furiosa/blob/vr_/optimum/furiosa/quantization.py#L169)

( dataset: Dataset calibration\_config: CalibrationConfig batch\_size: int = 1 )

Parameters

* **dataset** (`Dataset`) — The dataset to use when performing the calibration step.
* **calibration\_config** ([\~CalibrationConfig](https://huggingface.co/docs/optimum.furiosa/pr_/en/package_reference/configuration#optimum.furiosa.CalibrationConfig)) — The configuration containing the parameters related to the calibration step.
* **batch\_size** (`int`, *optional*, defaults to 1) — The batch size to use when collecting the quantization ranges values.

Performs the calibration step and computes the quantization ranges.

**from\_pretrained**

[\<source>](https://github.com/huggingface/optimum.furiosa/blob/vr_/optimum/furiosa/quantization.py#L119)

( model\_or\_path: Union file\_name: Optional = None )

Parameters

* **model\_or\_path** (`Union[FuriosaAIModel, str, Path]`) — Can be either:
  * A path to a saved exported ONNX Intermediate Representation (IR) model, e.g., \`./my\_model\_directory/.
  * Or an `FuriosaAIModelModelForXX` class, e.g., `FuriosaAIModelModelForImageClassification`.
* **file\_name(`Optional[str]`,** *optional*) — Overwrites the default model file name from `"model.onnx"` to `file_name`. This allows you to load different model files from the same repository or directory.

Instantiates a `FuriosaAIQuantizer` from a model path.

**get\_calibration\_dataset**

[\<source>](https://github.com/huggingface/optimum.furiosa/blob/vr_/optimum/furiosa/quantization.py#L315)

( dataset\_name: str num\_samples: int = 100 dataset\_config\_name: Optional = None dataset\_split: Optional = None preprocess\_function: Optional = None preprocess\_batch: bool = True seed: int = 2016 use\_auth\_token: bool = False )

Parameters

* **dataset\_name** (`str`) — The dataset repository name on the Hugging Face Hub or path to a local directory containing data files to load to use for the calibration step.
* **num\_samples** (`int`, *optional*, defaults to 100) — The maximum number of samples composing the calibration dataset.
* **dataset\_config\_name** (`Optional[str]`, *optional*) — The name of the dataset configuration.
* **dataset\_split** (`Optional[str]`, *optional*) — Which split of the dataset to use to perform the calibration step.
* **preprocess\_function** (`Optional[Callable]`, *optional*) — Processing function to apply to each example after loading dataset.
* **preprocess\_batch** (`bool`, *optional*, defaults to `True`) — Whether the `preprocess_function` should be batched.
* **seed** (`int`, *optional*, defaults to 2016) — The random seed to use when shuffling the calibration dataset.
* **use\_auth\_token** (`bool`, *optional*, defaults to `False`) — Whether to use the token generated when running `transformers-cli login` (necessary for some datasets like ImageNet).

Creates the calibration `datasets.Dataset` to use for the post-training static quantization calibration step.

**partial\_fit**

[\<source>](https://github.com/huggingface/optimum.furiosa/blob/vr_/optimum/furiosa/quantization.py#L207)

( dataset: Dataset calibration\_config: CalibrationConfig batch\_size: int = 1 )

Parameters

* **dataset** (`Dataset`) — The dataset to use when performing the calibration step.
* **calibration\_config** (`CalibrationConfig`) — The configuration containing the parameters related to the calibration step.
* **batch\_size** (`int`, *optional*, defaults to 1) — The batch size to use when collecting the quantization ranges values.

Performs the calibration step and collects the quantization ranges without computing them.

**quantize**

[\<source>](https://github.com/huggingface/optimum.furiosa/blob/vr_/optimum/furiosa/quantization.py#L261)

( quantization\_config: QuantizationConfig save\_dir: Union file\_suffix: Optional = 'quantized' calibration\_tensors\_range: Optional = None )

Parameters

* **quantization\_config** (`QuantizationConfig`) — The configuration containing the parameters related to quantization.
* **save\_dir** (`Union[str, Path]`) — The directory where the quantized model should be saved.
* **file\_suffix** (`Optional[str]`, *optional*, defaults to `"quantized"`) — The file\_suffix used to save the quantized model.
* **calibration\_tensors\_range** (`Optional[Dict[NodeName, Tuple[float, float]]]`, *optional*) — The dictionary mapping the nodes name to their quantization ranges, used and required only when applying static quantization.

Quantizes a model given the optimization specifications defined in `quantization_config`.