# Quantization

## Quantization

### FuriosaAIQuantizer

#### class optimum.furiosa.FuriosaAIQuantizer

[\<source>](https://github.com/huggingface/optimum.furiosa/blob/vr_/optimum/furiosa/quantization.py#L91)

( model\_path: Path config: Optional = None )

Handles the FuriosaAI quantization process for models shared on huggingface.co/models.

**compute\_ranges**

[\<source>](https://github.com/huggingface/optimum.furiosa/blob/vr_/optimum/furiosa/quantization.py#L246)

( )

Computes the quantization ranges.

**fit**

[\<source>](https://github.com/huggingface/optimum.furiosa/blob/vr_/optimum/furiosa/quantization.py#L169)

( dataset: Dataset calibration\_config: CalibrationConfig batch\_size: int = 1 )

Parameters

* **dataset** (`Dataset`) — The dataset to use when performing the calibration step.
* **calibration\_config** ([\~CalibrationConfig](https://huggingface.co/docs/optimum.furiosa/pr_/en/package_reference/configuration#optimum.furiosa.CalibrationConfig)) — The configuration containing the parameters related to the calibration step.
* **batch\_size** (`int`, *optional*, defaults to 1) — The batch size to use when collecting the quantization ranges values.

Performs the calibration step and computes the quantization ranges.

**from\_pretrained**

[\<source>](https://github.com/huggingface/optimum.furiosa/blob/vr_/optimum/furiosa/quantization.py#L119)

( model\_or\_path: Union file\_name: Optional = None )

Parameters

* **model\_or\_path** (`Union[FuriosaAIModel, str, Path]`) — Can be either:
  * A path to a saved exported ONNX Intermediate Representation (IR) model, e.g., \`./my\_model\_directory/.
  * Or an `FuriosaAIModelModelForXX` class, e.g., `FuriosaAIModelModelForImageClassification`.
* **file\_name(`Optional[str]`,** *optional*) — Overwrites the default model file name from `"model.onnx"` to `file_name`. This allows you to load different model files from the same repository or directory.

Instantiates a `FuriosaAIQuantizer` from a model path.

**get\_calibration\_dataset**

[\<source>](https://github.com/huggingface/optimum.furiosa/blob/vr_/optimum/furiosa/quantization.py#L315)

( dataset\_name: str num\_samples: int = 100 dataset\_config\_name: Optional = None dataset\_split: Optional = None preprocess\_function: Optional = None preprocess\_batch: bool = True seed: int = 2016 use\_auth\_token: bool = False )

Parameters

* **dataset\_name** (`str`) — The dataset repository name on the Hugging Face Hub or path to a local directory containing data files to load to use for the calibration step.
* **num\_samples** (`int`, *optional*, defaults to 100) — The maximum number of samples composing the calibration dataset.
* **dataset\_config\_name** (`Optional[str]`, *optional*) — The name of the dataset configuration.
* **dataset\_split** (`Optional[str]`, *optional*) — Which split of the dataset to use to perform the calibration step.
* **preprocess\_function** (`Optional[Callable]`, *optional*) — Processing function to apply to each example after loading dataset.
* **preprocess\_batch** (`bool`, *optional*, defaults to `True`) — Whether the `preprocess_function` should be batched.
* **seed** (`int`, *optional*, defaults to 2016) — The random seed to use when shuffling the calibration dataset.
* **use\_auth\_token** (`bool`, *optional*, defaults to `False`) — Whether to use the token generated when running `transformers-cli login` (necessary for some datasets like ImageNet).

Creates the calibration `datasets.Dataset` to use for the post-training static quantization calibration step.

**partial\_fit**

[\<source>](https://github.com/huggingface/optimum.furiosa/blob/vr_/optimum/furiosa/quantization.py#L207)

( dataset: Dataset calibration\_config: CalibrationConfig batch\_size: int = 1 )

Parameters

* **dataset** (`Dataset`) — The dataset to use when performing the calibration step.
* **calibration\_config** (`CalibrationConfig`) — The configuration containing the parameters related to the calibration step.
* **batch\_size** (`int`, *optional*, defaults to 1) — The batch size to use when collecting the quantization ranges values.

Performs the calibration step and collects the quantization ranges without computing them.

**quantize**

[\<source>](https://github.com/huggingface/optimum.furiosa/blob/vr_/optimum/furiosa/quantization.py#L261)

( quantization\_config: QuantizationConfig save\_dir: Union file\_suffix: Optional = 'quantized' calibration\_tensors\_range: Optional = None )

Parameters

* **quantization\_config** (`QuantizationConfig`) — The configuration containing the parameters related to quantization.
* **save\_dir** (`Union[str, Path]`) — The directory where the quantized model should be saved.
* **file\_suffix** (`Optional[str]`, *optional*, defaults to `"quantized"`) — The file\_suffix used to save the quantized model.
* **calibration\_tensors\_range** (`Optional[Dict[NodeName, Tuple[float, float]]]`, *optional*) — The dictionary mapping the nodes name to their quantization ranges, used and required only when applying static quantization.

Quantizes a model given the optimization specifications defined in `quantization_config`.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://boinc-ai.gitbook.io/optimum/furiosa/reference/quantization.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.