Optimization

Optimization

ORTOptimizer

class optimum.onnxruntime.ORTOptimizer

<source>

( onnx_model_path: typing.List[os.PathLike]config: PretrainedConfigfrom_ortmodel: bool = False )

Handles the ONNX Runtime optimization process for models shared on boincai.com/models.

from_pretrained

<source>

( model_or_path: typing.Union[str, os.PathLike, optimum.onnxruntime.modeling_ort.ORTModel]file_names: typing.Optional[typing.List[str]] = None )

Parameters

• model_or_path (Union[str, os.PathLike, ORTModel]) — The path to a local directory hosting the model to optimize or an instance of an ORTModel to quantize. Can be either:

  • A path to a local directory containing the model to optimize.

  • An instance of ORTModel.

• file_names(Optional[List[str]], defaults to None) — The list of file names of the models to optimize.

get_fused_operators

<source>

( onnx_model_path: typing.Union[str, os.PathLike] )

Parameters

• onnx_model_path (Union[str, os.PathLike]) — Path of the ONNX model.

Computes the dictionary mapping the name of the fused operators to their number of apparition in the model.

get_nodes_number_difference

<source>

( onnx_model_path: typing.Union[str, os.PathLike]onnx_optimized_model_path: typing.Union[str, os.PathLike] )

Parameters

• onnx_model_path (Union[str, os.PathLike]) — Path of the ONNX model.

• onnx_optimized_model_path (Union[str, os.PathLike]) — Path of the optimized ONNX model.

Compute the difference in the number of nodes between the original and the optimized model.

get_operators_difference

<source>

( onnx_model_path: typing.Union[str, os.PathLike]onnx_optimized_model_path: typing.Union[str, os.PathLike] )

Parameters

• onnx_model_path (Union[str, os.PathLike]) — Path of the ONNX model.

• onnx_optimized_model_path (Union[str, os.PathLike]) — Path of the optimized ONNX model.

Compute the dictionary mapping the operators name to the difference in the number of corresponding nodes between the original and the optimized model.

optimize

<source>

( optimization_config: OptimizationConfigsave_dir: typing.Union[str, os.PathLike]file_suffix: typing.Optional[str] = 'optimized'use_external_data_format: typing.Optional[bool] = Noneone_external_file: bool = True )

Parameters

• optimization_config (OptimizationConfig) — The configuration containing the parameters related to optimization.

• save_dir (Union[str, os.PathLike]) — The path used to save the optimized model.

• file_suffix (str, defaults to "optimized") — The file suffix used to save the optimized model.

• use_external_data_format (Optional[bool], defaults to None) — Whether to use external data format to store model of size >= 2Gb. This argument is deprecated.

• one_external_file (bool, defaults to True) — When use_external_data_format=True, whether to save all tensors to one external file. If False, save each tensor to a file named with the tensor name.

Optimizes a model given the optimization specifications defined in optimization_config.