# Optimization

## Optimization

### Transformation

#### class optimum.fx.optimization.Transformation

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L89)

( )

Parameters

* **preserves\_computation** (`bool`, defaults to `False`) — Whether the transformation preserves the graph computation or not. If `True`, the original and the transformed graph should produce the same outputs.

A torch.fx graph transformation.

It must implement the [transform()](https://huggingface.co/docs/optimum/main/en/torch_fx/package_reference/optimization#optimum.fx.optimization.Transformation.transform) method, and be used as a callable.

**\_\_call\_\_**

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L112)

( graph\_module: GraphModulelint\_and\_recompile: bool = True ) → `torch.fx.GraphModule`

Parameters

* **graph\_module** (`torch.fx.GraphModule`) — The module to transform.
* **lint\_and\_recompile** (`bool`, defaults to `True`) — Whether the transformed module should be linted and recompiled. This can be set to `False` when chaining transformations together to perform this operation only once.

Returns

`torch.fx.GraphModule`

The transformed module.

**get\_transformed\_nodes**

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L165)

( graph\_module: GraphModule ) → `List[torch.fx.Node]`

Parameters

* **graph\_module** (`torch.fx.GraphModule`) — The graph\_module to get the nodes from.

Returns

`List[torch.fx.Node]`

Gives the list of nodes that were transformed by the transformation.

**mark\_as\_transformed**

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L141)

( node: Node )

Parameters

* **node** (`torch.fx.Node`) — The node to mark as transformed.

Marks a node as transformed by this transformation.

**transform**

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L99)

( graph\_module: GraphModule ) → `torch.fx.GraphModule`

Parameters

* **graph\_module** (`torch.fx.GraphModule`) — The module to transform.

Returns

`torch.fx.GraphModule`

The transformed module.

**transformed**

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L153)

( node: Node ) → `bool`

Parameters

* **node** (`torch.fx.Node`) — The node to check.

Returns

`bool`

Specifies whether the node was transformed by this transformation or not.

### Reversible transformation

#### class optimum.fx.optimization.ReversibleTransformation

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L180)

( )

Parameters

* **preserves\_computation** (`bool`, defaults to `False`) — Whether the transformation preserves the graph computation or not. If `True`, the original and the transformed graph should produce the same outputs.

A torch.fx graph transformation that is reversible.

It must implement the [transform()](https://huggingface.co/docs/optimum/main/en/torch_fx/package_reference/optimization#optimum.fx.optimization.Transformation.transform) and [reverse()](https://huggingface.co/docs/optimum/main/en/torch_fx/package_reference/optimization#optimum.fx.optimization.ReversibleTransformation.reverse) methods, and be used as a callable.

**\_\_call\_\_**

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L201)

( graph\_module: GraphModulelint\_and\_recompile: bool = Truereverse: bool = False ) → `torch.fx.GraphModule`

Parameters

* **graph\_module** (`torch.fx.GraphModule`) — The module to transform.
* **lint\_and\_recompile** (`bool`, defaults to `True`) — Whether the transformed module should be linted and recompiled. This can be set to `False` when chaining transformations together to perform this operation only once.
* **reverse** (`bool`, defaults to `False`) — If `True`, the reverse transformation is performed.

Returns

`torch.fx.GraphModule`

The transformed module.

**mark\_as\_restored**

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L226)

( node: Node )

Parameters

* **node** (`torch.fx.Node`) — The node to mark as restored.

Marks a node as restored back to its original state.

**reverse**

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L188)

( graph\_module: GraphModule ) → `torch.fx.GraphModule`

Parameters

* **graph\_module** (`torch.fx.GraphModule`) — The module to transform.

Returns

`torch.fx.GraphModule`

The reverse transformed module.

**optimum.fx.optimization.compose**

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L725)

( \*args: Transformationinplace: bool = True )

Parameters

* **args** ([Transformation](https://huggingface.co/docs/optimum/main/en/torch_fx/package_reference/optimization#optimum.fx.optimization.Transformation)) — The transformations to compose together.
* **inplace** (`bool`, defaults to `True`) — Whether the resulting transformation should be inplace, or create a new graph module.

Composes a list of transformations together.

Example:

Copied

```
>>> from transformers import BertModel
>>> from transformers.utils.fx import symbolic_trace
>>> from optimum.fx.optimization import ChangeTrueDivToMulByInverse, MergeLinears, compose

>>> model = BertModel.from_pretrained("bert-base-uncased")
>>> traced = symbolic_trace(
...     model,
...     input_names=["input_ids", "attention_mask", "token_type_ids"],
... )
>>> composition = compose(ChangeTrueDivToMulByInverse(), MergeLinears())
>>> transformed_model = composition(traced)
```

#### Transformations

#### class optimum.fx.optimization.MergeLinears

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L241)

( )

Parameters

* **preserves\_computation** (`bool`, defaults to `False`) — Whether the transformation preserves the graph computation or not. If `True`, the original and the transformed graph should produce the same outputs.

Transformation that merges linear layers that take the same input into one big linear layer.

Example:

Copied

```
>>> from transformers import BertModel
>>> from transformers.utils.fx import symbolic_trace
>>> from optimum.fx.optimization import MergeLinears

>>> model = BertModel.from_pretrained("bert-base-uncased")
>>> traced = symbolic_trace(
...     model,
...     input_names=["input_ids", "attention_mask", "token_type_ids"],
... )
>>> transformation = MergeLinears()
>>> transformed_model = transformation(traced)
>>> restored_model = transformation(transformed_model, reverse=True)
```

#### class optimum.fx.optimization.FuseBiasInLinear

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L397)

( )

Parameters

* **preserves\_computation** (`bool`, defaults to `False`) — Whether the transformation preserves the graph computation or not. If `True`, the original and the transformed graph should produce the same outputs.

Transformation that fuses the bias to the weight in torch.nn.Linear.

Example:

Copied

```
>>> from transformers import BertModel
>>> from transformers.utils.fx import symbolic_trace
>>> from optimum.fx.optimization import FuseBiasInLinear

>>> model = BertModel.from_pretrained("bert-base-uncased")
>>> traced = symbolic_trace(
...     model,
...     input_names=["input_ids", "attention_mask", "token_type_ids"],
... )
>>> transformation = FuseBiasInLinear()
>>> transformed_model = transformation(traced)
>>> restored_model = transformation(transformed_model, reverse=True)
```

#### class optimum.fx.optimization.ChangeTrueDivToMulByInverse

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L451)

( )

Parameters

* **preserves\_computation** (`bool`, defaults to `False`) — Whether the transformation preserves the graph computation or not. If `True`, the original and the transformed graph should produce the same outputs.

Transformation that changes truediv nodes to multiplication by the inverse nodes when the denominator is static. For example, that is sometimes the case for the scaling factor in attention layers.

Example:

Copied

```
>>> from transformers import BertModel
>>> from transformers.utils.fx import symbolic_trace
>>> from optimum.fx.optimization import ChangeTrueDivToMulByInverse

>>> model = BertModel.from_pretrained("bert-base-uncased")
>>> traced = symbolic_trace(
...     model,
...     input_names=["input_ids", "attention_mask", "token_type_ids"],
... )
>>> transformation = ChangeTrueDivToMulByInverse()
>>> transformed_model = transformation(traced)
>>> restored_model = transformation(transformed_model, reverse=True)
```

#### class optimum.fx.optimization.FuseBatchNorm2dInConv2d

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L482)

( )

Parameters

* **preserves\_computation** (`bool`, defaults to `False`) — Whether the transformation preserves the graph computation or not. If `True`, the original and the transformed graph should produce the same outputs.

Transformation that fuses `nn.BatchNorm2d` following `nn.Conv2d` into a single `nn.Conv2d`. The fusion will be done only if the convolution has the batch normalization as sole following node.

For example, fusion will not be done in the case

Copied

```
     Conv2d
     /   \
    /     \
ReLU   BatchNorm2d
```

Example:

Copied

```
>>> from transformers.utils.fx import symbolic_trace
>>> from transformers import AutoModelForImageClassification

>>> from optimum.fx.optimization import FuseBatchNorm2dInConv2d

>>> model = AutoModelForImageClassification.from_pretrained("microsoft/resnet-50")
>>> model.eval()
>>> traced_model = symbolic_trace(
...     model,
...     input_names=["pixel_values"],
...     disable_check=True
... )

>>> transformation = FuseBatchNorm2dInConv2d()
>>> transformed_model = transformation(traced_model)
```

#### class optimum.fx.optimization.FuseBatchNorm1dInLinear

[\<source>](https://github.com/huggingface/optimum/blob/main/optimum/fx/optimization/transformations.py#L565)

( )

Parameters

* **preserves\_computation** (`bool`, defaults to `False`) — Whether the transformation preserves the graph computation or not. If `True`, the original and the transformed graph should produce the same outputs.

Transformation that fuses `nn.BatchNorm1d` following or preceding `nn.Linear` into a single `nn.Linear`. The fusion will be done only if the linear layer has the batch normalization as sole following node, or the batch normalization has the linear layer as sole following node.

For example, fusion will not be done in the case

Copied

```
     Linear
     /   \
    /     \
ReLU   BatchNorm1d
```

Example:

Copied

```
>>> from transformers.utils.fx import symbolic_trace
>>> from transformers import AutoModel

>>> from optimum.fx.optimization import FuseBatchNorm1dInLinear

>>> model = AutoModel.from_pretrained("nvidia/groupvit-gcc-yfcc")
>>> model.eval()
>>> traced_model = symbolic_trace(
...     model,
...     input_names=["input_ids", "attention_mask", "pixel_values"],
...     disable_check=True
... )

>>> transformation = FuseBatchNorm1dInLinear()
>>> transformed_model = transformation(traced_model)
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://boinc-ai.gitbook.io/optimum/torch-fx/reference/optimization.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.