> For the complete documentation index, see [llms.txt](https://boinc-ai.gitbook.io/accelerate/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://boinc-ai.gitbook.io/accelerate/reference/kwargs-handlers.md).

# Kwargs handlers

## Kwargs Handlers

The following objects can be passed to the main [Accelerator](https://huggingface.co/docs/accelerate/v0.24.0/en/package_reference/accelerator#accelerate.Accelerator) to customize how some PyTorch objects related to distributed training or mixed precision are created.

### AutocastKwargs

#### class accelerate.AutocastKwargs

[\<source>](https://github.com/huggingface/accelerate/blob/v0.24.0/src/accelerate/utils/dataclasses.py#L61)

( enabled: bool = Truecache\_enabled: bool = None )

Use this object in your [Accelerator](https://huggingface.co/docs/accelerate/v0.24.0/en/package_reference/accelerator#accelerate.Accelerator) to customize how `torch.autocast` behaves. Please refer to the documentation of this [context manager](https://pytorch.org/docs/stable/amp.html#torch.autocast) for more information on each argument.

Example:

Copied

```
from accelerate import Accelerator
from accelerate.utils import AutocastKwargs

kwargs = AutocastKwargs(cache_enabled=True)
accelerator = Accelerator(kwargs_handlers=[kwargs])
```

### DistributedDataParallelKwargs

#### class accelerate.DistributedDataParallelKwargs

[\<source>](https://github.com/huggingface/accelerate/blob/v0.24.0/src/accelerate/utils/dataclasses.py#L83)

( dim: int = 0broadcast\_buffers: bool = Truebucket\_cap\_mb: int = 25find\_unused\_parameters: bool = Falsecheck\_reduction: bool = Falsegradient\_as\_bucket\_view: bool = Falsestatic\_graph: bool = False )

Use this object in your [Accelerator](https://huggingface.co/docs/accelerate/v0.24.0/en/package_reference/accelerator#accelerate.Accelerator) to customize how your model is wrapped in a `torch.nn.parallel.DistributedDataParallel`. Please refer to the documentation of this [wrapper](https://pytorch.org/docs/stable/generated/torch.nn.parallel.DistributedDataParallel.html) for more information on each argument.

`gradient_as_bucket_view` is only available in PyTorch 1.7.0 and later versions.

`static_graph` is only available in PyTorch 1.11.0 and later versions.

Example:

Copied

```
from accelerate import Accelerator
from accelerate.utils import DistributedDataParallelKwargs

kwargs = DistributedDataParallelKwargs(find_unused_parameters=True)
accelerator = Accelerator(kwargs_handlers=[kwargs])
```

### FP8RecipeKwargs

#### class accelerate.utils.FP8RecipeKwargs

[\<source>](https://github.com/huggingface/accelerate/blob/v0.24.0/src/accelerate/utils/dataclasses.py#L173)

( margin: int = 0interval: int = 1fp8\_format: str = 'E4M3'amax\_history\_len: int = 1amax\_compute\_algo: str = 'most\_recent'override\_linear\_precision: typing.Tuple\[bool, bool, bool] = (False, False, False) )

Use this object in your [Accelerator](https://huggingface.co/docs/accelerate/v0.24.0/en/package_reference/accelerator#accelerate.Accelerator) to customize the initialization of the recipe for FP8 mixed precision training. Please refer to the documentation of this [class](https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/api/common.html#transformer_engine.common.recipe.DelayedScaling) for more information on each argument.

Copied

```
from accelerate import Accelerator
from accelerate.utils import FP8RecipeKwargs

kwargs = FP8RecipeKwargs(fp8_format="HYBRID")
accelerator = Accelerator(mixed_precision="fp8", kwargs_handlers=[kwargs])
```

### GradScalerKwargs

#### class accelerate.GradScalerKwargs

[\<source>](https://github.com/huggingface/accelerate/blob/v0.24.0/src/accelerate/utils/dataclasses.py#L119)

( init\_scale: float = 65536.0growth\_factor: float = 2.0backoff\_factor: float = 0.5growth\_interval: int = 2000enabled: bool = True )

Use this object in your [Accelerator](https://huggingface.co/docs/accelerate/v0.24.0/en/package_reference/accelerator#accelerate.Accelerator) to customize the behavior of mixed precision, specifically how the `torch.cuda.amp.GradScaler` used is created. Please refer to the documentation of this [scaler](https://pytorch.org/docs/stable/amp.html?highlight=gradscaler) for more information on each argument.

`GradScaler` is only available in PyTorch 1.5.0 and later versions.

Example:

Copied

```
from accelerate import Accelerator
from accelerate.utils import GradScalerKwargs

kwargs = GradScalerKwargs(backoff_filter=0.25)
accelerator = Accelerator(kwargs_handlers=[kwargs])
```

### InitProcessGroupKwargs

#### class accelerate.InitProcessGroupKwargs

[\<source>](https://github.com/huggingface/accelerate/blob/v0.24.0/src/accelerate/utils/dataclasses.py#L150)

( backend: typing.Optional\[str] = 'nccl'init\_method: typing.Optional\[str] = Nonetimeout: timedelta = datetime.timedelta(seconds=1800) )

Use this object in your [Accelerator](https://huggingface.co/docs/accelerate/v0.24.0/en/package_reference/accelerator#accelerate.Accelerator) to customize the initialization of the distributed processes. Please refer to the documentation of this [method](https://pytorch.org/docs/stable/distributed.html#torch.distributed.init_process_group) for more information on each argument.

Copied

```
from datetime import timedelta
from accelerate import Accelerator
from accelerate.utils import InitProcessGroupKwargs

kwargs = InitProcessGroupKwargs(timeout=timedelta(seconds=800))
accelerator = Accelerator(kwargs_handlers=[kwargs])
```


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://boinc-ai.gitbook.io/accelerate/reference/kwargs-handlers.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.