> For the complete documentation index, see [llms.txt](https://boinc-ai.gitbook.io/optimum/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://boinc-ai.gitbook.io/optimum/habana/reference/distributed-runner.md).

# Distributed Runner

## DistributedRunner

#### class optimum.habana.distributed.DistributedRunner

[\<source>](https://github.com/huggingface/optimum-habana/blob/main/optimum/habana/distributed/distributed_runner.py#L32)

( command\_list: typing.List = \[] world\_size: int = 1 hostfile: typing.Union\[str, pathlib.Path] = None use\_mpi: bool = False use\_deepspeed: bool = False use\_env: bool = False map\_by: bool = 'socket' multi\_hls = None )

Set up training/inference hardware configurations and run distributed commands.

**create\_multi\_node\_setup**

[\<source>](https://github.com/huggingface/optimum-habana/blob/main/optimum/habana/distributed/distributed_runner.py#L184)

( )

Multi-node configuration setup for DeepSpeed.

**create\_single\_card\_setup**

[\<source>](https://github.com/huggingface/optimum-habana/blob/main/optimum/habana/distributed/distributed_runner.py#L145)

( use\_deepspeed = False )

Single-card setup.

**create\_single\_node\_setup**

[\<source>](https://github.com/huggingface/optimum-habana/blob/main/optimum/habana/distributed/distributed_runner.py#L173)

( )

Single-node multi-card configuration setup.

**create\_single\_node\_setup\_deepspeed**

[\<source>](https://github.com/huggingface/optimum-habana/blob/main/optimum/habana/distributed/distributed_runner.py#L166)

( )

Single-node multi-card configuration setup for DeepSpeed.

**create\_single\_node\_setup\_mpirun**

[\<source>](https://github.com/huggingface/optimum-habana/blob/main/optimum/habana/distributed/distributed_runner.py#L155)

( )

Single-node multi-card configuration setup for mpirun.

**process\_hostfile**

[\<source>](https://github.com/huggingface/optimum-habana/blob/main/optimum/habana/distributed/distributed_runner.py#L226)

( ) → str

Returns

str

address of the master node.

Returns the master address to use for multi-node runs with DeepSpeed. Directly inspired from <https://github.com/microsoft/DeepSpeed/blob/316c4a43e0802a979951ee17f735daf77ea9780f/deepspeed/autotuning/utils.py#L145>.

**run**

[\<source>](https://github.com/huggingface/optimum-habana/blob/main/optimum/habana/distributed/distributed_runner.py#L192)

( )

Runs the desired command with configuration specified by the user.