Installation
Installation and Configuration
Before you start, you will need to setup your environment, install the appropriate packages, and configure ๐ Accelerate. ๐ Accelerate is tested on Python 3.8+.
Installing ๐ Accelerate
๐ Accelerate is available on pypi and conda, as well as on GitHub. Details to install from each are below:
pip
To install ๐ Accelerate from pypi, perform:
Copied
pip install accelerate
conda
๐ Accelerate can also be installed with conda with:
Copied
conda install -c conda-forge accelerate
Source
New features are added every day that havenโt been released yet. To try them out yourself, install from the GitHub repository:
Copied
pip install git+https://github.com/boincai/accelerate
If youโre working on contributing to the library or wish to play with the source code and see live results as you run the code, an editable version can be installed from a locally-cloned version of the repository:
Copied
git clone https://github.com/boincai/accelerate
cd accelerate
pip install -e .
Configuring ๐ Accelerate
After installing, you need to configure๐ Accelerate for how the current system is setup for training. To do so run the following and answer the questions prompted to you:
Copied
accelerate config
To write a barebones configuration that doesnโt include options such as DeepSpeed configuration or running on TPUs, you can quickly run:
Copied
python -c "from accelerate.utils import write_basic_config; write_basic_config(mixed_precision='fp16')"
๐ Accelerate will automatically utilize the maximum number of GPUs available and set the mixed precision mode.
To check that your configuration looks fine, run:
Copied
accelerate env
An example output is shown below, which describes two GPUs on a single machine with no mixed precision being used:
Copied
- `Accelerate` version: 0.11.0.dev0
- Platform: Linux-5.10.0-15-cloud-amd64-x86_64-with-debian-11.3
- Python version: 3.7.12
- Numpy version: 1.19.5
- PyTorch version (GPU?): 1.12.0+cu102 (True)
- `Accelerate` default config:
- compute_environment: LOCAL_MACHINE
- distributed_type: MULTI_GPU
- mixed_precision: no
- use_cpu: False
- num_processes: 2
- machine_rank: 0
- num_machines: 1
- main_process_ip: None
- main_process_port: None
- main_training_function: main
- deepspeed_config: {}
- fsdp_config: {}
Last updated