> For the complete documentation index, see [llms.txt](https://boinc-ai.gitbook.io/timm/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://boinc-ai.gitbook.io/timm/reference/learning-rate-schedulers.md).

# Learning Rate Schedulers

## Learning Rate Schedulers

This page contains the API reference documentation for learning rate schedulers included in `timm`.

### Schedulers

#### Factory functions

**timm.scheduler.create\_scheduler**

[\<source>](https://github.com/huggingface/pytorch-image-models/blob/v0.9.8/timm/scheduler/scheduler_factory.py#L48)

( argsoptimizer: Optimizerupdates\_per\_epoch: int = 0 )

**timm.scheduler.create\_scheduler\_v2**

[\<source>](https://github.com/huggingface/pytorch-image-models/blob/v0.9.8/timm/scheduler/scheduler_factory.py#L60)

( optimizer: Optimizersched: str = 'cosine'num\_epochs: int = 300decay\_epochs: int = 90decay\_milestones: typing.List\[int] = (90, 180, 270)cooldown\_epochs: int = 0patience\_epochs: int = 10decay\_rate: float = 0.1min\_lr: float = 0warmup\_lr: float = 1e-05warmup\_epochs: int = 0warmup\_prefix: bool = Falsenoise: typing.Union\[float, typing.List\[float]] = Nonenoise\_pct: float = 0.67noise\_std: float = 1.0noise\_seed: int = 42cycle\_mul: float = 1.0cycle\_decay: float = 0.1cycle\_limit: int = 1k\_decay: float = 1.0plateau\_mode: str = 'max'step\_on\_epochs: bool = Trueupdates\_per\_epoch: int = 0 )

#### Scheduler Classes

#### class timm.scheduler.CosineLRScheduler

[\<source>](https://github.com/huggingface/pytorch-image-models/blob/v0.9.8/timm/scheduler/cosine_lr.py#L18)

( optimizer: Optimizert\_initial: intlr\_min: float = 0.0cycle\_mul: float = 1.0cycle\_decay: float = 1.0cycle\_limit: int = 1warmup\_t = 0warmup\_lr\_init = 0warmup\_prefix = Falset\_in\_epochs = Truenoise\_range\_t = Nonenoise\_pct = 0.67noise\_std = 1.0noise\_seed = 42k\_decay = 1.0initialize = True )

Cosine decay with restarts. This is described in the paper <https://arxiv.org/abs/1608.03983>.

Inspiration from <https://github.com/allenai/allennlp/blob/master/allennlp/training/learning_rate_schedulers/cosine.py>

k-decay option based on `k-decay: A New Method For Learning Rate Schedule` - <https://arxiv.org/abs/2004.05909>

#### class timm.scheduler.MultiStepLRScheduler

[\<source>](https://github.com/huggingface/pytorch-image-models/blob/v0.9.8/timm/scheduler/multistep_lr.py#L10)

( optimizer: Optimizerdecay\_t: typing.List\[int]decay\_rate: float = 1.0warmup\_t = 0warmup\_lr\_init = 0warmup\_prefix = Truet\_in\_epochs = Truenoise\_range\_t = Nonenoise\_pct = 0.67noise\_std = 1.0noise\_seed = 42initialize = True )

#### class timm.scheduler.PlateauLRScheduler

[\<source>](https://github.com/huggingface/pytorch-image-models/blob/v0.9.8/timm/scheduler/plateau_lr.py#L12)

( optimizerdecay\_rate = 0.1patience\_t = 10verbose = Truethreshold = 0.0001cooldown\_t = 0warmup\_t = 0warmup\_lr\_init = 0lr\_min = 0mode = 'max'noise\_range\_t = Nonenoise\_type = 'normal'noise\_pct = 0.67noise\_std = 1.0noise\_seed = Noneinitialize = True )

Decay the LR by a factor every time the validation loss plateaus.

#### class timm.scheduler.PolyLRScheduler

[\<source>](https://github.com/huggingface/pytorch-image-models/blob/v0.9.8/timm/scheduler/poly_lr.py#L18)

( optimizer: Optimizert\_initial: intpower: float = 0.5lr\_min: float = 0.0cycle\_mul: float = 1.0cycle\_decay: float = 1.0cycle\_limit: int = 1warmup\_t = 0warmup\_lr\_init = 0warmup\_prefix = Falset\_in\_epochs = Truenoise\_range\_t = Nonenoise\_pct = 0.67noise\_std = 1.0noise\_seed = 42k\_decay = 1.0initialize = True )

Polynomial LR Scheduler w/ warmup, noise, and k-decay

k-decay option based on `k-decay: A New Method For Learning Rate Schedule` - <https://arxiv.org/abs/2004.05909>

#### class timm.scheduler.StepLRScheduler

[\<source>](https://github.com/huggingface/pytorch-image-models/blob/v0.9.8/timm/scheduler/step_lr.py#L13)

( optimizer: Optimizerdecay\_t: floatdecay\_rate: float = 1.0warmup\_t = 0warmup\_lr\_init = 0warmup\_prefix = Truet\_in\_epochs = Truenoise\_range\_t = Nonenoise\_pct = 0.67noise\_std = 1.0noise\_seed = 42initialize = True )

#### class timm.scheduler.TanhLRScheduler

[\<source>](https://github.com/huggingface/pytorch-image-models/blob/v0.9.8/timm/scheduler/tanh_lr.py#L18)

( optimizer: Optimizert\_initial: intlb: float = -7.0ub: float = 3.0lr\_min: float = 0.0cycle\_mul: float = 1.0cycle\_decay: float = 1.0cycle\_limit: int = 1warmup\_t = 0warmup\_lr\_init = 0warmup\_prefix = Falset\_in\_epochs = Truenoise\_range\_t = Nonenoise\_pct = 0.67noise\_std = 1.0noise\_seed = 42initialize = True )

Hyberbolic-Tangent decay with restarts. This is described in the paper <https://arxiv.org/abs/1806.01593>