Overview
Last updated
Last updated
🌍 Diffusers provides many scheduler functions for the diffusion process. A scheduler takes a model’s output (the sample which the diffusion process is iterating on) and a timestep to return a denoised sample. The timestep is important because it dictates where in the diffusion process the step is; data is generated by iterating forward n timesteps and inference occurs by propagating backward through the timesteps. Based on the timestep, a scheduler may be discrete in which case the timestep is an int or continuous in which case the timestep is a float.
Depending on the context, a scheduler defines how to iteratively add noise to an image or how to update a sample based on a model’s output:
during training, a scheduler adds noise (there are different algorithms for how to add noise) to a sample to train a diffusion model
during inference, a scheduler defines how to update a sample based on a pretrained model’s output
Many schedulers are implemented from the library by , and they’re also widely used in A1111. To help you map the schedulers from k-diffusion and A1111 to the schedulers in 🌍 Diffusers, take a look at the table below:
DPM++ 2M
DPM++ 2M Karras
init with use_karras_sigmas=True
DPM++ 2M SDE
init with algorithm_type="sde-dpmsolver++"
DPM++ 2M SDE Karras
init with use_karras_sigmas=True and algorithm_type="sde-dpmsolver++"
DPM++ 2S a
N/A
very similar to DPMSolverSinglestepScheduler
DPM++ 2S a Karras
N/A
very similar to DPMSolverSinglestepScheduler(use_karras_sigmas=True, ...)
DPM++ SDE
DPM++ SDE Karras
init with use_karras_sigmas=True
DPM2
DPM2 Karras
init with use_karras_sigmas=True
DPM2 a
DPM2 a Karras
init with use_karras_sigmas=True
DPM adaptive
N/A
DPM fast
N/A
Euler
Euler a
Heun
LMS
LMS Karras
init with use_karras_sigmas=True
N/A
N/A
( )
Base class for all schedulers.
Class attributes:
from_pretrained
( pretrained_model_name_or_path: typing.Dict[str, typing.Any] = Nonesubfolder: typing.Optional[str] = Nonereturn_unused_kwargs = False**kwargs )
Parameters
pretrained_model_name_or_path (str or os.PathLike, optional) — Can be either:
A string, the model id (for example google/ddpm-celebahq-256) of a pretrained model hosted on the Hub.
subfolder (str, optional) — The subfolder location of a model file within a larger model repository on the Hub or locally.
return_unused_kwargs (bool, optional, defaults to False) — Whether kwargs that are not consumed by the Python class should be returned or not.
cache_dir (Union[str, os.PathLike], optional) — Path to a directory where a downloaded pretrained model configuration is cached if the standard cache is not used.
force_download (bool, optional, defaults to False) — Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist.
resume_download (bool, optional, defaults to False) — Whether or not to resume downloading the model weights and configuration files. If set to False, any incompletely downloaded files are deleted.
proxies (Dict[str, str], optional) — A dictionary of proxy servers to use by protocol or endpoint, for example, {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.
output_loading_info(bool, optional, defaults to False) — Whether or not to also return a dictionary containing missing keys, unexpected keys and error messages.
local_files_only(bool, optional, defaults to False) — Whether to only load local model weights and configuration files or not. If set to True, the model won’t be downloaded from the Hub.
use_auth_token (str or bool, optional) — The token to use as HTTP bearer authorization for remote files. If True, the token generated from diffusers-cli login (stored in ~/.boincai) is used.
revision (str, optional, defaults to "main") — The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier allowed by Git.
Instantiate a scheduler from a pre-defined JSON configuration file in a local directory or Hub repository.
save_pretrained
( save_directory: typing.Union[str, os.PathLike]push_to_hub: bool = False**kwargs )
Parameters
save_directory (str or os.PathLike) — Directory where the configuration JSON file will be saved (will be created if it does not exist).
push_to_hub (bool, optional, defaults to False) — Whether or not to push your model to the BOINC AI Hub after saving it. You can specify the repository you want to push to with repo_id (will default to the name of save_directory in your namespace).
( prev_sample: FloatTensor )
Parameters
prev_sample (torch.FloatTensor of shape (batch_size, num_channels, height, width) for images) — Computed sample (x_{t-1}) of previous timestep. prev_sample should be used as next model input in the denoising loop.
Base class for the output of a scheduler’s step function.
KarrasDiffusionSchedulers are a broad generalization of schedulers in 🌍 Diffusers. The schedulers in this class are distinguished at a high level by their noise sampling strategy, the type of network and scaling, the training strategy, and how the loss is weighed.
( )
A Mixin to push a model, scheduler, or pipeline to the BOINC AI Hub.
push_to_hub
( repo_id: strcommit_message: typing.Optional[str] = Noneprivate: typing.Optional[bool] = Nonetoken: typing.Optional[str] = Nonecreate_pr: bool = Falsesafe_serialization: bool = Truevariant: typing.Optional[str] = None )
Parameters
repo_id (str) — The name of the repository you want to push your model, scheduler, or pipeline files to. It should contain your organization name when pushing to an organization. repo_id can also be a path to a local directory.
commit_message (str, optional) — Message to commit while pushing. Default to "Upload {object}".
private (bool, optional) — Whether or not the repository created should be private.
token (str, optional) — The token to use as HTTP bearer authorization for remote files. The token generated when running boincai-cli login (stored in ~/.boincai).
create_pr (bool, optional, defaults to False) — Whether or not to create a PR with the uploaded files or directly commit.
safe_serialization (bool, optional, defaults to True) — Whether or not to convert the model weights to the safetensors format.
variant (str, optional) — If specified, weights are saved in the format pytorch_model.<variant>.bin.
Upload model, scheduler, or pipeline files to the 🌍 BOINC AI Hub.
Examples:
Copied
All schedulers are built from the base class which implements low level utilities shared by all schedulers.
contains common functions shared by all schedulers such as general loading and saving functionalities.
takes care of storing the configuration attributes (like num_train_timesteps) that are passed to the scheduler’s __init__ function, and the attributes can be accessed by scheduler.config.num_train_timesteps.
_compatibles (List[str]) — A list of scheduler classes that are compatible with the parent scheduler class. Use to load a different compatible scheduler class (should be overridden by parent class).
A path to a directory (for example ./my_model_directory) containing the scheduler configuration saved with .
To use private or , log-in with boincai-cli login. You can also activate the special to use this method in a firewalled environment.
kwargs (Dict[str, Any], optional) — Additional keyword arguments passed along to the method.
Save a scheduler configuration object to a directory so that it can be reloaded using the class method.
The different schedulers in this class, depending on the ordinary differential equations (ODE) solver type, fall into the above taxonomy and provide a good abstraction for the design of the main schedulers implemented in 🌍 Diffusers. The schedulers in this class are given .