Diffusers BOINC AI docs
  • 🌍GET STARTED
    • Diffusers
    • Quicktour
    • Effective and efficient diffusion
    • Installation
  • 🌍TUTORIALS
    • Overview
    • Understanding models and schedulers
    • AutoPipeline
    • Train a diffusion model
  • 🌍USING DIFFUSERS
    • 🌍LOADING & HUB
      • Overview
      • Load pipelines, models, and schedulers
      • Load and compare different schedulers
      • Load community pipelines
      • Load safetensors
      • Load different Stable Diffusion formats
      • Push files to the Hub
    • 🌍TASKS
      • Unconditional image generation
      • Text-to-image
      • Image-to-image
      • Inpainting
      • Depth-to-image
    • 🌍TECHNIQUES
      • Textual inversion
      • Distributed inference with multiple GPUs
      • Improve image quality with deterministic generation
      • Control image brightness
      • Prompt weighting
    • 🌍PIPELINES FOR INFERENCE
      • Overview
      • Stable Diffusion XL
      • ControlNet
      • Shap-E
      • DiffEdit
      • Distilled Stable Diffusion inference
      • Create reproducible pipelines
      • Community pipelines
      • How to contribute a community pipeline
    • 🌍TRAINING
      • Overview
      • Create a dataset for training
      • Adapt a model to a new task
      • Unconditional image generation
      • Textual Inversion
      • DreamBooth
      • Text-to-image
      • Low-Rank Adaptation of Large Language Models (LoRA)
      • ControlNet
      • InstructPix2Pix Training
      • Custom Diffusion
      • T2I-Adapters
    • 🌍TAKING DIFFUSERS BEYOND IMAGES
      • Other Modalities
  • 🌍OPTIMIZATION/SPECIAL HARDWARE
    • Overview
    • Memory and Speed
    • Torch2.0 support
    • Stable Diffusion in JAX/Flax
    • xFormers
    • ONNX
    • OpenVINO
    • Core ML
    • MPS
    • Habana Gaudi
    • Token Merging
  • 🌍CONCEPTUAL GUIDES
    • Philosophy
    • Controlled generation
    • How to contribute?
    • Diffusers' Ethical Guidelines
    • Evaluating Diffusion Models
  • 🌍API
    • 🌍MAIN CLASSES
      • Attention Processor
      • Diffusion Pipeline
      • Logging
      • Configuration
      • Outputs
      • Loaders
      • Utilities
      • VAE Image Processor
    • 🌍MODELS
      • Overview
      • UNet1DModel
      • UNet2DModel
      • UNet2DConditionModel
      • UNet3DConditionModel
      • VQModel
      • AutoencoderKL
      • AsymmetricAutoencoderKL
      • Tiny AutoEncoder
      • Transformer2D
      • Transformer Temporal
      • Prior Transformer
      • ControlNet
    • 🌍PIPELINES
      • Overview
      • AltDiffusion
      • Attend-and-Excite
      • Audio Diffusion
      • AudioLDM
      • AudioLDM 2
      • AutoPipeline
      • Consistency Models
      • ControlNet
      • ControlNet with Stable Diffusion XL
      • Cycle Diffusion
      • Dance Diffusion
      • DDIM
      • DDPM
      • DeepFloyd IF
      • DiffEdit
      • DiT
      • IF
      • PaInstructPix2Pix
      • Kandinsky
      • Kandinsky 2.2
      • Latent Diffusionge
      • MultiDiffusion
      • MusicLDM
      • PaintByExample
      • Parallel Sampling of Diffusion Models
      • Pix2Pix Zero
      • PNDM
      • RePaint
      • Score SDE VE
      • Self-Attention Guidance
      • Semantic Guidance
      • Shap-E
      • Spectrogram Diffusion
      • 🌍STABLE DIFFUSION
        • Overview
        • Text-to-image
        • Image-to-image
        • Inpainting
        • Depth-to-image
        • Image variation
        • Safe Stable Diffusion
        • Stable Diffusion 2
        • Stable Diffusion XL
        • Latent upscaler
        • Super-resolution
        • LDM3D Text-to-(RGB, Depth)
        • Stable Diffusion T2I-adapter
        • GLIGEN (Grounded Language-to-Image Generation)
      • Stable unCLIP
      • Stochastic Karras VE
      • Text-to-image model editing
      • Text-to-video
      • Text2Video-Zero
      • UnCLIP
      • Unconditional Latent Diffusion
      • UniDiffuser
      • Value-guided sampling
      • Versatile Diffusion
      • VQ Diffusion
      • Wuerstchen
    • 🌍SCHEDULERS
      • Overview
      • CMStochasticIterativeScheduler
      • DDIMInverseScheduler
      • DDIMScheduler
      • DDPMScheduler
      • DEISMultistepScheduler
      • DPMSolverMultistepInverse
      • DPMSolverMultistepScheduler
      • DPMSolverSDEScheduler
      • DPMSolverSinglestepScheduler
      • EulerAncestralDiscreteScheduler
      • EulerDiscreteScheduler
      • HeunDiscreteScheduler
      • IPNDMScheduler
      • KarrasVeScheduler
      • KDPM2AncestralDiscreteScheduler
      • KDPM2DiscreteScheduler
      • LMSDiscreteScheduler
      • PNDMScheduler
      • RePaintScheduler
      • ScoreSdeVeScheduler
      • ScoreSdeVpScheduler
      • UniPCMultistepScheduler
      • VQDiffusionScheduler
Powered by GitBook
On this page
  • Diffusers
  • Supported pipelines
  1. GET STARTED

Diffusers

PreviousGET STARTEDNextQuicktour

Last updated 1 year ago

Diffusers

🌍 Diffusers is the go-to library for state-of-the-art pretrained diffusion models for generating images, audio, and even 3D structures of molecules. Whether you’re looking for a simple inference solution or want to train your own diffusion model, 🌍 Diffusers is a modular toolbox that supports both. Our library is designed with a focus on , , and .

The library has three main components:

  • State-of-the-art for inference with just a few lines of code.

  • Interchangeable for balancing trade-offs between generation speed and quality.

  • Pretrained that can be used as building blocks, and combined with schedulers, for creating your own end-to-end diffusion systems.

Supported pipelines

Pipeline
Paper/Repository
Tasks

Image-to-Image Text-Guided Generation

Unconditional Audio Generation

Image-to-Image Text-Guided Generation

Image-to-Image Text-Guided Generation

Unconditional Audio Generation

Unconditional Image Generation

Unconditional Image Generation

Image Generation

Image-to-Image Generation

Image-to-Image Generation

Text-to-Image Generation

Super Resolution Image-to-Image

Unconditional Image Generation

Image-Guided Image Inpainting

Unconditional Image Generation

Unconditional Image Generation

Unconditional Image Generation

Text-Guided Generation

Image-to-Image Text-Guided Generation

Text-to-Image Generation

Image-to-Image Text-Guided Generation

Text-Guided Image Inpainting

Text-to-Panorama Generation

Text-Guided Image Editing

Text-Guided Image Editing

Text-to-Image Generation

Text-to-Image Generation Unconditional Image Generation

Image-to-Image Generation

Text-Guided Super Resolution Image-to-Image

Text-to-Image Model Editing

Text-to-Image Generation

Text-Guided Image Inpainting

Depth-to-Image Generation

Text-Guided Super Resolution Image-to-Image

Text-Guided Generation

Stable unCLIP

Text-to-Image Generation

Stable unCLIP

Image-to-Image Text-Guided Generation

Unconditional Image Generation

Text-to-Video Generation

Text-to-Image Generation

Text-to-Image Generation

Image Variations Generation

Dual Image and Text Guided Generation

Text-to-Image Generation

Text to Image and Depth Generation

(implementation by )

🌍
usability over performance
simple over easy
customizability over abstractions
diffusion pipelines
noise schedulers
models
alt_diffusion
AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities
audio_diffusion
Audio Diffusion
controlnet
Adding Conditional Control to Text-to-Image Diffusion Models
cycle_diffusion
Unifying Diffusion Models’ Latent Space, with Applications to CycleDiffusion and Guidance
dance_diffusion
Dance Diffusion
ddpm
Denoising Diffusion Probabilistic Models
ddim
Denoising Diffusion Implicit Models
if
IF
if_img2img
IF
if_inpainting
IF
latent_diffusion
High-Resolution Image Synthesis with Latent Diffusion Models
latent_diffusion
High-Resolution Image Synthesis with Latent Diffusion Models
latent_diffusion_uncond
High-Resolution Image Synthesis with Latent Diffusion Models
paint_by_example
Paint by Example: Exemplar-based Image Editing with Diffusion Models
pndm
Pseudo Numerical Methods for Diffusion Models on Manifolds
score_sde_ve
Score-Based Generative Modeling through Stochastic Differential Equations
score_sde_vp
Score-Based Generative Modeling through Stochastic Differential Equations
semantic_stable_diffusion
Semantic Guidance
stable_diffusion_adapter
T2I-Adapter
stable_diffusion_text2img
Stable Diffusion
stable_diffusion_img2img
Stable Diffusion
stable_diffusion_inpaint
Stable Diffusion
stable_diffusion_panorama
MultiDiffusion
stable_diffusion_pix2pix
InstructPix2Pix: Learning to Follow Image Editing Instructions
stable_diffusion_pix2pix_zero
Zero-shot Image-to-Image Translation
stable_diffusion_attend_and_excite
Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models
stable_diffusion_self_attention_guidance
Improving Sample Quality of Diffusion Models Using Self-Attention Guidance
stable_diffusion_image_variation
Stable Diffusion Image Variations
stable_diffusion_latent_upscale
Stable Diffusion Latent Upscaler
stable_diffusion_model_editing
Editing Implicit Assumptions in Text-to-Image Diffusion Models
stable_diffusion_2
Stable Diffusion 2
stable_diffusion_2
Stable Diffusion 2
stable_diffusion_2
Depth-Conditional Stable Diffusion
stable_diffusion_2
Stable Diffusion 2
stable_diffusion_safe
Safe Stable Diffusion
stable_unclip
stable_unclip
stochastic_karras_ve
Elucidating the Design Space of Diffusion-Based Generative Models
text_to_video_sd
Modelscope’s Text-to-video-synthesis Model in Open Domain
unclip
Hierarchical Text-Conditional Image Generation with CLIP Latents
kakaobrain
versatile_diffusion
Versatile Diffusion: Text, Images and Variations All in One Diffusion Model
versatile_diffusion
Versatile Diffusion: Text, Images and Variations All in One Diffusion Model
versatile_diffusion
Versatile Diffusion: Text, Images and Variations All in One Diffusion Model
vq_diffusion
Vector Quantized Diffusion Model for Text-to-Image Synthesis
stable_diffusion_ldm3d
LDM3D: Latent Diffusion Model for 3D