> For the complete documentation index, see [llms.txt](https://boinc-ai.gitbook.io/diffusers/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://boinc-ai.gitbook.io/diffusers/api/pipelines/latent-diffusionge.md). # Latent Diffusionge ### LDMTextToImagePipeline #### class diffusers.LDMTextToImagePipeline [\](https://github.com/huggingface/diffusers/blob/v0.21.0/src/diffusers/pipelines/latent_diffusion/pipeline_latent_diffusion.py#L32) ( vqvae: typing.Union\[diffusers.models.vq\_model.VQModel, diffusers.models.autoencoder\_kl.AutoencoderKL]bert: PreTrainedModeltokenizer: PreTrainedTokenizerunet: typing.Union\[diffusers.models.unet\_2d.UNet2DModel, diffusers.models.unet\_2d\_condition.UNet2DConditionModel]scheduler: typing.Union\[diffusers.schedulers.scheduling\_ddim.DDIMScheduler, diffusers.schedulers.scheduling\_pndm.PNDMScheduler, diffusers.schedulers.scheduling\_lms\_discrete.LMSDiscreteScheduler] ) Parameters * **vqvae** ([VQModel](https://huggingface.co/docs/diffusers/v0.21.0/en/api/models/vq#diffusers.VQModel)) — Vector-quantized (VQ) model to encode and decode images to and from latent representations. * **bert** (`LDMBertModel`) — Text-encoder model based on `BERT`. * **tokenizer** (`BertTokenizer`) — A `BertTokenizer` to tokenize text. * **unet** ([UNet2DConditionModel](https://huggingface.co/docs/diffusers/v0.21.0/en/api/models/unet2d-cond#diffusers.UNet2DConditionModel)) — A `UNet2DConditionModel` to denoise the encoded image latents. * **scheduler** ([SchedulerMixin](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/overview#diffusers.SchedulerMixin)) — A scheduler to be used in combination with `unet` to denoise the encoded image latents. Can be one of [DDIMScheduler](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/ddim#diffusers.DDIMScheduler), [LMSDiscreteScheduler](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/lms_discrete#diffusers.LMSDiscreteScheduler), or [PNDMScheduler](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/pndm#diffusers.PNDMScheduler). Pipeline for text-to-image generation using latent diffusion. This model inherits from [DiffusionPipeline](https://huggingface.co/docs/diffusers/v0.21.0/en/api/pipelines/overview#diffusers.DiffusionPipeline). Check the superclass documentation for the generic methods implemented for all pipelines (downloading, saving, running on a particular device, etc.). **\_\_call\_\_** [\](https://github.com/huggingface/diffusers/blob/v0.21.0/src/diffusers/pipelines/latent_diffusion/pipeline_latent_diffusion.py#L66) ( prompt: typing.Union\[str, typing.List\[str]]height: typing.Optional\[int] = Nonewidth: typing.Optional\[int] = Nonenum\_inference\_steps: typing.Optional\[int] = 50guidance\_scale: typing.Optional\[float] = 1.0eta: typing.Optional\[float] = 0.0generator: typing.Union\[torch.\_C.Generator, typing.List\[torch.\_C.Generator], NoneType] = Nonelatents: typing.Optional\[torch.FloatTensor] = Noneoutput\_type: typing.Optional\[str] = 'pil'return\_dict: bool = True\*\*kwargs ) → [ImagePipelineOutput](https://huggingface.co/docs/diffusers/v0.21.0/en/api/pipelines/latent_diffusion_uncond#diffusers.ImagePipelineOutput) or `tuple` Parameters * **prompt** (`str` or `List[str]`) — The prompt or prompts to guide the image generation. * **height** (`int`, *optional*, defaults to `self.unet.config.sample_size * self.vae_scale_factor`) — The height in pixels of the generated image. * **width** (`int`, *optional*, defaults to `self.unet.config.sample_size * self.vae_scale_factor`) — The width in pixels of the generated image. * **num\_inference\_steps** (`int`, *optional*, defaults to 50) — The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. * **guidance\_scale** (`float`, *optional*, defaults to 1.0) — A higher guidance scale value encourages the model to generate images closely linked to the text `prompt` at the expense of lower image quality. Guidance scale is enabled when `guidance_scale > 1`. * **generator** (`torch.Generator`, *optional*) — A [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. * **latents** (`torch.FloatTensor`, *optional*) — Pre-generated noisy latents sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor is generated by sampling using the supplied random `generator`. * **output\_type** (`str`, *optional*, defaults to `"pil"`) — The output format of the generated image. Choose between `PIL.Image` or `np.array`. * **return\_dict** (`bool`, *optional*, defaults to `True`) — Whether or not to return a [ImagePipelineOutput](https://huggingface.co/docs/diffusers/v0.21.0/en/api/pipelines/latent_diffusion_uncond#diffusers.ImagePipelineOutput) instead of a plain tuple. Returns [ImagePipelineOutput](https://huggingface.co/docs/diffusers/v0.21.0/en/api/pipelines/latent_diffusion_uncond#diffusers.ImagePipelineOutput) or `tuple` If `return_dict` is `True`, [ImagePipelineOutput](https://huggingface.co/docs/diffusers/v0.21.0/en/api/pipelines/latent_diffusion_uncond#diffusers.ImagePipelineOutput) is returned, otherwise a `tuple` is returned where the first element is a list with the generated images. The call function to the pipeline for generation. Example: Copied ``` >>> from diffusers import DiffusionPipeline >>> # load model and scheduler >>> ldm = DiffusionPipeline.from_pretrained("CompVis/ldm-text2im-large-256") >>> # run pipeline in inference (sample random noise and denoise) >>> prompt = "A painting of a squirrel eating a burger" >>> images = ldm([prompt], num_inference_steps=50, eta=0.3, guidance_scale=6).images >>> # save images >>> for idx, image in enumerate(images): ... image.save(f"squirrel-{idx}.png") ``` ### LDMSuperResolutionPipeline #### class diffusers.LDMSuperResolutionPipeline [\](https://github.com/huggingface/diffusers/blob/v0.21.0/src/diffusers/pipelines/latent_diffusion/pipeline_latent_diffusion_superresolution.py#L33) ( vqvae: VQModelunet: UNet2DModelscheduler: typing.Union\[diffusers.schedulers.scheduling\_ddim.DDIMScheduler, diffusers.schedulers.scheduling\_pndm.PNDMScheduler, diffusers.schedulers.scheduling\_lms\_discrete.LMSDiscreteScheduler, diffusers.schedulers.scheduling\_euler\_discrete.EulerDiscreteScheduler, diffusers.schedulers.scheduling\_euler\_ancestral\_discrete.EulerAncestralDiscreteScheduler, diffusers.schedulers.scheduling\_dpmsolver\_multistep.DPMSolverMultistepScheduler] ) Parameters * **vqvae** ([VQModel](https://huggingface.co/docs/diffusers/v0.21.0/en/api/models/vq#diffusers.VQModel)) — Vector-quantized (VQ) model to encode and decode images to and from latent representations. * **unet** ([UNet2DModel](https://huggingface.co/docs/diffusers/v0.21.0/en/api/models/unet2d#diffusers.UNet2DModel)) — A `UNet2DModel` to denoise the encoded image. * **scheduler** ([SchedulerMixin](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/overview#diffusers.SchedulerMixin)) — A scheduler to be used in combination with `unet` to denoise the encoded image latens. Can be one of [DDIMScheduler](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/ddim#diffusers.DDIMScheduler), [LMSDiscreteScheduler](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/lms_discrete#diffusers.LMSDiscreteScheduler), [EulerDiscreteScheduler](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/euler#diffusers.EulerDiscreteScheduler), [EulerAncestralDiscreteScheduler](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/euler_ancestral#diffusers.EulerAncestralDiscreteScheduler), [DPMSolverMultistepScheduler](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/multistep_dpm_solver#diffusers.DPMSolverMultistepScheduler), or [PNDMScheduler](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/pndm#diffusers.PNDMScheduler). A pipeline for image super-resolution using latent diffusion. This model inherits from [DiffusionPipeline](https://huggingface.co/docs/diffusers/v0.21.0/en/api/pipelines/overview#diffusers.DiffusionPipeline). Check the superclass documentation for the generic methods implemented for all pipelines (downloading, saving, running on a particular device, etc.). **\_\_call\_\_** [\](https://github.com/huggingface/diffusers/blob/v0.21.0/src/diffusers/pipelines/latent_diffusion/pipeline_latent_diffusion_superresolution.py#L67) ( image: typing.Union\[torch.Tensor, PIL.Image.Image] = Nonebatch\_size: typing.Optional\[int] = 1num\_inference\_steps: typing.Optional\[int] = 100eta: typing.Optional\[float] = 0.0generator: typing.Union\[torch.\_C.Generator, typing.List\[torch.\_C.Generator], NoneType] = Noneoutput\_type: typing.Optional\[str] = 'pil'return\_dict: bool = True ) → [ImagePipelineOutput](https://huggingface.co/docs/diffusers/v0.21.0/en/api/pipelines/latent_diffusion_uncond#diffusers.ImagePipelineOutput) or `tuple` Parameters * **image** (`torch.Tensor` or `PIL.Image.Image`) — `Image` or tensor representing an image batch to be used as the starting point for the process. * **batch\_size** (`int`, *optional*, defaults to 1) — Number of images to generate. * **num\_inference\_steps** (`int`, *optional*, defaults to 100) — The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. * **eta** (`float`, *optional*, defaults to 0.0) — Corresponds to parameter eta (η) from the [DDIM](https://arxiv.org/abs/2010.02502) paper. Only applies to the [DDIMScheduler](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/ddim#diffusers.DDIMScheduler), and is ignored in other schedulers. * **generator** (`torch.Generator` or `List[torch.Generator]`, *optional*) — A [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. * **output\_type** (`str`, *optional*, defaults to `"pil"`) — The output format of the generated image. Choose between `PIL.Image` or `np.array`. * **return\_dict** (`bool`, *optional*, defaults to `True`) — Whether or not to return a [ImagePipelineOutput](https://huggingface.co/docs/diffusers/v0.21.0/en/api/pipelines/latent_diffusion_uncond#diffusers.ImagePipelineOutput) instead of a plain tuple. Returns [ImagePipelineOutput](https://huggingface.co/docs/diffusers/v0.21.0/en/api/pipelines/latent_diffusion_uncond#diffusers.ImagePipelineOutput) or `tuple` If `return_dict` is `True`, [ImagePipelineOutput](https://huggingface.co/docs/diffusers/v0.21.0/en/api/pipelines/latent_diffusion_uncond#diffusers.ImagePipelineOutput) is returned, otherwise a `tuple` is returned where the first element is a list with the generated images The call function to the pipeline for generation. Example: Copied ``` >>> import requests >>> from PIL import Image >>> from io import BytesIO >>> from diffusers import LDMSuperResolutionPipeline >>> import torch >>> # load model and scheduler >>> pipeline = LDMSuperResolutionPipeline.from_pretrained("CompVis/ldm-super-resolution-4x-openimages") >>> pipeline = pipeline.to("cuda") >>> # let's download an image >>> url = ( ... "https://user-images.githubusercontent.com/38061659/199705896-b48e17b8-b231-47cd-a270-4ffa5a93fa3e.png" ... ) >>> response = requests.get(url) >>> low_res_img = Image.open(BytesIO(response.content)).convert("RGB") >>> low_res_img = low_res_img.resize((128, 128)) >>> # run pipeline in inference (sample random noise and denoise) >>> upscaled_image = pipeline(low_res_img, num_inference_steps=100, eta=1).images[0] >>> # save image >>> upscaled_image.save("ldm_generated_image.png") ``` ### ImagePipelineOutput #### class diffusers.ImagePipelineOutput [\](https://github.com/huggingface/diffusers/blob/v0.21.0/src/diffusers/pipelines/pipeline_utils.py#L112) ( images: typing.Union\[typing.List\[PIL.Image.Image], numpy.ndarray] ) Parameters * **images** (`List[PIL.Image.Image]` or `np.ndarray`) — List of denoised PIL images of length `batch_size` or NumPy array of shape `(batch_size, height, width, num_channels)`. Output class for image pipelines.