# Image variation ## Image variation The Stable Diffusion model can also generate variations from an input image. It uses a fine-tuned version of a Stable Diffusion model by [Justin Pinkney](https://www.justinpinkney.com/) from [Lambda](https://lambdalabs.com/). The original codebase can be found at [LambdaLabsML/lambda-diffusers](https://github.com/LambdaLabsML/lambda-diffusers#stable-diffusion-image-variations) and additional official checkpoints for image variation can be found at [lambdalabs/sd-image-variations-diffusers](https://huggingface.co/lambdalabs/sd-image-variations-diffusers). Make sure to check out the Stable Diffusion [Tips](https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/overview#tips) section to learn how to explore the tradeoff between scheduler speed and quality, and how to reuse pipeline components efficiently! ### StableDiffusionImageVariationPipeline #### class diffusers.StableDiffusionImageVariationPipeline [\](https://github.com/huggingface/diffusers/blob/v0.21.0/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_image_variation.py#L37) ( vae: AutoencoderKLimage\_encoder: CLIPVisionModelWithProjectionunet: UNet2DConditionModelscheduler: KarrasDiffusionSchedulerssafety\_checker: StableDiffusionSafetyCheckerfeature\_extractor: CLIPImageProcessorrequires\_safety\_checker: bool = True ) Parameters * **vae** ([AutoencoderKL](https://huggingface.co/docs/diffusers/v0.21.0/en/api/models/autoencoderkl#diffusers.AutoencoderKL)) — Variational Auto-Encoder (VAE) model to encode and decode images to and from latent representations. * **image\_encoder** (`CLIPVisionModelWithProjection`) — Frozen CLIP image-encoder ([clip-vit-large-patch14](https://huggingface.co/openai/clip-vit-large-patch14)). * **text\_encoder** (`CLIPTextModel`) — Frozen text-encoder ([clip-vit-large-patch14](https://huggingface.co/openai/clip-vit-large-patch14)). * **tokenizer** (`CLIPTokenizer`) — A `CLIPTokenizer` to tokenize text. * **unet** ([UNet2DConditionModel](https://huggingface.co/docs/diffusers/v0.21.0/en/api/models/unet2d-cond#diffusers.UNet2DConditionModel)) — A `UNet2DConditionModel` to denoise the encoded image latents. * **scheduler** ([SchedulerMixin](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/overview#diffusers.SchedulerMixin)) — A scheduler to be used in combination with `unet` to denoise the encoded image latents. Can be one of [DDIMScheduler](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/ddim#diffusers.DDIMScheduler), [LMSDiscreteScheduler](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/lms_discrete#diffusers.LMSDiscreteScheduler), or [PNDMScheduler](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/pndm#diffusers.PNDMScheduler). * **safety\_checker** (`StableDiffusionSafetyChecker`) — Classification module that estimates whether generated images could be considered offensive or harmful. Please refer to the [model card](https://huggingface.co/runwayml/stable-diffusion-v1-5) for more details about a model’s potential harms. * **feature\_extractor** (`CLIPImageProcessor`) — A `CLIPImageProcessor` to extract features from generated images; used as inputs to the `safety_checker`. Pipeline to generate image variations from an input image using Stable Diffusion. This model inherits from [DiffusionPipeline](https://huggingface.co/docs/diffusers/v0.21.0/en/api/pipelines/overview#diffusers.DiffusionPipeline). Check the superclass documentation for the generic methods implemented for all pipelines (downloading, saving, running on a particular device, etc.). **\_\_call\_\_** [\](https://github.com/huggingface/diffusers/blob/v0.21.0/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_image_variation.py#L242) ( image: typing.Union\[PIL.Image.Image, typing.List\[PIL.Image.Image], torch.FloatTensor]height: typing.Optional\[int] = Nonewidth: typing.Optional\[int] = Nonenum\_inference\_steps: int = 50guidance\_scale: float = 7.5num\_images\_per\_prompt: typing.Optional\[int] = 1eta: float = 0.0generator: typing.Union\[torch.\_C.Generator, typing.List\[torch.\_C.Generator], NoneType] = Nonelatents: typing.Optional\[torch.FloatTensor] = Noneoutput\_type: typing.Optional\[str] = 'pil'return\_dict: bool = Truecallback: typing.Union\[typing.Callable\[\[int, int, torch.FloatTensor], NoneType], NoneType] = Nonecallback\_steps: int = 1 ) → [StableDiffusionPipelineOutput](https://huggingface.co/docs/diffusers/v0.21.0/en/api/pipelines/stable_diffusion/img2img#diffusers.pipelines.stable_diffusion.StableDiffusionPipelineOutput) or `tuple` Parameters * **image** (`PIL.Image.Image` or `List[PIL.Image.Image]` or `torch.FloatTensor`) — Image or images to guide image generation. If you provide a tensor, it needs to be compatible with [`CLIPImageProcessor`](https://huggingface.co/lambdalabs/sd-image-variations-diffusers/blob/main/feature_extractor/preprocessor_config.json). * **height** (`int`, *optional*, defaults to `self.unet.config.sample_size * self.vae_scale_factor`) — The height in pixels of the generated image. * **width** (`int`, *optional*, defaults to `self.unet.config.sample_size * self.vae_scale_factor`) — The width in pixels of the generated image. * **num\_inference\_steps** (`int`, *optional*, defaults to 50) — The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. This parameter is modulated by `strength`. * **guidance\_scale** (`float`, *optional*, defaults to 7.5) — A higher guidance scale value encourages the model to generate images closely linked to the text `prompt` at the expense of lower image quality. Guidance scale is enabled when `guidance_scale > 1`. * **num\_images\_per\_prompt** (`int`, *optional*, defaults to 1) — The number of images to generate per prompt. * **eta** (`float`, *optional*, defaults to 0.0) — Corresponds to parameter eta (η) from the [DDIM](https://arxiv.org/abs/2010.02502) paper. Only applies to the [DDIMScheduler](https://huggingface.co/docs/diffusers/v0.21.0/en/api/schedulers/ddim#diffusers.DDIMScheduler), and is ignored in other schedulers. * **generator** (`torch.Generator` or `List[torch.Generator]`, *optional*) — A [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. * **latents** (`torch.FloatTensor`, *optional*) — Pre-generated noisy latents sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor is generated by sampling using the supplied random `generator`. * **output\_type** (`str`, *optional*, defaults to `"pil"`) — The output format of the generated image. Choose between `PIL.Image` or `np.array`. * **return\_dict** (`bool`, *optional*, defaults to `True`) — Whether or not to return a [StableDiffusionPipelineOutput](https://huggingface.co/docs/diffusers/v0.21.0/en/api/pipelines/stable_diffusion/img2img#diffusers.pipelines.stable_diffusion.StableDiffusionPipelineOutput) instead of a plain tuple. * **callback** (`Callable`, *optional*) — A function that calls every `callback_steps` steps during inference. The function is called with the following arguments: `callback(step: int, timestep: int, latents: torch.FloatTensor)`. * **callback\_steps** (`int`, *optional*, defaults to 1) — The frequency at which the `callback` function is called. If not specified, the callback is called at every step. Returns [StableDiffusionPipelineOutput](https://huggingface.co/docs/diffusers/v0.21.0/en/api/pipelines/stable_diffusion/img2img#diffusers.pipelines.stable_diffusion.StableDiffusionPipelineOutput) or `tuple` If `return_dict` is `True`, [StableDiffusionPipelineOutput](https://huggingface.co/docs/diffusers/v0.21.0/en/api/pipelines/stable_diffusion/img2img#diffusers.pipelines.stable_diffusion.StableDiffusionPipelineOutput) is returned, otherwise a `tuple` is returned where the first element is a list with the generated images and the second element is a list of `bool`s indicating whether the corresponding generated image contains “not-safe-for-work” (nsfw) content. The call function to the pipeline for generation. Examples: Copied ``` from diffusers import StableDiffusionImageVariationPipeline from PIL import Image from io import BytesIO import requests pipe = StableDiffusionImageVariationPipeline.from_pretrained( "lambdalabs/sd-image-variations-diffusers", revision="v2.0" ) pipe = pipe.to("cuda") url = "https://lh3.googleusercontent.com/y-iFOHfLTwkuQSUegpwDdgKmOjRSTvPxat63dQLB25xkTs4lhIbRUFeNBWZzYf370g=s1200" response = requests.get(url) image = Image.open(BytesIO(response.content)).convert("RGB") out = pipe(image, num_images_per_prompt=3, guidance_scale=15) out["images"][0].save("result.jpg") ``` **enable\_attention\_slicing** [\](https://github.com/huggingface/diffusers/blob/v0.21.0/src/diffusers/pipelines/pipeline_utils.py#L1780) ( slice\_size: typing.Union\[str, int, NoneType] = 'auto' ) Parameters * **slice\_size** (`str` or `int`, *optional*, defaults to `"auto"`) — When `"auto"`, halves the input to the attention heads, so attention will be computed in two steps. If `"max"`, maximum amount of memory will be saved by running only one slice at a time. If a number is provided, uses as many slices as `attention_head_dim // slice_size`. In this case, `attention_head_dim` must be a multiple of `slice_size`. Enable sliced attention computation. When this option is enabled, the attention module splits the input tensor in slices to compute attention in several steps. For more than one attention head, the computation is performed sequentially over each head. This is useful to save some memory in exchange for a small speed decrease. ⚠️ Don’t enable attention slicing if you’re already using `scaled_dot_product_attention` (SDPA) from PyTorch 2.0 or xFormers. These attention computations are already very memory efficient so you won’t need to enable this function. If you enable attention slicing with SDPA or xFormers, it can lead to serious slow downs! Examples: Copied ``` >>> import torch >>> from diffusers import StableDiffusionPipeline >>> pipe = StableDiffusionPipeline.from_pretrained( ... "runwayml/stable-diffusion-v1-5", ... torch_dtype=torch.float16, ... use_safetensors=True, ... ) >>> prompt = "a photo of an astronaut riding a horse on mars" >>> pipe.enable_attention_slicing() >>> image = pipe(prompt).images[0] ``` **disable\_attention\_slicing** [\](https://github.com/huggingface/diffusers/blob/v0.21.0/src/diffusers/pipelines/pipeline_utils.py#L1820) ( ) Disable sliced attention computation. If `enable_attention_slicing` was previously called, attention is computed in one step. **enable\_xformers\_memory\_efficient\_attention** [\](https://github.com/huggingface/diffusers/blob/v0.21.0/src/diffusers/pipelines/pipeline_utils.py#L1719) ( attention\_op: typing.Optional\[typing.Callable] = None ) Parameters * **attention\_op** (`Callable`, *optional*) — Override the default `None` operator for use as `op` argument to the [`memory_efficient_attention()`](https://facebookresearch.github.io/xformers/components/ops.html#xformers.ops.memory_efficient_attention) function of xFormers. Enable memory efficient attention from [xFormers](https://facebookresearch.github.io/xformers/). When this option is enabled, you should observe lower GPU memory usage and a potential speed up during inference. Speed up during training is not guaranteed. ⚠️ When memory efficient attention and sliced attention are both enabled, memory efficient attention takes precedent. Examples: Copied ``` >>> import torch >>> from diffusers import DiffusionPipeline >>> from xformers.ops import MemoryEfficientAttentionFlashAttentionOp >>> pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1", torch_dtype=torch.float16) >>> pipe = pipe.to("cuda") >>> pipe.enable_xformers_memory_efficient_attention(attention_op=MemoryEfficientAttentionFlashAttentionOp) >>> # Workaround for not accepting attention shape using VAE for Flash Attention >>> pipe.vae.enable_xformers_memory_efficient_attention(attention_op=None) ``` **disable\_xformers\_memory\_efficient\_attention** [\](https://github.com/huggingface/diffusers/blob/v0.21.0/src/diffusers/pipelines/pipeline_utils.py#L1754) ( ) Disable memory efficient attention from [xFormers](https://facebookresearch.github.io/xformers/). ### StableDiffusionPipelineOutput #### class diffusers.pipelines.stable\_diffusion.StableDiffusionPipelineOutput [\](https://github.com/huggingface/diffusers/blob/v0.21.0/src/diffusers/pipelines/stable_diffusion/pipeline_output.py#L15) ( images: typing.Union\[typing.List\[PIL.Image.Image], numpy.ndarray]nsfw\_content\_detected: typing.Optional\[typing.List\[bool]] ) Parameters * **images** (`List[PIL.Image.Image]` or `np.ndarray`) — List of denoised PIL images of length `batch_size` or NumPy array of shape `(batch_size, height, width, num_channels)`. * **nsfw\_content\_detected** (`List[bool]`) — List indicating whether the corresponding generated image contains “not-safe-for-work” (nsfw) content or `None` if safety checking could not be performed. Output class for Stable Diffusion pipelines. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://boinc-ai.gitbook.io/diffusers/api/pipelines/stable-diffusion/image-variation.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.