# Parallel Sampling of Diffusion Models (ParaDiGMS) ## Overview [Parallel Sampling of Diffusion Models](https://arxiv.org/abs/2305.16317) by Andy Shih, Suneel Belkhale, Stefano Ermon, Dorsa Sadigh, Nima Anari. The abstract of the paper is the following: *Diffusion models are powerful generative models but suffer from slow sampling, often taking 1000 sequential denoising steps for one sample. As a result, considerable efforts have been directed toward reducing the number of denoising steps, but these methods hurt sample quality. Instead of reducing the number of denoising steps (trading quality for speed), in this paper we explore an orthogonal approach: can we run the denoising steps in parallel (trading compute for speed)? In spite of the sequential nature of the denoising steps, we show that surprisingly it is possible to parallelize sampling via Picard iterations, by guessing the solution of future denoising steps and iteratively refining until convergence. With this insight, we present ParaDiGMS, a novel method to accelerate the sampling of pretrained diffusion models by denoising multiple steps in parallel. ParaDiGMS is the first diffusion sampling method that enables trading compute for speed and is even compatible with existing fast sampling techniques such as DDIM and DPMSolver. Using ParaDiGMS, we improve sampling speed by 2-4x across a range of robotics and image generation models, giving state-of-the-art sampling speeds of 0.2s on 100-step DiffusionPolicy and 16s on 1000-step StableDiffusion-v2 with no measurable degradation of task reward, FID score, or CLIP score.* Resources: * [Paper](https://arxiv.org/abs/2305.16317). * [Original Code](https://github.com/AndyShih12/paradigms). ## Available Pipelines: | Pipeline | Tasks | Demo |---|---|:---:| | [StableDiffusionParadigmsPipeline](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_paradigms.py) | *Faster Text-to-Image Generation* | | This pipeline was contributed by [`AndyShih12`](https://github.com/AndyShih12) in this [PR](https://github.com/huggingface/diffusers/pull/3716/). ## Usage example ```python import torch from diffusers import DDPMParallelScheduler from diffusers import StableDiffusionParadigmsPipeline scheduler = DDPMParallelScheduler.from_pretrained("runwayml/stable-diffusion-v1-5", subfolder="scheduler") pipe = StableDiffusionParadigmsPipeline.from_pretrained( "runwayml/stable-diffusion-v1-5", scheduler=scheduler, torch_dtype=torch.float16 ) pipe = pipe.to("cuda") ngpu, batch_per_device = torch.cuda.device_count(), 5 pipe.wrapped_unet = torch.nn.DataParallel(pipe.unet, device_ids=[d for d in range(ngpu)]) prompt = "a photo of an astronaut riding a horse on mars" image = pipe(prompt, parallel=ngpu * batch_per_device, num_inference_steps=1000).images[0] ``` This pipeline improves sampling speed by running denoising steps in parallel, at the cost of increased total FLOPs. Therefore, it is better to call this pipeline when running on multiple GPUs. Otherwise, without enough GPU bandwidth sampling may be even slower than sequential sampling. The two parameters to play with are `parallel` (batch size) and `tolerance`. - If it fits in memory, for 1000-step DDPM you can aim for a batch size of around 100 (e.g. 8 GPUs and batch_per_device=12 to get parallel=96). Higher batch size may not fit in memory, and lower batch size gives less parallelism. - For tolerance, using a higher tolerance may get better speedups but can risk sample quality degradation. If there is quality degradation with the default tolerance, then use a lower tolerance (e.g. 0.001). For 1000-step DDPM on 8 A100 GPUs, you can expect around a 3x speedup by StableDiffusionParadigmsPipeline instead of StableDiffusionPipeline by setting parallel=80 and tolerance=0.1. Diffusers also offers distributed inference support for generating multiple prompts in parallel on multiple GPUs. Check out the docs [here](https://huggingface.co/docs/diffusers/main/en/training/distributed_inference). In contrast, this pipeline is designed for speeding up sampling of a single prompt (by using multiple GPUs). ## StableDiffusionParadigmsPipeline [[autodoc]] StableDiffusionParadigmsPipeline - __call__ - all