• Takuma Mori's avatar
    Add support for Multi-ControlNet to StableDiffusionControlNetPipeline (#2627) · d9b8adc4
    Takuma Mori authored
    
    
    * support for List[ControlNetModel] on init()
    
    * Add to support for multiple ControlNetCondition
    
    * rename conditioning_scale to scale
    
    * scaling bugfix
    
    * Manually merge `MultiControlNet` #2621
    Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
    
    * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_controlnet.py
    Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
    
    * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_controlnet.py
    Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
    
    * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_controlnet.py
    Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
    
    * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_controlnet.py
    Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
    
    * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_controlnet.py
    Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
    
    * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_controlnet.py
    Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
    
    * cleanups
    - don't expose ControlNetCondition
    - move scaling to ControlNetModel
    
    * make style error correct
    
    * remove ControlNetCondition to reduce code diff
    
    * refactoring image/cond_scale
    
    * add explain for `images`
    
    * Add docstrings
    
    * all fast-test passed
    
    * Add a slow test
    
    * nit
    
    * Apply suggestions from code review
    
    * small precision fix
    
    * nits
    
    MultiControlNet -> MultiControlNetModel - Matches existing naming a bit
    closer
    
    MultiControlNetModel inherit from model utils class - Don't have to
    re-write fp16 test
    
    Skip tests that save multi controlnet pipeline - Clearer than changing
    test body
    
    Don't auto-batch the number of input images to the number of controlnets.
    We generally like to require the user to pass the expected number of
    inputs. This simplifies the processing code a bit more
    
    Use existing image pre-processing code a bit more. We can rely on the
    existing image pre-processing code and keep the inference loop a bit
    simpler.
    
    ---------
    Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
    Co-authored-by: default avatarWilliam Berman <WLBberman@gmail.com>
    d9b8adc4
controlnet.py 21.5 KB