1. 06 Dec, 2023 1 commit
    • UmerHA's avatar
      Add ControlNet-XS support (#5827) · e192ae08
      UmerHA authored
      
      
      * Check in 23-10-05
      
      * check-in 23-10-06
      
      * check-in 23-10-07 2pm
      
      * check-in 23-10-08
      
      * check-in 231009T1200
      
      * check-in 230109
      
      * checkin 231010
      
      * init + forward run
      
      * checkin
      
      * checkin
      
      * ControlNetXSModel is now saveable+loadable
      
      * Forward works
      
      * checkin
      
      * Pipeline works with `no_control=True`
      
      * checkin
      
      * debug: save intermediate outputs of resnet
      
      * checkin
      
      * Understood time error + fixed connection error
      
      * checkin
      
      * checkin 231106T1600
      
      * turned off detailled debug prints
      
      * time debug logs
      
      * small fix
      
      * Separated control_scale for connections/time
      
      * simplified debug logging
      
      * Full denoising works with control scale = 0
      
      * aligned logs
      
      * Added control_attention_head_dim param
      
      * Passing n_heads instead of dim_head into ctrl unet
      
      * Fixed ctrl midblock bug
      
      * Cleanup
      
      * Fixed time dtype bug
      
      * checkin
      
      * 1. from_unet, 2. base passed, 3. all unet params
      
      * checkin
      
      * Finished docstrings
      
      * cleanup
      
      * make style
      
      * checkin
      
      * more tests pass
      
      * Fixed tests
      
      * removed debug logs
      
      * make style + quality
      
      * make fix-copies
      
      * fixed documentation
      
      * added cnxs to doc toc
      
      * added control start/end param
      
      * Update controlnetxs_sdxl.md
      
      * tried to fix copies..
      
      * Fixed norm_num_groups in from_unet
      
      * added sdxl-depth test
      
      * created SD2.1 controlnet-xs pipeline
      
      * re-added debug logs
      
      * Adjusting group norm ; readded logs
      
      * Added debug log statements
      
      * removed debug logs ; started tests for sd2.1
      
      * updated sd21 tests
      
      * fixed tests
      
      * fixed tests
      
      * slightly increased error tolerance for 1 test
      
      * make style & quality
      
      * Added docs for CNXS-SD
      
      * make fix-copies
      
      * Fixed sd compile test ; fixed gradient ckpointing
      
      * vae downs = cnxs conditioning downs; removed guess
      
      * make style & quality
      
      * Fixed tests
      
      * fixed test
      
      * Incorporated review feedback
      
      * simplified control model surgery
      
      * fixed tests & make style / quality
      
      * Updated docs; deleted pip & cursor files
      
      * Rolled back minimal change to resnet
      
      * Update resnet.py
      
      * Update resnet.py
      
      * Update src/diffusers/models/controlnetxs.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/models/controlnetxs.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Incorporated review feedback
      
      * Update docs/source/en/api/pipelines/controlnetxs_sdxl.md
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update docs/source/en/api/pipelines/controlnetxs.md
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update docs/source/en/api/pipelines/controlnetxs.md
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update docs/source/en/api/pipelines/controlnetxs.md
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update src/diffusers/models/controlnetxs.py
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update src/diffusers/models/controlnetxs.py
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update src/diffusers/pipelines/controlnet_xs/pipeline_controlnet_xs.py
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update docs/source/en/api/pipelines/controlnetxs.md
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update src/diffusers/pipelines/controlnet_xs/pipeline_controlnet_xs_sd_xl.py
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Incorporated doc feedback
      
      ---------
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      e192ae08
  2. 11 Sep, 2023 1 commit
    • Dhruv Nair's avatar
      Lazy Import for Diffusers (#4829) · b6e0b016
      Dhruv Nair authored
      
      
      * initial commit
      
      * move modules to import struct
      
      * add dummy objects and _LazyModule
      
      * add lazy import to schedulers
      
      * clean up unused imports
      
      * lazy import on models module
      
      * lazy import for schedulers module
      
      * add lazy import to pipelines module
      
      * lazy import altdiffusion
      
      * lazy import audio diffusion
      
      * lazy import audioldm
      
      * lazy import consistency model
      
      * lazy import controlnet
      
      * lazy import dance diffusion ddim ddpm
      
      * lazy import deepfloyd
      
      * lazy import kandinksy
      
      * lazy imports
      
      * lazy import semantic diffusion
      
      * lazy imports
      
      * lazy import stable diffusion
      
      * move sd output to its own module
      
      * clean up
      
      * lazy import t2iadapter
      
      * lazy import unclip
      
      * lazy import versatile and vq diffsuion
      
      * lazy import vq diffusion
      
      * helper to fetch objects from modules
      
      * lazy import sdxl
      
      * lazy import txt2vid
      
      * lazy import stochastic karras
      
      * fix model imports
      
      * fix bug
      
      * lazy import
      
      * clean up
      
      * clean up
      
      * fixes for tests
      
      * fixes for tests
      
      * clean up
      
      * remove import of torch_utils from utils module
      
      * clean up
      
      * clean up
      
      * fix mistake import statement
      
      * dedicated modules for exporting and loading
      
      * remove testing utils from utils module
      
      * fixes from  merge conflicts
      
      * Update src/diffusers/pipelines/kandinsky2_2/__init__.py
      
      * fix docs
      
      * fix alt diffusion copied from
      
      * fix check dummies
      
      * fix more docs
      
      * remove accelerate import from utils module
      
      * add type checking
      
      * make style
      
      * fix check dummies
      
      * remove torch import from xformers check
      
      * clean up error message
      
      * fixes after upstream merges
      
      * dummy objects fix
      
      * fix tests
      
      * remove unused module import
      
      ---------
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      b6e0b016
  3. 28 Aug, 2023 1 commit
  4. 26 Aug, 2023 1 commit
    • Sayak Paul's avatar
      [Core] Support negative conditions in SDXL (#4774) · 3be0ff90
      Sayak Paul authored
      * add: support negative conditions.
      
      * fix: key
      
      * add: tests
      
      * address PR feedback.
      
      * add documentation
      
      * add img2img support.
      
      * add inpainting support.
      
      * ad controlnet support
      
      * Apply suggestions from code review
      
      * modify wording in the doc.
      3be0ff90
  5. 28 Jul, 2023 1 commit
    • Patrick von Platen's avatar
      [SDXL Refiner] Fix refiner forward pass for batched input (#4327) · 18b018c8
      Patrick von Platen authored
      * fix_batch_xl
      
      * Fix other pipelines as well
      
      * up
      
      * up
      
      * Update tests/pipelines/stable_diffusion_xl/test_stable_diffusion_xl_inpaint.py
      
      * sort
      
      * up
      
      * Finish it all up Co-authored-by: Bagheera <bghira@users.github.com>
      
      * Co-authored-by: Bagheera bghira@users.github.com
      
      * Co-authored-by: Bagheera <bghira@users.github.com>
      
      * Finish it all up Co-authored-by: Bagheera <bghira@users.github.com>
      18b018c8
  6. 27 Jul, 2023 1 commit
  7. 24 Jul, 2023 1 commit
  8. 21 Jul, 2023 1 commit
  9. 10 Jul, 2023 1 commit
  10. 07 Jul, 2023 1 commit
  11. 06 Jul, 2023 1 commit
    • Patrick von Platen's avatar
      [SD-XL] Add new pipelines (#3859) · bc9a8cef
      Patrick von Platen authored
      
      
      * Add new text encoder
      
      * add transformers depth
      
      * More
      
      * Correct conversion script
      
      * Fix more
      
      * Fix more
      
      * Correct more
      
      * correct text encoder
      
      * Finish all
      
      * proof that in works in run local xl
      
      * clean up
      
      * Get refiner to work
      
      * Add red castle
      
      * Fix batch size
      
      * Improve pipelines more
      
      * Finish text2image tests
      
      * Add img2img test
      
      * Fix more
      
      * fix import
      
      * Fix embeddings for classic models (#3888)
      
      Fix embeddings for classic SD models.
      
      * Allow multiple prompts to be passed to the refiner (#3895)
      
      * finish more
      
      * Apply suggestions from code review
      
      * add watermarker
      
      * Model offload (#3889)
      
      * Model offload.
      
      * Model offload for refiner / img2img
      
      * Hardcode encoder offload on img2img vae encode
      
      Saves some GPU RAM in img2img / refiner tasks so it remains below 8 GB.
      
      ---------
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * correct
      
      * fix
      
      * clean print
      
      * Update install warning for `invisible-watermark`
      
      * add: missing docstrings.
      
      * fix and simplify the usage example in img2img.
      
      * fix setup for watermarking.
      
      * Revert "fix setup for watermarking."
      
      This reverts commit 491bc9f5a640bbf46a97a8e52d6eff7e70eb8e4b.
      
      * fix: watermarking setup.
      
      * fix: op.
      
      * run make fix-copies.
      
      * make sure tests pass
      
      * improve convert
      
      * make tests pass
      
      * make tests pass
      
      * better error message
      
      * fiinsh
      
      * finish
      
      * Fix final test
      
      ---------
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      bc9a8cef