1. 18 Dec, 2023 1 commit
  2. 15 Dec, 2023 1 commit
  3. 14 Dec, 2023 1 commit
  4. 07 Dec, 2023 1 commit
  5. 06 Dec, 2023 4 commits
    • UmerHA's avatar
      Add ControlNet-XS support (#5827) · e192ae08
      UmerHA authored
      
      
      * Check in 23-10-05
      
      * check-in 23-10-06
      
      * check-in 23-10-07 2pm
      
      * check-in 23-10-08
      
      * check-in 231009T1200
      
      * check-in 230109
      
      * checkin 231010
      
      * init + forward run
      
      * checkin
      
      * checkin
      
      * ControlNetXSModel is now saveable+loadable
      
      * Forward works
      
      * checkin
      
      * Pipeline works with `no_control=True`
      
      * checkin
      
      * debug: save intermediate outputs of resnet
      
      * checkin
      
      * Understood time error + fixed connection error
      
      * checkin
      
      * checkin 231106T1600
      
      * turned off detailled debug prints
      
      * time debug logs
      
      * small fix
      
      * Separated control_scale for connections/time
      
      * simplified debug logging
      
      * Full denoising works with control scale = 0
      
      * aligned logs
      
      * Added control_attention_head_dim param
      
      * Passing n_heads instead of dim_head into ctrl unet
      
      * Fixed ctrl midblock bug
      
      * Cleanup
      
      * Fixed time dtype bug
      
      * checkin
      
      * 1. from_unet, 2. base passed, 3. all unet params
      
      * checkin
      
      * Finished docstrings
      
      * cleanup
      
      * make style
      
      * checkin
      
      * more tests pass
      
      * Fixed tests
      
      * removed debug logs
      
      * make style + quality
      
      * make fix-copies
      
      * fixed documentation
      
      * added cnxs to doc toc
      
      * added control start/end param
      
      * Update controlnetxs_sdxl.md
      
      * tried to fix copies..
      
      * Fixed norm_num_groups in from_unet
      
      * added sdxl-depth test
      
      * created SD2.1 controlnet-xs pipeline
      
      * re-added debug logs
      
      * Adjusting group norm ; readded logs
      
      * Added debug log statements
      
      * removed debug logs ; started tests for sd2.1
      
      * updated sd21 tests
      
      * fixed tests
      
      * fixed tests
      
      * slightly increased error tolerance for 1 test
      
      * make style & quality
      
      * Added docs for CNXS-SD
      
      * make fix-copies
      
      * Fixed sd compile test ; fixed gradient ckpointing
      
      * vae downs = cnxs conditioning downs; removed guess
      
      * make style & quality
      
      * Fixed tests
      
      * fixed test
      
      * Incorporated review feedback
      
      * simplified control model surgery
      
      * fixed tests & make style / quality
      
      * Updated docs; deleted pip & cursor files
      
      * Rolled back minimal change to resnet
      
      * Update resnet.py
      
      * Update resnet.py
      
      * Update src/diffusers/models/controlnetxs.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/models/controlnetxs.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Incorporated review feedback
      
      * Update docs/source/en/api/pipelines/controlnetxs_sdxl.md
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update docs/source/en/api/pipelines/controlnetxs.md
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update docs/source/en/api/pipelines/controlnetxs.md
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update docs/source/en/api/pipelines/controlnetxs.md
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update src/diffusers/models/controlnetxs.py
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update src/diffusers/models/controlnetxs.py
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update src/diffusers/pipelines/controlnet_xs/pipeline_controlnet_xs.py
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update docs/source/en/api/pipelines/controlnetxs.md
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Update src/diffusers/pipelines/controlnet_xs/pipeline_controlnet_xs_sd_xl.py
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * Incorporated doc feedback
      
      ---------
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      e192ae08
    • Steven Liu's avatar
      [docs] SDXL Turbo (#6065) · 87a09d66
      Steven Liu authored
      api docs
      87a09d66
    • Lucain's avatar
      Harmonize HF environment variables + deprecate use_auth_token (#6066) · 75ada250
      Lucain authored
      * Harmonize HF environment variables + deprecate use_auth_token
      
      * fix import
      
      * fix
      75ada250
    • Sayak Paul's avatar
      [feat] allow SDXL pipeline to run with fused QKV projections (#6030) · a2bc2e14
      Sayak Paul authored
      
      
      * debug
      
      * from step
      
      * print
      
      * turn sigma a list
      
      * make str
      
      * init_noise_sigma
      
      * comment
      
      * remove prints
      
      * feat: introduce fused projections
      
      * change to a better name
      
      * no grad
      
      * device.
      
      * device
      
      * dtype
      
      * okay
      
      * print
      
      * more print
      
      * fix: unbind -> split
      
      * fix: qkv >-> k
      
      * enable disable
      
      * apply attention processor within the method
      
      * attn processors
      
      * _enable_fused_qkv_projections
      
      * remove print
      
      * add fused projection to vae
      
      * add todos.
      
      * add: documentation and cleanups.
      
      * add: test for qkv projection fusion.
      
      * relax assertions.
      
      * relax further
      
      * fix: docs
      
      * fix-copies
      
      * correct error message.
      
      * Empty-Commit
      
      * better conditioning on disable_fused_qkv_projections
      
      * check
      
      * check processor
      
      * bfloat16 computation.
      
      * check latent dtype
      
      * style
      
      * remove copy temporarily
      
      * cast latent to bfloat16
      
      * fix: vae -> self.vae
      
      * remove print.
      
      * add _change_to_group_norm_32
      
      * comment out stuff that didn't work
      
      * Apply suggestions from code review
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * reflect patrick's suggestions.
      
      * fix imports
      
      * fix: disable call.
      
      * fix more
      
      * fix device and dtype
      
      * fix conditions.
      
      * fix more
      
      * Apply suggestions from code review
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      ---------
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      a2bc2e14
  6. 04 Dec, 2023 3 commits
  7. 01 Dec, 2023 1 commit
  8. 29 Nov, 2023 8 commits
    • Steven Liu's avatar
      [docs] Fix video link (#5986) · f72b28c7
      Steven Liu authored
      Update svd.md
      f72b28c7
    • Suraj Patil's avatar
      Fix SVD doc (#5983) · ada8109d
      Suraj Patil authored
      fix url
      ada8109d
    • Patrick von Platen's avatar
      [SDXL Turbo] Add some docs (#5982) · b34acbdc
      Patrick von Platen authored
      
      
      * add diffusers example
      
      * add diffusers example
      
      * Comment about making it faster
      
      * Apply suggestions from code review
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      ---------
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      b34acbdc
    • Suraj Patil's avatar
      Add SVD (#5895) · 63f767ef
      Suraj Patil authored
      
      
      * begin model
      
      * finish blocks
      
      * add_embedding
      
      * addition_time_embed_dim
      
      * use TimestepEmbedding
      
      * fix temporal res block
      
      * fix time_pos_embed
      
      * fix add_embedding
      
      * add conversion script
      
      * fix model
      
      * up
      
      * add new resnet blocks
      
      * make forward work
      
      * return sample in original shape
      
      * fix temb shape in TemporalResnetBlock
      
      * add spatio temporal transformers
      
      * add vae blocks
      
      * fix blocks
      
      * update
      
      * update
      
      * fix shapes in Alphablender and add time activation in res blcok
      
      * use new blocks
      
      * style
      
      * fix temb shape
      
      * fix SpatioTemporalResBlock
      
      * reuse TemporalBasicTransformerBlock
      
      * fix TemporalBasicTransformerBlock
      
      * use TransformerSpatioTemporalModel
      
      * fix TransformerSpatioTemporalModel
      
      * fix time_context dim
      
      * clean up
      
      * make temb optional
      
      * add blocks
      
      * rename model
      
      * update conversion script
      
      * remove UNetMidBlockSpatioTemporal
      
      * add in init
      
      * remove unused arg
      
      * remove unused arg
      
      * remove more unsed args
      
      * up
      
      * up
      
      * check for None
      
      * update vae
      
      * update up/mid blocks for decoder
      
      * begin pipeline
      
      * adapt scheduler
      
      * add guidance scalings
      
      * fix norm eps in temporal transformers
      
      * add temporal autoencoder
      
      * make pipeline run
      
      * fix frame decodig
      
      * decode in float32
      
      * decode n frames at a time
      
      * pass decoding_t to decode_latents
      
      * fix decode_latents
      
      * vae encode/decode in fp32
      
      * fix dtype in TransformerSpatioTemporalModel
      
      * type image_latents same as image_embeddings
      
      * allow using differnt eps in temporal block for video decoder
      
      * fix default values in vae
      
      * pass num frames in decode
      
      * switch spatial to temporal for mixing in VAE
      
      * fix num frames during split decoding
      
      * cast alpha to sample dtype
      
      * fix attention in MidBlockTemporalDecoder
      
      * fix typo
      
      * fix guidance_scales dtype
      
      * fix missing activation in TemporalDecoder
      
      * skip_post_quant_conv
      
      * add vae conversion
      
      * style
      
      * take guidance scale as input
      
      * up
      
      * allow passing PIL to export_video
      
      * accept fps as arg
      
      * add pipeline and vae in init
      
      * remove hack
      
      * use AutoencoderKLTemporalDecoder
      
      * don't scale image latents
      
      * add unet tests
      
      * clean up unet
      
      * clean TransformerSpatioTemporalModel
      
      * add slow svd test
      
      * clean up
      
      * make temb optional in Decoder mid block
      
      * fix norm eps in TransformerSpatioTemporalModel
      
      * clean up temp decoder
      
      * clean up
      
      * clean up
      
      * use c_noise values for timesteps
      
      * use math for log
      
      * update
      
      * fix copies
      
      * doc
      
      * upcast vae
      
      * update forward pass for gradient checkpointing
      
      * make added_time_ids is tensor
      
      * up
      
      * fix upcasting
      
      * remove post quant conv
      
      * add _resize_with_antialiasing
      
      * fix _compute_padding
      
      * cleanup model
      
      * more cleanup
      
      * more cleanup
      
      * more cleanup
      
      * remove freeu
      
      * remove attn slice
      
      * small clean
      
      * up
      
      * up
      
      * remove extra step kwargs
      
      * remove eta
      
      * remove dropout
      
      * remove callback
      
      * remove merge factor args
      
      * clean
      
      * clean up
      
      * move to dedicated folder
      
      * remove attention_head_dim
      
      * docstr and small fix
      
      * update unet doc strings
      
      * rename decoding_t
      
      * correct linting
      
      * store c_skip and c_out
      
      * cleanup
      
      * clean TemporalResnetBlock
      
      * more cleanup
      
      * clean up vae
      
      * clean up
      
      * begin doc
      
      * more cleanup
      
      * up
      
      * up
      
      * doc
      
      * Improve
      
      * better naming
      
      * better naming
      
      * better naming
      
      * better naming
      
      * better naming
      
      * better naming
      
      * better naming
      
      * better naming
      
      * Apply suggestions from code review
      
      * Default chunk size to None
      
      * add example
      
      * Better
      
      * Apply suggestions from code review
      
      * update doc
      
      * Update src/diffusers/pipelines/stable_diffusion_video/pipeline_stable_diffusion_video.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * style
      
      * Get torch compile working
      
      * up
      
      * rename
      
      * fix doc
      
      * add chunking
      
      * torch compile
      
      * torch compile
      
      * add modelling outputs
      
      * torch compile
      
      * Improve chunking
      
      * Apply suggestions from code review
      
      * Update docs/source/en/using-diffusers/svd.md
      
      * Close diff tag
      
      * remove slicing
      
      * resnet docstr
      
      * add docstr in resnet
      
      * rename
      
      * Apply suggestions from code review
      
      * update tests
      
      * Fix output type latents
      
      * fix more
      
      * fix more
      
      * Update docs/source/en/using-diffusers/svd.md
      
      * fix more
      
      * add pipeline tests
      
      * remove unused arg
      
      * clean  up
      
      * make sure get_scaling receives tensors
      
      * fix euler scheduler
      
      * fix get_scalings
      
      * simply euler for now
      
      * remove old test file
      
      * use randn_tensor to create noise
      
      * fix device for rand tensor
      
      * increase expected_max_difference
      
      * fix test_inference_batch_single_identical
      
      * actually fix test_inference_batch_single_identical
      
      * disable test_save_load_float16
      
      * skip test_float16_inference
      
      * skip test_inference_batch_single_identical
      
      * fix test_xformers_attention_forwardGenerator_pass
      
      * Apply suggestions from code review
      
      * update StableVideoDiffusionPipelineSlowTests
      
      * update image
      
      * add diffusers example
      
      * fix more
      
      ---------
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarapolinário <joaopaulo.passos@gmail.com>
      63f767ef
    • vahramtadevosyan's avatar
      [Pipeline] Add TextToVideoZeroSDXLPipeline (#4695) · d63a498c
      vahramtadevosyan authored
      
      
      * integrated sdxl for the text2video-zero pipeline
      
      * make fix-copies
      
      * fixed CI issues
      
      * make fix-copies
      
      * added docs and `copied from` statements
      
      * added fast tests
      
      * made a small change in docs
      
      * quality+style check fix
      
      * updated docs. added controlnet inference with sdxl
      
      * added device compatibility for fast tests
      
      * fixed docstrings
      
      * changing vae upcasting
      
      * remove torch.empty_cache to speed up inference
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * made fast tests to run on dummy models only, fixed copied from statements
      
      * fixed testing utils imports
      
      * Added bullet points for SDXL support
      
      * fixed formatting & quality
      
      * Update tests/pipelines/text_to_video/test_text_to_video_zero_sdxl.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update tests/pipelines/text_to_video/test_text_to_video_zero_sdxl.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * fixed minor error for merging
      
      * fixed updates of sdxl
      
      * made fast tests inherit from `PipelineTesterMixin` and run in 3-4secs on CPU
      
      * make style && make quality
      
      * reimplemented fast tests w/o default attn processor
      
      * make style & make quality
      
      * make fix-copies
      
      * make fix-copies
      
      * fixed docs
      
      * make style & make quality & make fix-copies
      
      * bug fix in cross attention
      
      * make style && make quality
      
      * make fix-copies
      
      * fix gpu issues
      
      * make fix-copies
      
      * updated pipeline signature
      
      ---------
      Co-authored-by: default avatarVahram <vahram.tadevosyan@lambda-loginnode02.cm.cluster>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      d63a498c
    • Steven Liu's avatar
      [docs] LCM training (#5796) · ddd8bd53
      Steven Liu authored
      * first draft
      
      * feedback
      ddd8bd53
    • Charchit Sharma's avatar
      added doc for Kandinsky3.0 (#5937) · 6031ecbd
      Charchit Sharma authored
      
      
      * added en doc for Kandinsky3.0
      
      * required changes
      
      * Update docs/source/en/api/pipelines/kandinsky3.md
      
      * Update docs/source/en/api/pipelines/kandinsky3.md
      
      * Update docs/source/en/api/pipelines/kandinsky3.md
      
      ---------
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      6031ecbd
    • Steven Liu's avatar
      [docs] Update pipeline list (#5952) · 172acc98
      Steven Liu authored
      add to list
      172acc98
  9. 28 Nov, 2023 1 commit
  10. 27 Nov, 2023 2 commits
  11. 24 Nov, 2023 2 commits
  12. 21 Nov, 2023 1 commit
  13. 20 Nov, 2023 2 commits
    • Steven Liu's avatar
      [docs] Loader APIs (#5813) · 7457aa67
      Steven Liu authored
      * first draft
      
      * remove old loader doc
      
      * start adding lora code examples
      
      * finish
      
      * add link to loralinearlayer
      
      * feedback
      
      * fix
      7457aa67
    • M. Tolga Cangöz's avatar
      Revert "[`Docs`] Update and make improvements" (#5858) · c72a1739
      M. Tolga Cangöz authored
      * Revert "[`Docs`] Update and make improvements (#5819)"
      
      This reverts commit c697f524.
      
      * Update README.md
      
      * Update memory.md
      
      * Update basic_training.md
      
      * Update write_own_pipeline.md
      
      * Update fp16.md
      
      * Update basic_training.md
      
      * Update write_own_pipeline.md
      
      * Update write_own_pipeline.md
      c72a1739
  14. 17 Nov, 2023 2 commits
  15. 16 Nov, 2023 2 commits
  16. 15 Nov, 2023 3 commits
  17. 14 Nov, 2023 2 commits
  18. 13 Nov, 2023 3 commits