GitLab

* 7529 do not disable autocast for cuda devices

* Remove typecasting error check for non-mps platforms, as a correct autocast implementation makes it a non-issue

* add autocast fix to other training examples

* disable native_amp for dreambooth (sdxl)

* disable native_amp for pix2pix (sdxl)

* remove tests from remaining files

* disable native_amp on huggingface accelerator for every training example that uses it

* convert more usages of autocast to nullcontext, make style fixes

* make style fixes

* style.

* Empty-Commit

---------
Co-authored-by: bghira <bghira@users.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* remove class assignments for linear and conv.

* fix: self.nn

* add from_pipe



---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* Fix SVD bug (shape of `time_context`)

* Formatting code

* Formatting src/diffusers/models/transformers/transformer_temporal.py by `make style && make quality`

---------
Co-authored-by: kevinkhwu <kevinkhwu@tencent.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* fix ip adapter support

* Update sag pipelines tests, adjust sag pipeline to pass tests

---------
Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Add `final_sigma_zero` to UniPCMultistep

Effectively the same trick as DDIM's `set_alpha_to_one` and
DPM's `final_sigma_type='zero'`.
Currently False by default but maybe this should be True?

* `final_sigma_zero: bool` -> `final_sigmas_type: str`

Should 1:1 match DPM Multistep now.

* Set `final_sigmas_type='sigma_min'` in UniPC UTs

Fixed important typo

* Initial commit

* Implemented block lora

- implemented block lora
- updated docs
- added tests

* Finishing up

* Reverted unrelated changes made by make style

* Fixed typo

* Fixed bug + Made text_encoder_2 scalable

* Integrated some review feedback

* Incorporated review feedback

* Fix tests

* Made every module configurable

* Adapter to new lora test structure

* Final cleanup

* Some more final fixes

- Included examples in `using_peft_for_inference.md`
- Added hint that only attns are scaled
- Removed NoneTypes
- Added test to check mismatching lens of adapter names / weights raise error

* Update using_peft_for_inference.md

* Update using_peft_for_inference.md

* Make style, quality, fix-copies

* Updated tutorial;Warning if scale/adapter mismatch

* floats are forwarded as-is; changed tutorial scale

* make style, quality, fix-copies

* Fixed typo in tutorial

* Moved some warnings into `lora_loader_utils.py`

* Moved scale/lora mismatch warnings back

* Integrated final review suggestions

* Empty commit to trigger CI

* Reverted emoty commit to trigger CI

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* speed up test_vae_slicing in animatediff

* speed up test_karras_schedulers_shape for attend and excite.

* style.

* get the static slices out.

* specify torch print options.

* modify

* test run with controlnet

* specify kwarg

* fix: things

* not None

* flatten

* controlnet img2img

* complete controlet sd

* finish more

* finish more

* finish more

* finish more

* finish the final batch

* add cpu check for expected_pipe_slice.

* finish the rest

* remove print

* style

* fix ssd1b controlnet test

* checking ssd1b

* disable the test.

* make the test_ip_adapter_single controlnet test more robust

* fix: simple inpaint

* multi

* disable panorama

* enable again

* panorama is shaky so leave it for now

* remove print

* raise tolerance.

* Bug fix for controlnetpipeline check_image

Bug fix for controlnetpipeline check_image when using multicontrolnet and prompt list

* Update test_inference_multiple_prompt_input function

* Update test_controlnet.py

add test for multiple prompts and multiple image conditioning

* Update test_controlnet.py

Fix format error

---------
Co-authored-by: Lvkesheng Shen <45848260+Fantast416@users.noreply.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* add remove_all_hooks

* a few more fix and tests

* up

* Update src/diffusers/pipelines/pipeline_utils.py
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

* split tests

* add

---------
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

import load_model_dict_into_meta only once

fix
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

diffusers#7426 fix stable diffusion xl inference on MPS when dtypes shift unexpectedly due to pytorch bugs (#7446)

* mps: fix XL pipeline inference at training time due to upstream pytorch bug

* Update src/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* apply the safe-guarding logic elsewhere.

---------
Co-authored-by: bghira <bghira@users.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Fix typos

* Add docstring to `decode` method in `ConsistencyDecoderVAE`

* Fix tiling

* Enable tiled VAE decoding with customizable tile sample size and overlap factor

* Revert "Enable tiled VAE decoding with customizable tile sample size and overlap factor"

This reverts commit 181049675e83cea7b33ae2bbeba2aff7ae1b1761.

* Add VAE tiling test for `ConsistencyDecoderVAE`

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* feat: support dora loras from community

* safe-guard dora operations under peft version.

* pop use_dora when False

* make dora lora from kohya work.

* fix: kohya conversion utils.

* add a fast test for DoRA compatibility..

* add a nightly test.

skip dynamo tests when python is 3.12.

* Fix typos

* Fix typos

* Fix typos

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

update

[`IP-Adapter`] Fix IP-Adapter Support and Refactor Callback for `StableDiffusionPanoramaPipeline` (#7262)

* Add properties and `IPAdapterTesterMixin` tests for `StableDiffusionPanoramaPipeline`

* Update torch manual seed to use `torch.Generator(device=device)`

* Refactor 📞🔙 to support `callback_on_step_end`

* make fix-copies

* strtobool

* replace Command from setuptools.

* fix freeinit impl

* fix progress bar

* fix progress bar and remove old code

* fix num_inference_steps==1 case for freeinit by atleast running 1 step when fast sampling enabled

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Fix typos

* Fix typo in SVD.md

* checking to improve pipelines.

* more fixes.

* add: tip to encourage the usage of revision

* Apply suggestions from code review

* retrigger ci

---------
Co-authored-by: YiYi Xu <yixu310@gmail.com>

* cleanse and refactor lora testing suite.

* more cleanup.

* make check_if_lora_correctly_set a utility function

* fix: typo

* retrigger ci

* style

update

* Change path to posix

* running isort

* run style and quality checks

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Fix ControlNetModel.from_unet do not load add_embedding

* delete white space in blank line

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update pipeline_controlnet_sd_xl_img2img.py

fix: safetensors load error

* fix for pass test

---------
Co-authored-by: YiYi Xu <yixu310@gmail.com>

Fix issue with prompt embeds and latents in SD Cascade Decoder with multiple image embeddings for a single prompt.  (#7381)

* fix

* update

* update

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* debugging

* let's see the numbers

* let's see the numbers

* let's see the numbers

* restrict tolerance.

* increase inference steps.

* shallow copy of cross_attentionkwargs

* remove print

* fix

* fix

* add a tests

* fix

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: yiyixuxu <yixu310@gmail,com>

* Correction for non-integral image resolutions with quantizations other than float32.

* Support for training, and use of diffusers-style casting.

* add missing copied from statements in tcd scheduler

* update docstring

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* pop scale from the top-level unet instead of getting it.

* improve readability.

* Apply suggestions from code review
Co-authored-by: YiYi Xu <yixu310@gmail.com>

* fix a little bit.

---------
Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Add Cascade to Auto T2I + Decoder mappings

* ruff autofix

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>