Commits · 28f48f4051e80082cbe97f2d62b365dbb01040ec · renzhc / diffusers_dcu

12 Feb, 2025 3 commits

[Single File] Add Single File support for Lumina Image 2.0 Transformer (#10781) · 28f48f40
Dhruv Nair authored Feb 12, 2025
```
* update

* update
```
28f48f40

Thanh Le authored Feb 12, 2025



* Update peft_utils.py

* Update peft_utils.py

* Update peft_utils.py

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

067eab1b

Refactor OmniGen (#10771) · 57ac6738

Aryan authored Feb 12, 2025



* OmniGen model.py

* update OmniGenTransformerModel

* omnigen pipeline

* omnigen pipeline

* update omnigen_pipeline

* test case for omnigen

* update omnigenpipeline

* update docs

* update docs

* offload_transformer

* enable_transformer_block_cpu_offload

* update docs

* reformat

* reformat

* reformat

* update docs

* update docs

* make style

* make style

* Update docs/source/en/api/models/omnigen_transformer.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/using-diffusers/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/using-diffusers/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* update docs

* revert changes to examples/

* update OmniGen2DModel

* make style

* update test cases

* Update docs/source/en/api/pipelines/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/using-diffusers/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/using-diffusers/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/using-diffusers/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/using-diffusers/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/using-diffusers/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* update docs

* typo

* Update src/diffusers/models/embeddings.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/models/attention.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/models/transformers/transformer_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/models/transformers/transformer_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/models/transformers/transformer_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/pipelines/omnigen/pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/pipelines/omnigen/pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/pipelines/omnigen/pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update tests/pipelines/omnigen/test_pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update tests/pipelines/omnigen/test_pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/pipelines/omnigen/pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/pipelines/omnigen/pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/pipelines/omnigen/pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* consistent attention processor

* updata

* update

* check_inputs

* make style

* update testpipeline

* update testpipeline

* refactor omnigen

* more updates

* apply review suggestion

---------
Co-authored-by: shitao <2906698981@qq.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: hlky <hlky@hlky.ac>

57ac6738

11 Feb, 2025 6 commits

Add support for lumina2 (#10642) · 81440fd4

Le Zhuo authored Feb 12, 2025



* Add support for lumina2


---------
Co-authored-by: csuhan <hanjiaming@whu.edu.cn>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Aryan <aryan@huggingface.co>
Co-authored-by: hlky <hlky@hlky.ac>

81440fd4

feat: new community mixture_tiling_sdxl pipeline for SDXL (#10759) · c4702748

Eliseu Silva authored Feb 11, 2025

* feat: new community mixture_tiling_sdxl pipeline for SDXL mixture-of-diffusers support

* fix use of variable latents to tile_latents

* removed references to modules that are not being used in this pipeline

* make style, make quality

c4702748

Add OmniGen (#10148) · 798e1718

Shitao Xiao authored Feb 12, 2025



* OmniGen model.py

* update OmniGenTransformerModel

* omnigen pipeline

* omnigen pipeline

* update omnigen_pipeline

* test case for omnigen

* update omnigenpipeline

* update docs

* update docs

* offload_transformer

* enable_transformer_block_cpu_offload

* update docs

* reformat

* reformat

* reformat

* update docs

* update docs

* make style

* make style

* Update docs/source/en/api/models/omnigen_transformer.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/using-diffusers/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/using-diffusers/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* update docs

* revert changes to examples/

* update OmniGen2DModel

* make style

* update test cases

* Update docs/source/en/api/pipelines/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/using-diffusers/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/using-diffusers/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/using-diffusers/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/using-diffusers/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/using-diffusers/omnigen.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* update docs

* typo

* Update src/diffusers/models/embeddings.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/models/attention.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/models/transformers/transformer_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/models/transformers/transformer_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/models/transformers/transformer_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/pipelines/omnigen/pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/pipelines/omnigen/pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/pipelines/omnigen/pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update tests/pipelines/omnigen/test_pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update tests/pipelines/omnigen/test_pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/pipelines/omnigen/pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/pipelines/omnigen/pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* Update src/diffusers/pipelines/omnigen/pipeline_omnigen.py
Co-authored-by: hlky <hlky@hlky.ac>

* consistent attention processor

* updata

* update

* check_inputs

* make style

* update testpipeline

* update testpipeline

---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: hlky <hlky@hlky.ac>
Co-authored-by: Aryan <aryan@huggingface.co>

798e1718

[CI] Fix Truffle Hog failure (#10769) · ed4b7522
Dhruv Nair authored Feb 11, 2025
```
* update

* update
```
ed4b7522
speedup hunyuan encoder causal mask generation (#10764) · 8ae8008b
Mathias Parger authored Feb 11, 2025
```
* speedup causal mask generation

* fixing hunyuan attn mask test case
```
8ae8008b
[Tests] Test layerwise casting with training (#10765) · c80eda9d
Sayak Paul authored Feb 11, 2025
```
* add a test to check if we can train with layerwise casting.

* updates

* updates

* style
```
c80eda9d

10 Feb, 2025 2 commits
- Add `Self` type hint to `ModelMixin`'s `from_pretrained` (#10742) · 7fb481f8
  hlky authored Feb 10, 2025
  
  7fb481f8
- [LoRA] fix peft state dict parsing (#10532) · 9f5ad1db
  Sayak Paul authored Feb 10, 2025
```
* fix peft state dict parsing

* updates
```
  9f5ad1db
07 Feb, 2025 1 commit
- EDMEulerScheduler accept sigmas, add final_sigmas_type (#10734) · 464374fb
  hlky authored Feb 07, 2025
  
  464374fb
06 Feb, 2025 3 commits

Quantized Flux with IP-Adapter (#10728) · d43ce14e
hlky authored Feb 06, 2025

d43ce14e

[bugfix] NPU Adaption for Sana (#10724) · cd0a4a82

Leo Jiang authored Feb 06, 2025



* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* NPU Adaption for Sanna

* [bugfix]NPU Adaption for Sanna

---------
Co-authored-by: J石页 <jiangshuo9@h-partners.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

cd0a4a82

[Community] Enhanced `Model Search` (#10417) · 145522cb

suzukimain authored Feb 06, 2025

* Added `auto_load_textual_inversion` and `auto_load_lora_weights`

* update README.md

* fix

* make quality

* Fix and `make style`

145522cb

05 Feb, 2025 1 commit
- add provider_options in from_pretrained (#10719) · 23bc56a0
  xieofxie authored Feb 06, 2025
```
Co-authored-by: hualxie <hualxie@microsoft.com>
```
  23bc56a0
04 Feb, 2025 4 commits
- [Fix] Type Hint in from_pretrained() to Ensure Correct Type Inference (#10714) · 5b1dcd15
  SahilCarterr authored Feb 05, 2025
```
* Update pipeline_utils.py

Added Self in from_pretrained method so  inference will correctly recognize pipeline

* Use typing_extensions

---------
Co-authored-by: hlky <hlky@hlky.ac>
```
  5b1dcd15
- Notebooks for Community Scripts-6 (#10713) · dbe0094e
  Parag Ekbote authored Feb 04, 2025
```
* Fix Doc Tutorial.

* Add 4 Notebooks and improve their example
scripts.
```
  dbe0094e
- Fix train_text_to_image.py --help (#10711) · f63d3223
  Nicolas authored Feb 03, 2025
  
  f63d3223
- [bitsandbytes] Simplify bnb int8 dequant (#10401) · 5e8e6cb4
  Sayak Paul authored Feb 04, 2025
```
* fix dequantization for latest bnb.

* smol fixes.

* fix type annotation

* update peft link

* updates
```
  5e8e6cb4
03 Feb, 2025 2 commits
- Fix Documentation about Image-to-Image Pipeline (#10704) · 3e35f56b
  Parag Ekbote authored Feb 03, 2025
```
Fix Doc Tutorial.
```
  3e35f56b
- Fixed grammar in "write_own_pipeline" readme (#10706) · 537891e6
  Ikpreet S Babra authored Feb 03, 2025
  
  537891e6
01 Feb, 2025 1 commit

feat(training-utils): support device and dtype params in... · 9f28f1ab

Vedat Baday authored Feb 02, 2025


feat(training-utils): support device and dtype params in compute_density_for_timestep_sampling (#10699)

* feat(training-utils): support device and dtype params in compute_density_for_timestep_sampling

* chore: update type hint

* refactor: use union for type hint

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

9f28f1ab

31 Jan, 2025 2 commits

Fix inconsistent random transform in instruct pix2pix (#10698) · 5d2d2398

Thanh Le authored Jan 31, 2025

* Update train_instruct_pix2pix.py

Fix inconsistent random transform in instruct_pix2pix

* Update train_instruct_pix2pix_sdxl.py

5d2d2398

Fix enable memory efficient attention on ROCm (#10564) · 1ae9b059

Max Podkorytov authored Jan 31, 2025

* fix enable memory efficient attention on ROCm

while calling CK implementation

* Update attention_processor.py

refactor of picking a set element

1ae9b059

29 Jan, 2025 4 commits
- [FIX] check_inputs function in Auraflow Pipeline (#10678) · aad69ac2
  SahilCarterr authored Jan 30, 2025
```
fix_shape_error
```
  aad69ac2
- fix(hunyuan-video): typo in height and width input check (#10684) · ea76880b
  Vedat Baday authored Jan 30, 2025
  
  ea76880b
- support StableDiffusionAdapterPipeline.from_single_file (#10552) · 33f93615
  Teriks authored Jan 29, 2025
```
* support StableDiffusionAdapterPipeline.from_single_file

* make style

---------
Co-authored-by: Teriks <Teriks@users.noreply.github.com>
Co-authored-by: hlky <hlky@hlky.ac>
```
  33f93615
- [tests] update llamatokenizer in hunyuanvideo tests (#10681) · e6037e82
  Sayak Paul authored Jan 29, 2025
```
update llamatokenizer in hunyuanvideo tests
```
  e6037e82
28 Jan, 2025 4 commits

Fix pipeline dtype unexpected change when using SDXL reference community... · 196aef5a

Dimitri Barbot authored Jan 28, 2025

Fix pipeline dtype unexpected change when using SDXL reference community pipelines in float16 mode (#10670)

Fix pipeline dtype unexpected change when using SDXL reference community pipelines

196aef5a

[Tests] conditionally check `fp8_e4m3_bf16_max_memory < fp8_e4m3_fp32_max_memory` (#10669) · 7b100ce5
Sayak Paul authored Jan 28, 2025
```
* conditionally check if compute capability is met.

* log info.

* fix condition.

* updates

* updates

* updates

* updates
```
7b100ce5

Refactor gradient checkpointing (#10611) · c4d4ac21

Aryan authored Jan 28, 2025

* update

* remove unused fn

* apply suggestions based on review

* update + cleanup 🧹

* more cleanup 🧹

* make fix-copies

* update test

c4d4ac21

[fix] refer use_framewise_encoding on AutoencoderKLHunyuanVideo._encode (#10600) · f295e2ee

Hanch Han authored Jan 28, 2025



* fix: refer to use_framewise_encoding on AutoencoderKLHunyuanVideo._encode

* fix: comment about tile_sample_min_num_frames

---------
Co-authored-by: Aryan <aryan@huggingface.co>

f295e2ee

27 Jan, 2025 7 commits

[core] Pyramid Attention Broadcast (#9562) · 658e24e8

Aryan authored Jan 28, 2025



* start pyramid attention broadcast

* add coauthor
Co-Authored-By: Xuanlei Zhao <43881818+oahzxl@users.noreply.github.com>

* update

* make style

* update

* make style

* add docs

* add tests

* update

* Update docs/source/en/api/pipelines/cogvideox.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/api/pipelines/cogvideox.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Pyramid Attention Broadcast rewrite + introduce hooks (#9826)

* rewrite implementation with hooks

* make style

* update

* merge pyramid-attention-rewrite-2

* make style

* remove changes from latte transformer

* revert docs changes

* better debug message

* add todos for future

* update tests

* make style

* cleanup

* fix

* improve log message; fix latte test

* refactor

* update

* update

* update

* revert changes to tests

* update docs

* update tests

* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* update

* fix flux test

* reorder

* refactor

* make fix-copies

* update docs

* fixes

* more fixes

* make style

* update tests

* update code example

* make fix-copies

* refactor based on reviews

* use maybe_free_model_hooks

* CacheMixin

* make style

* update

* add current_timestep property; update docs

* make fix-copies

* update

* improve tests

* try circular import fix

* apply suggestions from review

* address review comments

* Apply suggestions from code review

* refactor hook implementation

* add test suite for hooks

* PAB Refactor (#10667)

* update

* update

* update

---------
Co-authored-by: DN6 <dhruv.nair@gmail.com>

* update

* fix remove hook behaviour

---------
Co-authored-by: Xuanlei Zhao <43881818+oahzxl@users.noreply.github.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: DN6 <dhruv.nair@gmail.com>

658e24e8

Revert RePaint scheduler 'fix' (#10644) · fb420664
Giuseppe Catalano authored Jan 27, 2025
```
Co-authored-by: Giuseppe Catalano <giuseppelorenzo.catalano@unito.it>
```
fb420664

SDXL ControlNet Union pipelines, make control_image argument immutible (#10663) · e89ab5bc

Teriks authored Jan 27, 2025



controlnet union XL, make control_image immutible

when this argument is passed a list, __call__
modifies its content, since it is pass by reference
the list passed by the caller gets its content
modified unexpectedly

make a copy at method intro so this does not happen
Co-authored-by: Teriks <Teriks@users.noreply.github.com>

e89ab5bc

fix check_inputs func in LuminaText2ImgPipeline (#10651) · 8ceec90d
victolee0 authored Jan 28, 2025

8ceec90d
Add provider_options to OnnxRuntimeModel (#10661) · 158c5c4d
hlky authored Jan 27, 2025

158c5c4d
[training] Convert to ImageFolder script (#10664) · 41571773
hlky authored Jan 27, 2025
```
* [training] Convert to ImageFolder script

* make
```
41571773
ControlNet Union controlnet_conditioning_scale for multiple control inputs (#10666) · 18f7d1d9
hlky authored Jan 27, 2025

18f7d1d9