Commits · be2070991f1b916977020c45ecdfec225de21862 · renzhc / diffusers_dcu

21 Dec, 2024 1 commit

Support Flux IP Adapter (#10261) · be207099

hlky authored Dec 21, 2024



* Flux IP-Adapter

* test cfg

* make style

* temp remove copied from

* fix test

* fix test

* v2

* fix

* make style

* temp remove copied from

* Apply suggestions from code review
Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Move encoder_hid_proj to inside FluxTransformer2DModel

* merge

* separate encode_prompt, add copied from, image_encoder offload

* make

* fix test

* fix

* Update src/diffusers/pipelines/flux/pipeline_flux.py

* test_flux_prompt_embeds change not needed

* true_cfg -> true_cfg_scale

* fix merge conflict

* test_flux_ip_adapter_inference

* add fast test

* FluxIPAdapterMixin not test mixin

* Update pipeline_flux.py
Co-authored-by: YiYi Xu <yixu310@gmail.com>

---------
Co-authored-by: YiYi Xu <yixu310@gmail.com>

be207099

20 Dec, 2024 8 commits

[BUG FIX] [Stable Audio Pipeline] Resolve torch.Tensor.new_zeros() TypeError... · 90200860

Aditya Raj authored Dec 20, 2024


[BUG FIX] [Stable Audio Pipeline] Resolve torch.Tensor.new_zeros() TypeError in function prepare_latents caused by audio_vae_length (#10306)

[BUG FIX] [Stable Audio Pipeline] TypeError: new_zeros(): argument 'size' failed to unpack the object at pos 3 with error "type must be tuple of ints,but got float"

torch.Tensor.new_zeros() takes a single argument size (int...) – a list, tuple, or torch.Size of integers defining the shape of the output tensor.

in function prepare_latents:
audio_vae_length = self.transformer.config.sample_size * self.vae.hop_length
audio_shape = (batch_size // num_waveforms_per_prompt, audio_channels, audio_vae_length)
...
audio = initial_audio_waveforms.new_zeros(audio_shape)

audio_vae_length evaluates to float because self.transformer.config.sample_size returns a float
Co-authored-by: hlky <hlky@hlky.ac>

90200860

docs: fix a mistake in docstring (#10319) · c8ee4af2
Leojc authored Dec 20, 2024
```
Update pipeline_hunyuan_video.py

docs: fix a mistake
```
c8ee4af2

[LoRA] feat: support loading regular Flux LoRAs into Flux Control, and Fill (#10259) · 17128c42

Sayak Paul authored Dec 20, 2024



* lora expansion with dummy zeros.

* updates

* fix working 🥳

* working.

* use torch.device meta for state dict expansion.

* tests
Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com>

* fixes

* fixes

* switch to debug

* fix

* Apply suggestions from code review
Co-authored-by: Aryan <aryan@huggingface.co>

* fix stuff

* docs

---------
Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com>
Co-authored-by: Aryan <aryan@huggingface.co>

17128c42

[Single File] Add GGUF support for LTX (#10298) · dbc1d505
Dhruv Nair authored Dec 20, 2024
```
* update

* add docs.

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
```
dbc1d505
Make tensors in ResNet contiguous for Hunyuan VAE (#10309) · 151b74cd
Aryan authored Dec 20, 2024
```
contiguous tensors in resnet
Co-authored-by: YiYi Xu <yixu310@gmail.com>
```
151b74cd
Add support for sharded models when TorchAO quantization is enabled (#10256) · 41ba8c0b
Aryan authored Dec 20, 2024
```
* add sharded + device_map check
```
41ba8c0b

[WIP] SD3.5 IP-Adapter Pipeline Integration (#9987) · 31912484

Daniel Regado authored Dec 20, 2024




* Added support for single IPAdapter on SD3.5 pipeline



---------
Co-authored-by: hlky <hlky@hlky.ac>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>

31912484

Enable Gradient Checkpointing for UNet2DModel (New) (#7201) · 648d968c

dg845 authored Dec 19, 2024



* Port UNet2DModel gradient checkpointing code from #6718.


---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Vincent Neemie <92559302+VincentNeemie@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
Co-authored-by: hlky <hlky@hlky.ac>

648d968c

19 Dec, 2024 9 commits

unet's `sample_size` attribute is to accept tuple(h, w) in `StableDiffusionPipeline` (#10181) · b756ec6e
djm authored Dec 20, 2024

b756ec6e
Fix `local_files_only` for checkpoints with shards (#10294) · 074798b2
hlky authored Dec 19, 2024

074798b2
Allow Mochi Transformer to be split across multiple GPUs (#10300) · 3ee96695
Dhruv Nair authored Dec 19, 2024
```
update
```
3ee96695
[Single File] Add single file support for Mochi Transformer (#10268) · 9764f229
Dhruv Nair authored Dec 19, 2024
```
update
```
9764f229

[LoRA] Support HunyuanVideo (#10254) · 1826a1e7

Shenghai Yuan authored Dec 19, 2024



* 1217

* 1217

* 1217

* update

* reverse

* add test

* update test

* make style

* update

* make style

---------
Co-authored-by: Aryan <aryan@huggingface.co>

1826a1e7

Check correct model type is passed to `from_pretrained` (#10189) · 0ed09a17

hlky authored Dec 19, 2024



* Check correct model type is passed to `from_pretrained`

* Flax, skip scheduler

* test_wrong_model

* Fix for scheduler

* Update tests/pipelines/test_pipelines.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* EnumMeta

* Flax

* scheduler in expected types

* make

* type object 'CLIPTokenizer' has no attribute '_PipelineFastTests__name'

* support union

* fix typing in kandinsky

* make

* add LCMScheduler

* 'LCMScheduler' object has no attribute 'sigmas'

* tests for wrong scheduler

* make

* update

* warning

* tests

* Update src/diffusers/pipelines/pipeline_utils.py
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* import FlaxSchedulerMixin

* skip scheduler

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

0ed09a17

Update lora_conversion_utils.py (#9980) · 2f7a417d

赵三石 authored Dec 19, 2024



x-flux single-blocks lora load
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>

2f7a417d

Add Flux Control to AutoPipeline (#10292) · 4450d26b
hlky authored Dec 19, 2024

4450d26b
Hunyuan VAE tiling fixes and transformer docs (#10295) · f781b8c3
Aryan authored Dec 19, 2024
```
* update

* udpate

* fix test
```
f781b8c3

18 Dec, 2024 10 commits

[tests] Fix broken cuda, nightly and lora tests on main for CogVideoX (#10270) · c4c99c39
Aryan authored Dec 18, 2024
```
fix joint pos embedding device
```
c4c99c39
[Single File] Add single file support for Flux Canny, Depth and Fill (#10288) · 862a7d50
Dhruv Nair authored Dec 18, 2024
```
update
```
862a7d50
Make zeroing prompt embeds for Mochi Pipeline configurable (#10284) · 8304adce
Dhruv Nair authored Dec 18, 2024
```
update
```
8304adce
Fix sigma_last with use_flow_sigmas (#10267) · e222246b
hlky authored Dec 18, 2024

e222246b

Flux Control(Depth/Canny) + Inpaint (#10192) · 83709d5a

Andrés Romero authored Dec 18, 2024



* flux_control_inpaint - failing test_flux_different_prompts

* removing test_flux_different_prompts?

* fix style

* fix from PR comments

* fix style

* reducing guidance_scale in demo

* Update src/diffusers/pipelines/flux/pipeline_flux_control_inpaint.py
Co-authored-by: hlky <hlky@hlky.ac>

* make

* prepare_latents is not copied from

* update docs

* typos

---------
Co-authored-by: affromero <ubuntu@ip-172-31-17-146.ec2.internal>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: hlky <hlky@hlky.ac>

83709d5a

Support pass kwargs to sd3 custom attention processor (#9818) · 8eb73c87

Qin Zhou authored Dec 18, 2024



* Support pass kwargs to sd3 custom attention processor


---------
Co-authored-by: hlky <hlky@hlky.ac>
Co-authored-by: YiYi Xu <yixu310@gmail.com>

8eb73c87

Make `time_embed_dim` of `UNet2DModel` changeable (#10262) · 88b015dc
Xinyuan Zhao authored Dec 18, 2024

88b015dc
Use `torch` in `get_2d_rotary_pos_embed` (#10155) · 0ac52d6f
hlky authored Dec 18, 2024
```
* Use `torch` in `get_2d_rotary_pos_embed`

* Add deprecation
```
0ac52d6f
[chore] fix: licensing headers in mochi and ltx (#10275) · ba6fd6eb
Sayak Paul authored Dec 18, 2024
```
fix: licensing header.
```
ba6fd6eb

[LoRA] feat: lora support for SANA. (#10234) · 9408aa2d

Sayak Paul authored Dec 18, 2024



* feat: lora support for SANA.

* make fix-copies

* rename test class.

* attention_kwargs -> cross_attention_kwargs.

* Revert "attention_kwargs -> cross_attention_kwargs."

This reverts commit 23433bf9bccc12e0f2f55df26bae58a894e8b43b.

* exhaust 119 max line limit

* sana lora fine-tuning script.

* readme

* add a note about the supported models.

* Apply suggestions from code review
Co-authored-by: Aryan <aryan@huggingface.co>

* style

* docs for attention_kwargs.

* remove lora_scale from pag pipeline.

* copy fix

---------
Co-authored-by: Aryan <aryan@huggingface.co>

9408aa2d

17 Dec, 2024 5 commits

Add `set_shift` to FlowMatchEulerDiscreteScheduler (#10269) · ec1c7a79
hlky authored Dec 17, 2024

ec1c7a79

Fix Mochi Quality Issues (#10033) · 128b96f3

Dhruv Nair authored Dec 17, 2024



* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* Update src/diffusers/models/transformers/transformer_mochi.py
Co-authored-by: Aryan <aryan@huggingface.co>

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Aryan <aryan@huggingface.co>

128b96f3

[Single File] Add GGUF support (#9964) · e24941b2

Dhruv Nair authored Dec 17, 2024



* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* Update src/diffusers/quantizers/gguf/utils.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* Update docs/source/en/quantization/gguf.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* update

* update

* update

* update

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

e24941b2

[LoRA] Support LTX Video (#10228) · ac863934

Aryan authored Dec 17, 2024



* add lora support for ltx

* add tests

* fix copied from comments

* update

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

ac863934

Fix copied from comment in Mochi lora loader (#10255) · 0d96a894
Aryan authored Dec 17, 2024
```
update
```
0d96a894

16 Dec, 2024 7 commits
- [docs] Add missing AttnProcessors (#10246) · 7667cfcb
  Steven Liu authored Dec 16, 2024
```
* attnprocessors

* lora

* make style

* fix

* fix

* sana

* typo
```
  7667cfcb
- [core] TorchAO Quantizer (#10009) · 9f00c617
  Aryan authored Dec 17, 2024
```
* torchao quantizer


---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
```
  9f00c617
- fix downsample bug in MidResTemporalBlock1D (#10250) · aafed3f8
  Kaiwen Sheng authored Dec 16, 2024
  
  aafed3f8
- Add ControlNetUnion to AutoPipeline from_pretrained (#10219) · 5ed761a6
  hlky authored Dec 16, 2024
  
  5ed761a6
- Fix RePaint Scheduler (#10185) · 2f023d7b
  hlky authored Dec 16, 2024
```
Fix repaint scheduler
```
  2f023d7b
- Fix checkpoint in CogView3PlusPipeline example (#10211) · e9a3911b
  hlky authored Dec 16, 2024
  
  e9a3911b
- Add enable_vae_tiling to AllegroPipeline, fix example (#10212) · 7186bb45
  hlky authored Dec 16, 2024
  
  7186bb45