1. 22 Nov, 2024 1 commit
    • Fanli Lin's avatar
      make `pipelines` tests device-agnostic (part1) (#9399) · 64b3e0f5
      Fanli Lin authored
      
      
      * enable on xpu
      
      * add 1 more
      
      * add one more
      
      * enable more
      
      * add 1 more
      
      * add more
      
      * enable 1
      
      * enable more cases
      
      * enable
      
      * enable
      
      * update comment
      
      * one more
      
      * enable 1
      
      * add more cases
      
      * enable xpu
      
      * add one more caswe
      
      * add more cases
      
      * add 1
      
      * add more
      
      * add more cases
      
      * add case
      
      * enable
      
      * add more
      
      * add more
      
      * add more
      
      * enbale more
      
      * add more
      
      * update code
      
      * update test marker
      
      * add skip back
      
      * update comment
      
      * remove single files
      
      * remove
      
      * style
      
      * add
      
      * revert
      
      * reformat
      
      * update decorator
      
      * update
      
      * update
      
      * update
      
      * Update tests/pipelines/deepfloyd_if/test_if.py
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      
      * Update src/diffusers/utils/testing_utils.py
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      
      * Update tests/pipelines/animatediff/test_animatediff_controlnet.py
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      
      * Update tests/pipelines/animatediff/test_animatediff.py
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      
      * Update tests/pipelines/animatediff/test_animatediff_controlnet.py
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      
      * update float16
      
      * no unitest.skipt
      
      * update
      
      * apply style check
      
      * reapply format
      
      ---------
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      64b3e0f5
  2. 31 Oct, 2024 1 commit
    • Sayak Paul's avatar
      [CI] add a big GPU marker to run memory-intensive tests separately on CI (#9691) · ff182ad6
      Sayak Paul authored
      
      
      * add a marker for big gpu tests
      
      * update
      
      * trigger on PRs temporarily.
      
      * onnx
      
      * fix
      
      * total memory
      
      * fixes
      
      * reduce memory threshold.
      
      * bigger gpu
      
      * empty
      
      * g6e
      
      * Apply suggestions from code review
      
      * address comments.
      
      * fix
      
      * fix
      
      * fix
      
      * fix
      
      * fix
      
      * okay
      
      * further reduce.
      
      * updates
      
      * remove
      
      * updates
      
      * updates
      
      * updates
      
      * updates
      
      * fixes
      
      * fixes
      
      * updates.
      
      * fix
      
      * workflow fixes.
      
      ---------
      Co-authored-by: default avatarAryan <aryan@huggingface.co>
      ff182ad6
  3. 23 Oct, 2024 1 commit
  4. 21 Oct, 2024 1 commit
    • Sayak Paul's avatar
      [Quantization] Add quantization support for `bitsandbytes` (#9213) · b821f006
      Sayak Paul authored
      * quantization config.
      
      * fix-copies
      
      * fix
      
      * modules_to_not_convert
      
      * add bitsandbytes utilities.
      
      * make progress.
      
      * fixes
      
      * quality
      
      * up
      
      * up
      
      rotary embedding refactor 2: update comments, fix dtype for use_real=False (#9312)
      
      fix notes and dtype
      
      up
      
      up
      
      * minor
      
      * up
      
      * up
      
      * fix
      
      * provide credits where due.
      
      * make configurations work.
      
      * fixes
      
      * fix
      
      * update_missing_keys
      
      * fix
      
      * fix
      
      * make it work.
      
      * fix
      
      * provide credits to transformers.
      
      * empty commit
      
      * handle to() better.
      
      * tests
      
      * change to bnb from bitsandbytes
      
      * fix tests
      
      fix slow quality tests
      
      SD3 remark
      
      fix
      
      complete int4 tests
      
      add a readme to the test files.
      
      add model cpu offload tests
      
      warning test
      
      * better safeguard.
      
      * change merging status
      
      * courtesy to transformers.
      
      * move  upper.
      
      * better
      
      * make the unused kwargs warning friendlier.
      
      * harmonize changes with https://github.com/huggingface/transformers/pull/33122
      
      
      
      * style
      
      * trainin tests
      
      * feedback part i.
      
      * Add Flux inpainting and Flux Img2Img (#9135)
      
      ---------
      Co-authored-by: default avataryiyixuxu <yixu310@gmail.com>
      
      Update `UNet2DConditionModel`'s error messages (#9230)
      
      * refactor
      
      [CI] Update Single file Nightly Tests (#9357)
      
      * update
      
      * update
      
      feedback.
      
      improve README for flux dreambooth lora (#9290)
      
      * improve readme
      
      * improve readme
      
      * improve readme
      
      * improve readme
      
      fix one uncaught deprecation warning for accessing vae_latent_channels in VaeImagePreprocessor (#9372)
      
      deprecation warning vae_latent_channels
      
      add mixed int8 tests and more tests to nf4.
      
      [core] Freenoise memory improvements (#9262)
      
      * update
      
      * implement prompt interpolation
      
      * make style
      
      * resnet memory optimizations
      
      * more memory optimizations; todo: refactor
      
      * update
      
      * update animatediff controlnet with latest changes
      
      * refactor chunked inference changes
      
      * remove print statements
      
      * update
      
      * chunk -> split
      
      * remove changes from incorrect conflict resolution
      
      * remove changes from incorrect conflict resolution
      
      * add explanation of SplitInferenceModule
      
      * update docs
      
      * Revert "update docs"
      
      This reverts commit c55a50a271b2cefa8fe340a4f2a3ab9b9d374ec0.
      
      * update docstring for freenoise split inference
      
      * apply suggestions from review
      
      * add tests
      
      * apply suggestions from review
      
      quantization docs.
      
      docs.
      
      * Revert "Add Flux inpainting and Flux Img2Img (#9135)"
      
      This reverts commit 5799954dd4b3d753c7c1b8d722941350fe4f62ca.
      
      * tests
      
      * don
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * contribution guide.
      
      * changes
      
      * empty
      
      * fix tests
      
      * harmonize with https://github.com/huggingface/transformers/pull/33546
      
      .
      
      * numpy_cosine_distance
      
      * config_dict modification.
      
      * remove if config comment.
      
      * note for load_state_dict changes.
      
      * float8 check.
      
      * quantizer.
      
      * raise an error for non-True low_cpu_mem_usage values when using quant.
      
      * low_cpu_mem_usage shenanigans when using fp32 modules.
      
      * don't re-assign _pre_quantization_type.
      
      * make comments clear.
      
      * remove comments.
      
      * handle mixed types better when moving to cpu.
      
      * add tests to check if we're throwing warning rightly.
      
      * better check.
      
      * fix 8bit test_quality.
      
      * handle dtype more robustly.
      
      * better message when keep_in_fp32_modules.
      
      * handle dtype casting.
      
      * fix dtype checks in pipeline.
      
      * fix warning message.
      
      * Update src/diffusers/models/modeling_utils.py
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      
      * mitigate the confusing cpu warning
      
      ---------
      Co-authored-by: default avatarVishnu V Jaddipal <95531133+Gothos@users.noreply.github.com>
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      b821f006
  5. 09 Oct, 2024 1 commit
  6. 02 Oct, 2024 1 commit
  7. 26 Jun, 2024 1 commit
  8. 15 May, 2024 1 commit
    • Isamu Isozaki's avatar
      Adding VQGAN Training script (#5483) · d27e996c
      Isamu Isozaki authored
      
      
      * Init commit
      
      * Removed einops
      
      * Added default movq config for training
      
      * Update explanation of prompts
      
      * Fixed inheritance of discriminator and init_tracker
      
      * Fixed incompatible api between muse and here
      
      * Fixed output
      
      * Setup init training
      
      * Basic structure done
      
      * Removed attention for quick tests
      
      * Style fixes
      
      * Fixed vae/vqgan styles
      
      * Removed redefinition of wandb
      
      * Fixed log_validation and tqdm
      
      * Nothing commit
      
      * Added commit loss to lookup_from_codebook
      
      * Update src/diffusers/models/vq_model.py
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Adding perliminary README
      
      * Fixed one typo
      
      * Local changes
      
      * Fixed main issues
      
      * Merging
      
      * Update src/diffusers/models/vq_model.py
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Testing+Fixed bugs in training script
      
      * Some style fixes
      
      * Added wandb to docs
      
      * Fixed timm test
      
      * get testing suite ready.
      
      * remove return loss
      
      * remove return_loss
      
      * Remove diffs
      
      * Remove diffs
      
      * fix ruff format
      
      ---------
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      d27e996c
  9. 10 Apr, 2024 1 commit
    • Sayak Paul's avatar
      [Core] add "balanced" `device_map` support to pipelines (#6857) · 3e4a6bd2
      Sayak Paul authored
      
      
      * get device <-> component mapping when using multiple gpus.
      
      * condition the device_map bits.
      
      * relax condition
      
      * device_map progress.
      
      * device_map enhancement
      
      * some cleaning up and debugging
      
      * Apply suggestions from code review
      Co-authored-by: default avatarMarc Sun <57196510+SunMarc@users.noreply.github.com>
      
      * incorporate suggestions from PR.
      
      * remove multi-gpu condition for now.
      
      * guard check the component -> device mapping
      
      * fix: device_memory variable
      
      * dispatching transformers model to have force_hooks=True
      
      * better guarding for transformers device_map
      
      * introduce support balanced_low_memory and balanced_ultra_low_memory.
      
      * remove device_map patch.
      
      * fix: intermediate variable scoping.
      
      * fix: condition in cpu offload.
      
      * fix: flax class restrictions.
      
      * remove modifications from cpu_offload and model_offload
      
      * incorporate changes.
      
      * add a simple forward pass test
      
      * add: torch_device in get_inputs()
      
      * add: tests
      
      * remove print
      
      * safe-guard to(), model offloading and cpu offloading when balanced is used as a device_map.
      
      * style
      
      * remove .
      
      * safeguard device_map with more checks and remove invalid device_mapping strategues.
      
      * make  a class attribute and adjust tests accordingly.
      
      * fix device_map check
      
      * fix test
      
      * adjust comment
      
      * fix: device_map attribute
      
      * fix: dispatching.
      
      * max_memory test for pipeline
      
      * version guard the tests
      
      * fix guard.
      
      * address review feedback.
      
      * reset_device_map method.
      
      * add: test for reset_hf_device_map
      
      * fix a couple things.
      
      * add reset_device_map() in the error message.
      
      * add tests for checking reset_device_map doesn't have unintended consequences.
      
      * fix reset_device_map and offloading tests.
      
      * create _get_final_device_map utility.
      
      * hf_device_map -> _hf_device_map
      
      * add documentation
      
      * add notes suggested by Marc.
      
      * styling.
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * move updates within gpu condition.
      
      * other docs related things
      
      * note on ignore a device not specified in .
      
      * provide a suggestion if device mapping errors out.
      
      * fix: typo.
      
      * _hf_device_map -> hf_device_map
      
      * Empty-Commit
      
      * add: example hf_device_map.
      
      ---------
      Co-authored-by: default avatarMarc Sun <57196510+SunMarc@users.noreply.github.com>
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      3e4a6bd2
  10. 02 Apr, 2024 1 commit
    • Sayak Paul's avatar
      add: utility to format our docs too 📜 (#7314) · 4a343077
      Sayak Paul authored
      * add: utility to format our docs too 📜
      
      * debugging saga
      
      * fix: message
      
      * checking
      
      * should be fixed.
      
      * revert pipeline_fixture
      
      * remove empty line
      
      * make style
      
      * fix: setup.py
      
      * style.
      4a343077
  11. 29 Mar, 2024 1 commit
    • Sayak Paul's avatar
      [Tests] Speed up some fast pipeline tests (#7477) · fac76169
      Sayak Paul authored
      * speed up test_vae_slicing in animatediff
      
      * speed up test_karras_schedulers_shape for attend and excite.
      
      * style.
      
      * get the static slices out.
      
      * specify torch print options.
      
      * modify
      
      * test run with controlnet
      
      * specify kwarg
      
      * fix: things
      
      * not None
      
      * flatten
      
      * controlnet img2img
      
      * complete controlet sd
      
      * finish more
      
      * finish more
      
      * finish more
      
      * finish more
      
      * finish the final batch
      
      * add cpu check for expected_pipe_slice.
      
      * finish the rest
      
      * remove print
      
      * style
      
      * fix ssd1b controlnet test
      
      * checking ssd1b
      
      * disable the test.
      
      * make the test_ip_adapter_single controlnet test more robust
      
      * fix: simple inpaint
      
      * multi
      
      * disable panorama
      
      * enable again
      
      * panorama is shaky so leave it for now
      
      * remove print
      
      * raise tolerance.
      fac76169
  12. 26 Mar, 2024 1 commit
  13. 25 Mar, 2024 2 commits
  14. 05 Feb, 2024 1 commit
  15. 03 Feb, 2024 1 commit
  16. 02 Feb, 2024 1 commit
  17. 18 Jan, 2024 1 commit
  18. 16 Jan, 2024 1 commit
  19. 26 Dec, 2023 1 commit
  20. 15 Dec, 2023 1 commit
  21. 05 Dec, 2023 1 commit
    • Arsalan's avatar
      Device agnostic testing (#5612) · f427345a
      Arsalan authored
      * utils and test modifications to enable device agnostic testing
      
      * device for manual seed in unet1d
      
      * fix generator condition in vae test
      
      * consistency changes to testing
      
      * make style
      
      * add device agnostic testing changes to source and one model test
      
      * make dtype check fns private, log cuda fp16 case
      
      * remove dtype checks from import utils, move to testing_utils
      
      * adding tests for most model classes and one pipeline
      
      * fix vae import
      f427345a
  22. 26 Sep, 2023 1 commit
  23. 25 Sep, 2023 1 commit
  24. 22 Sep, 2023 1 commit
  25. 25 Aug, 2023 1 commit
  26. 22 Aug, 2023 1 commit
    • Alex McKinney's avatar
      Replaces `DIFFUSERS_TEST_DEVICE` backend list with trying device (#4673) · e34d9aa6
      Alex McKinney authored
      This is a better method than comparing against a list of supported backends as it allows for supporting any number of backends provided they are installed on the user's system.
      This should have no effect on the behaviour of tests in Huggingface's CI workers.
      See transformers#25506 where this approach has already been added.
      e34d9aa6
  27. 20 Jul, 2023 1 commit
  28. 06 Jul, 2023 1 commit
    • YiYi Xu's avatar
      Add Shap-E (#3742) · 45f6d52b
      YiYi Xu authored
      
      
      * refactor prior_transformer
      
      adding conversion script
      
      add pipeline
      
      add step_index from pipeline, + remove permute
      
      add zero pad token
      
      remove copy from statement for betas_for_alpha_bar function
      
      * add
      
      * add
      
      * update conversion script for renderer model
      
      * refactor camera a little bit
      
      * clean up
      
      * style
      
      * fix copies
      
      * Update src/diffusers/schedulers/scheduling_heun_discrete.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/shap_e/pipeline_shap_e.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/shap_e/pipeline_shap_e.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * alpha_transform_type
      
      * remove step_index argument
      
      * remove get_sigmas_karras
      
      * remove _yiyi_sigma_to_t
      
      * move the rescale prompt_embeds from prior_transformer to pipeline
      
      * replace baddbmm with einsum to match origial repo
      
      * Revert "replace baddbmm with einsum to match origial repo"
      
      This reverts commit 3f6b435d65dad3e5514cad2f5dd9e4419ca78e0b.
      
      * add step_index to scale_model_input
      
      * Revert "move the rescale prompt_embeds from prior_transformer to pipeline"
      
      This reverts commit 5b5a8e6be918fefd114a2945ed89d8e8fa8be21b.
      
      * move rescale from prior_transformer to pipeline
      
      * correct step_index in scale_model_input
      
      * remove print lines
      
      * refactor prior - reduce arguments
      
      * make style
      
      * add prior_image
      
      * arg embedding_proj_norm -> norm_embedding_proj
      
      * add pre-norm for proj_embedding
      
      * move rescale prompt from pipeline to _encode_prompt
      
      * add img2img pipeline
      
      * style
      
      * copies
      
      * Update src/diffusers/models/prior_transformer.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/models/prior_transformer.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/models/prior_transformer.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/models/prior_transformer.py
      
      add arg: encoder_hid_proj
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/models/prior_transformer.py
      
      add new config: norm_in_type
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/models/prior_transformer.py
      
      add new config: added_emb_type
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/models/prior_transformer.py
      
      rename out_dim -> clip_embed_dim
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/models/prior_transformer.py
      
      rename config: out_dim -> clip_embed_dim
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/models/prior_transformer.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/models/prior_transformer.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * finish refactor prior_tranformer
      
      * make style
      
      * refactor renderer
      
      * fix
      
      * make style
      
      * refactor img2img
      
      * remove params_proj
      
      * add test
      
      * add upcast_softmax to prior_transformer
      
      * enable num_images_per_prompt, add save_gif utility
      
      * add
      
      * add fast test
      
      * make style
      
      * add slow test
      
      * style
      
      * add test for img2img
      
      * refactor
      
      * enable batching
      
      * style
      
      * refactor scheduler
      
      * update test
      
      * style
      
      * attempt to solve batch related tests timeout
      
      * add doc
      
      * Update src/diffusers/pipelines/shap_e/pipeline_shap_e.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/shap_e/pipeline_shap_e_img2img.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * hardcode rendering related config
      
      * update betas_for_alpha_bar on ddpm_scheduler
      
      * fix copies
      
      * fix
      
      * export_to_gif
      
      * style
      
      * second attempt to speed up batching tests
      
      * add doc page to index
      
      * Remove intermediate clipping
      
      * 3rd attempt to speed up batching tests
      
      * Remvoe time index
      
      * simplify scheduler
      
      * Fix more
      
      * Fix more
      
      * fix more
      
      * make style
      
      * fix schedulers
      
      * fix some more tests
      
      * finish
      
      * add one more test
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * style
      
      * apply feedbacks
      
      * style
      
      * fix copies
      
      * add one example
      
      * style
      
      * add example for img2img
      
      * fix doc
      
      * fix more doc strings
      
      * size -> frame_size
      
      * style
      
      * update doc
      
      * style
      
      * fix on doc
      
      * update repo name
      
      * improve the usage example in shap-e img2img
      
      * add usage examples in the shap-e docs.
      
      * consolidate examples.
      
      * minor fix.
      
      * update doc
      
      * Apply suggestions from code review
      
      * Apply suggestions from code review
      
      * remove upcast
      
      * Make sure background is white
      
      * Update src/diffusers/pipelines/shap_e/pipeline_shap_e.py
      
      * Apply suggestions from code review
      
      * Finish
      
      * Apply suggestions from code review
      
      * Update src/diffusers/pipelines/shap_e/pipeline_shap_e.py
      
      * Make style
      
      ---------
      Co-authored-by: default avataryiyixuxu <yixu310@gmail,com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      45f6d52b
  29. 05 Jun, 2023 1 commit
  30. 30 May, 2023 1 commit
    • Patrick von Platen's avatar
      Fix temb attention (#3607) · c0f867af
      Patrick von Platen authored
      * Fix temb attention
      
      * Apply suggestions from code review
      
      * make style
      
      * Add tests and fix docker
      
      * Apply suggestions from code review
      c0f867af
  31. 23 May, 2023 1 commit
    • Pedro Cuenca's avatar
      Run `torch.compile` tests in separate subprocesses (#3503) · bde2cb5d
      Pedro Cuenca authored
      * Run ControlNet compile test in a separate subprocess
      
      `torch.compile()` spawns several subprocesses and the GPU memory used
      was not reclaimed after the test ran. This approach was taken from
      `transformers`.
      
      * Style
      
      * Prepare a couple more compile tests to run in subprocess.
      
      * Use require_torch_2 decorator.
      
      * Test inpaint_compile in subprocess.
      
      * Run img2img compile test in subprocess.
      
      * Run stable diffusion compile test in subprocess.
      
      * style
      
      * Temporarily trigger on pr to test.
      
      * Revert "Temporarily trigger on pr to test."
      
      This reverts commit 82d76868ddf9cc634a9f14b2b0aef1d5433cd750.
      bde2cb5d
  32. 22 May, 2023 1 commit
    • Patrick von Platen's avatar
      Refactor full determinism (#3485) · 51843fd7
      Patrick von Platen authored
      * up
      
      * fix more
      
      * Apply suggestions from code review
      
      * fix more
      
      * fix more
      
      * Check it
      
      * Remove 16:8
      
      * fix more
      
      * fix more
      
      * fix more
      
      * up
      
      * up
      
      * Test only stable diffusion
      
      * Test only two files
      
      * up
      
      * Try out spinning up processes that can be killed
      
      * up
      
      * Apply suggestions from code review
      
      * up
      
      * up
      51843fd7
  33. 27 Apr, 2023 1 commit
    • Nipun Jindal's avatar
      [2064]: Add stochastic sampler (sample_dpmpp_sde) (#3020) · fd512d74
      Nipun Jindal authored
      
      
      * [2064]: Add stochastic sampler
      
      * [2064]: Add stochastic sampler
      
      * [2064]: Add stochastic sampler
      
      * [2064]: Add stochastic sampler
      
      * [2064]: Add stochastic sampler
      
      * [2064]: Add stochastic sampler
      
      * [2064]: Add stochastic sampler
      
      * Review comments
      
      * [Review comment]: Add is_torchsde_available()
      
      * [Review comment]: Test and docs
      
      * [Review comment]
      
      * [Review comment]
      
      * [Review comment]
      
      * [Review comment]
      
      * [Review comment]
      
      ---------
      Co-authored-by: default avatarnjindal <njindal@adobe.com>
      fd512d74
  34. 19 Apr, 2023 1 commit
    • hwuebben's avatar
      Update pipeline_stable_diffusion_inpaint_legacy.py (#2903) · 3becd368
      hwuebben authored
      
      
      * Update pipeline_stable_diffusion_inpaint_legacy.py
      
      * fix preprocessing of Pil images with adequate batch size
      
      * revert map
      
      * add tests
      
      * reformat
      
      * Update test_stable_diffusion_inpaint_legacy.py
      
      * Update test_stable_diffusion_inpaint_legacy.py
      
      * Update test_stable_diffusion_inpaint_legacy.py
      
      * Update test_stable_diffusion_inpaint_legacy.py
      
      * next try to fix the style
      
      * wth is this
      
      * Update testing_utils.py
      
      * Update testing_utils.py
      
      * Update test_stable_diffusion_inpaint_legacy.py
      
      * Update test_stable_diffusion_inpaint_legacy.py
      
      * Update test_stable_diffusion_inpaint_legacy.py
      
      * Update test_stable_diffusion_inpaint_legacy.py
      
      * Update test_stable_diffusion_inpaint_legacy.py
      
      * Update test_stable_diffusion_inpaint_legacy.py
      
      ---------
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      3becd368
  35. 28 Mar, 2023 1 commit
  36. 23 Mar, 2023 1 commit
    • Kashif Rasul's avatar
      Music Spectrogram diffusion pipeline (#1044) · 2ef9bdd7
      Kashif Rasul authored
      
      
      * initial TokenEncoder and ContinuousEncoder
      
      * initial modules
      
      * added ContinuousContextTransformer
      
      * fix copy paste error
      
      * use numpy for get_sequence_length
      
      * initial terminal relative positional encodings
      
      * fix weights keys
      
      * fix assert
      
      * cross attend style: concat encodings
      
      * make style
      
      * concat once
      
      * fix formatting
      
      * Initial SpectrogramPipeline
      
      * fix input_tokens
      
      * make style
      
      * added mel output
      
      * ignore weights for config
      
      * move mel to numpy
      
      * import pipeline
      
      * fix class names and import
      
      * moved models to models folder
      
      * import ContinuousContextTransformer and SpectrogramDiffusionPipeline
      
      * initial spec diffusion converstion script
      
      * renamed config to t5config
      
      * added weight loading
      
      * use arguments instead of t5config
      
      * broadcast noise time to batch dim
      
      * fix call
      
      * added scale_to_features
      
      * fix weights
      
      * transpose laynorm weight
      
      * scale is a vector
      
      * scale the query outputs
      
      * added comment
      
      * undo scaling
      
      * undo depth_scaling
      
      * inital get_extended_attention_mask
      
      * attention_mask is none in self-attention
      
      * cleanup
      
      * manually invert attention
      
      * nn.linear need bias=False
      
      * added T5LayerFFCond
      
      * remove to fix conflict
      
      * make style and dummy
      
      * remove unsed variables
      
      * remove predict_epsilon
      
      * Move accelerate to a soft-dependency (#1134)
      
      * finish
      
      * finish
      
      * Update src/diffusers/modeling_utils.py
      
      * Update src/diffusers/pipeline_utils.py
      Co-authored-by: default avatarAnton Lozhkov <anton@huggingface.co>
      
      * more fixes
      
      * fix
      Co-authored-by: default avatarAnton Lozhkov <anton@huggingface.co>
      
      * fix order
      
      * added initial midi to note token data pipeline
      
      * added int to int tokenizer
      
      * remove duplicate
      
      * added logic for segments
      
      * add melgan to pipeline
      
      * move autoregressive gen into pipeline
      
      * added note_representation_processor_chain
      
      * fix dtypes
      
      * remove immutabledict req
      
      * initial doc
      
      * use np.where
      
      * require note_seq
      
      * fix typo
      
      * update dependency
      
      * added note-seq to test
      
      * added is_note_seq_available
      
      * fix import
      
      * added toc
      
      * added example usage
      
      * undo for now
      
      * moved docs
      
      * fix merge
      
      * fix imports
      
      * predict first segment
      
      * avoid un-needed copy to and from cpu
      
      * make style
      
      * Copyright
      
      * fix style
      
      * add test and fix inference steps
      
      * remove bogus files
      
      * reorder models
      
      * up
      
      * remove transformers dependency
      
      * make work with diffusers cross attention
      
      * clean more
      
      * remove @
      
      * improve further
      
      * up
      
      * uP
      
      * Apply suggestions from code review
      
      * Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
      
      * loop over all tokens
      
      * make style
      
      * Added a section on the model
      
      * fix formatting
      
      * grammer
      
      * formatting
      
      * make fix-copies
      
      * Update src/diffusers/pipelines/__init__.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/spectrogram_diffusion/pipeline_spectrogram_diffusion.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * added callback ad optional ionnx
      
      * do not squeeze batch dim
      
      * clean up more
      
      * upload
      
      * convert jax to nnumpy
      
      * make style
      
      * fix warning
      
      * make fix-copies
      
      * fix warning
      
      * add initial fast tests
      
      * add initial pipeline_params
      
      * eval mode due to dropout
      
      * skip batch tests as pipeline runs on a single file
      
      * make style
      
      * fix relative path
      
      * fix doc tests
      
      * Update src/diffusers/models/t5_film_transformer.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/models/t5_film_transformer.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update docs/source/en/api/pipelines/spectrogram_diffusion.mdx
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * add MidiProcessor
      
      * format
      
      * fix org
      
      * Apply suggestions from code review
      
      * Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
      
      * make style
      
      * pin protobuf to <4
      
      * fix formatting
      
      * white space
      
      * tensorboard needs protobuf
      
      ---------
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarAnton Lozhkov <anton@huggingface.co>
      2ef9bdd7
  37. 22 Mar, 2023 1 commit
    • Patrick von Platen's avatar
      [MS Text To Video] Add first text to video (#2738) · ca1a2229
      Patrick von Platen authored
      
      
      * [MS Text To Video} Add first text to video
      
      * upload
      
      * make first model example
      
      * match unet3d params
      
      * make sure weights are correcctly converted
      
      * improve
      
      * forward pass works, but diff result
      
      * make forward work
      
      * fix more
      
      * finish
      
      * refactor video output class.
      
      * feat: add support for a video export utility.
      
      * fix: opencv availability check.
      
      * run make fix-copies.
      
      * add: docs for the model components.
      
      * add: standalone pipeline doc.
      
      * edit docstring of the pipeline.
      
      * add: right path to TransformerTempModel
      
      * add: first set of tests.
      
      * complete fast tests for text to video.
      
      * fix bug
      
      * up
      
      * three fast tests failing.
      
      * add: note on slow tests
      
      * make work with all schedulers
      
      * apply styling.
      
      * add slow tests
      
      * change file name
      
      * update
      
      * more correction
      
      * more fixes
      
      * finish
      
      * up
      
      * Apply suggestions from code review
      
      * up
      
      * finish
      
      * make copies
      
      * fix pipeline tests
      
      * fix more tests
      
      * Apply suggestions from code review
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * apply suggestions
      
      * up
      
      * revert
      
      ---------
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      ca1a2229
  38. 07 Mar, 2023 1 commit
  39. 27 Feb, 2023 1 commit