1. 13 Apr, 2023 1 commit
  2. 11 Apr, 2023 1 commit
    • Chanchana Sornsoontorn's avatar
      Fix typo and format BasicTransformerBlock attributes (#2953) · 52c4d32d
      Chanchana Sornsoontorn authored
      * ️chore(train_controlnet) fix typo in logger message
      
      * ️chore(models) refactor modules order; make them the same as calling order
      
      When printing the BasicTransformerBlock to stdout, I think it's crucial that the attributes order are shown in proper order. And also previously the "3. Feed Forward" comment was not making sense. It should have been close to self.ff but it's instead next to self.norm3
      
      * correct many tests
      
      * remove bogus file
      
      * make style
      
      * correct more tests
      
      * finish tests
      
      * fix one more
      
      * make style
      
      * make unclip deterministic
      
      * 
      
      ️chore(models/attention) reorganize comments in BasicTransformerBlock class
      
      ---------
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      52c4d32d
  3. 27 Mar, 2023 1 commit
  4. 22 Mar, 2023 1 commit
  5. 21 Mar, 2023 1 commit
  6. 18 Mar, 2023 2 commits
  7. 03 Mar, 2023 1 commit
  8. 01 Mar, 2023 3 commits
  9. 28 Feb, 2023 1 commit
  10. 27 Feb, 2023 2 commits
  11. 16 Feb, 2023 3 commits
  12. 15 Feb, 2023 1 commit
  13. 14 Feb, 2023 1 commit
  14. 13 Feb, 2023 1 commit
  15. 10 Feb, 2023 1 commit
    • Patrick von Platen's avatar
      Correct fast tests (#2314) · 96c2279b
      Patrick von Platen authored
      * correct some
      
      * Apply suggestions from code review
      
      * correct
      
      * Update tests/pipelines/altdiffusion/test_alt_diffusion_img2img.py
      
      * Final
      96c2279b
  16. 08 Feb, 2023 1 commit
  17. 07 Feb, 2023 3 commits
    • Patrick von Platen's avatar
      Replace flake8 with ruff and update black (#2279) · a7ca03aa
      Patrick von Platen authored
      * before running make style
      
      * remove left overs from flake8
      
      * finish
      
      * make fix-copies
      
      * final fix
      
      * more fixes
      a7ca03aa
    • Patrick von Platen's avatar
      [Tests] Fix slow tests (#2271) · bbb46ad3
      Patrick von Platen authored
      bbb46ad3
    • YiYi Xu's avatar
      Stable Diffusion Latent Upscaler (#2059) · 1051ca81
      YiYi Xu authored
      
      
      * Modify UNet2DConditionModel
      
      - allow skipping mid_block
      
      - adding a norm_group_size argument so that we can set the `num_groups` for group norm using `num_channels//norm_group_size`
      
      - allow user to set dimension for the timestep embedding (`time_embed_dim`)
      
      - the kernel_size for `conv_in` and `conv_out` is now configurable
      
      - add random fourier feature layer (`GaussianFourierProjection`) for `time_proj`
      
      - allow user to add the time and class embeddings before passing through the projection layer together - `time_embedding(t_emb + class_label))`
      
      - added 2 arguments `attn1_types` and `attn2_types`
      
        * currently we have argument `only_cross_attention`: when it's set to `True`, we will have a to the
      `BasicTransformerBlock` block with 2 cross-attention , otherwise we
      get a self-attention followed by a cross-attention; in k-upscaler, we need to have blocks that include just one cross-attention, or self-attention -> cross-attention;
      so I added `attn1_types` and `attn2_types` to the unet's argument list to allow user specify the attention types for the 2 positions in each block;  note that I stil kept
      the `only_cross_attention` argument for unet for easy configuration, but it will be converted to `attn1_type` and `attn2_type` when passing down to the down blocks
      
      - the position of downsample layer and upsample layer is now configurable
      
      - in k-upscaler unet, there is only one skip connection per each up/down block (instead of each layer in stable diffusion unet), added `skip_freq = "block"` to support
      this use case
      
      - if user passes attention_mask to unet, it will prepare the mask and pass a flag to cross attention processer to skip the `prepare_attention_mask` step
      inside cross attention block
      
      add up/down blocks for k-upscaler
      
      modify CrossAttention class
      
      - make the `dropout` layer in `to_out` optional
      
      - `use_conv_proj` - use conv instead of linear for all projection layers (i.e. `to_q`, `to_k`, `to_v`, `to_out`) whenever possible. note that when it's used to do cross
      attention, to_k, to_v has to be linear because the `encoder_hidden_states` is not 2d
      
      - `cross_attention_norm` - add an optional layernorm on encoder_hidden_states
      
      - `attention_dropout`: add an optional dropout on attention score
      
      adapt BasicTransformerBlock
      
      - add an ada groupnorm layer  to conditioning attention input with timestep embedding
      
      - allow skipping the FeedForward layer in between the attentions
      
      - replaced the only_cross_attention argument with attn1_type and attn2_type for more flexible configuration
      
      update timestep embedding: add new act_fn  gelu and an optional act_2
      
      modified ResnetBlock2D
      
      - refactored with AdaGroupNorm class (the timestep scale shift normalization)
      
      - add `mid_channel` argument - allow the first conv to have a different output dimension from the second conv
      
      - add option to use input AdaGroupNorm on the input instead of groupnorm
      
      - add options to add a dropout layer after each conv
      
      - allow user to set the bias in conv_shortcut (needed for k-upscaler)
      
      - add gelu
      
      adding conversion script for k-upscaler unet
      
      add pipeline
      
      * fix attention mask
      
      * fix a typo
      
      * fix a bug
      
      * make sure model can be used with GPU
      
      * make pipeline work with fp16
      
      * fix an error in BasicTransfomerBlock
      
      * make style
      
      * fix typo
      
      * some more fixes
      
      * uP
      
      * up
      
      * correct more
      
      * some clean-up
      
      * clean time proj
      
      * up
      
      * uP
      
      * more changes
      
      * remove the upcast_attention=True from unet config
      
      * remove attn1_types, attn2_types etc
      
      * fix
      
      * revert incorrect changes up/down samplers
      
      * make style
      
      * remove outdated files
      
      * Apply suggestions from code review
      
      * attention refactor
      
      * refactor cross attention
      
      * Apply suggestions from code review
      
      * update
      
      * up
      
      * update
      
      * Apply suggestions from code review
      
      * finish
      
      * Update src/diffusers/models/cross_attention.py
      
      * more fixes
      
      * up
      
      * up
      
      * up
      
      * finish
      
      * more corrections of conversion state
      
      * act_2 -> act_2_fn
      
      * remove dropout_after_conv from ResnetBlock2D
      
      * make style
      
      * simplify KAttentionBlock
      
      * add fast test for latent upscaler pipeline
      
      * add slow test
      
      * slow test fp16
      
      * make style
      
      * add doc string for pipeline_stable_diffusion_latent_upscale
      
      * add api doc page for latent upscaler pipeline
      
      * deprecate attention mask
      
      * clean up embeddings
      
      * simplify resnet
      
      * up
      
      * clean up resnet
      
      * up
      
      * correct more
      
      * up
      
      * up
      
      * improve a bit more
      
      * correct more
      
      * more clean-ups
      
      * Update docs/source/en/api/pipelines/stable_diffusion/latent_upscale.mdx
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/latent_upscale.mdx
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * add docstrings for new unet config
      
      * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_latent_upscale.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_latent_upscale.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * # Copied from
      
      * encode the image if not latent
      
      * remove force casting vae to fp32
      
      * fix
      
      * add comments about preconditioning parameters from k-diffusion paper
      
      * attn1_type, attn2_type -> add_self_attention
      
      * clean up get_down_block and get_up_block
      
      * fix
      
      * fixed a typo(?) in ada group norm
      
      * update slice attention processer for cross attention
      
      * update slice
      
      * fix fast test
      
      * update the checkpoint
      
      * finish tests
      
      * fix-copies
      
      * fix-copy for modeling_text_unet.py
      
      * make style
      
      * make style
      
      * fix f-string
      
      * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_latent_upscale.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * fix import
      
      * correct changes
      
      * fix resnet
      
      * make fix-copies
      
      * correct euler scheduler
      
      * add missing #copied from for preprocess
      
      * revert
      
      * fix
      
      * fix copies
      
      * Update docs/source/en/api/pipelines/stable_diffusion/latent_upscale.mdx
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/latent_upscale.mdx
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/latent_upscale.mdx
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/latent_upscale.mdx
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update src/diffusers/models/cross_attention.py
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_latent_upscale.py
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_latent_upscale.py
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * clean up conversion script
      
      * KDownsample2d,KUpsample2d -> KDownsample2D,KUpsample2D
      
      * more
      
      * Update src/diffusers/models/unet_2d_condition.py
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * remove prepare_extra_step_kwargs
      
      * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_latent_upscale.py
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_latent_upscale.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * fix a typo in timestep embedding
      
      * remove num_image_per_prompt
      
      * fix fasttest
      
      * make style + fix-copies
      
      * fix
      
      * fix xformer test
      
      * fix style
      
      * doc string
      
      * make style
      
      * fix-copies
      
      * docstring for time_embedding_norm
      
      * make style
      
      * final finishes
      
      * make fix-copies
      
      * fix tests
      
      ---------
      Co-authored-by: default avataryiyixuxu <yixu@yis-macbook-pro.lan>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      1051ca81
  18. 25 Jan, 2023 1 commit
    • Patrick von Platen's avatar
      Reproducibility 3/3 (#1924) · 6ba2231d
      Patrick von Platen authored
      
      
      * make tests deterministic
      
      * run slow tests
      
      * prepare for testing
      
      * finish
      
      * refactor
      
      * add print statements
      
      * finish more
      
      * correct some test failures
      
      * more fixes
      
      * set up to correct tests
      
      * more corrections
      
      * up
      
      * fix more
      
      * more prints
      
      * add
      
      * up
      
      * up
      
      * up
      
      * uP
      
      * uP
      
      * more fixes
      
      * uP
      
      * up
      
      * up
      
      * up
      
      * up
      
      * fix more
      
      * up
      
      * up
      
      * clean tests
      
      * up
      
      * up
      
      * up
      
      * more fixes
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSuraj Patil <surajp815@gmail.com>
      
      * make
      
      * correct
      
      * finish
      
      * finish
      Co-authored-by: default avatarSuraj Patil <surajp815@gmail.com>
      6ba2231d
  19. 17 Jan, 2023 1 commit
  20. 29 Dec, 2022 1 commit
  21. 25 Dec, 2022 1 commit
  22. 19 Dec, 2022 1 commit
  23. 16 Dec, 2022 1 commit
  24. 13 Dec, 2022 1 commit
  25. 08 Dec, 2022 1 commit
    • Suraj Patil's avatar
      StableDiffusionDepth2ImgPipeline (#1531) · 5383188c
      Suraj Patil authored
      
      
      * begin depth pipeline
      
      * add depth estimation model
      
      * fix prepare_depth_mask
      
      * add a comment about autocast
      
      * copied from, quality, cleanup
      
      * begin tests
      
      * handle tensors
      
      * norm image tensor
      
      * fix batch size
      
      * fix tests
      
      * fix enable_sequential_cpu_offload
      
      * fix save load
      
      * fix test_save_load_float16
      
      * fix test_save_load_optional_components
      
      * fix test_float16_inference
      
      * fix test_cpu_offload_forward_pass
      
      * fix test_dict_tuple_outputs_equivalent
      
      * up
      
      * fix fast tests
      
      * fix test_stable_diffusion_img2img_multiple_init_images
      
      * fix few more fast tests
      
      * don't use device map for DPT
      
      * fix test_stable_diffusion_pipeline_with_sequential_cpu_offloading
      
      * accept external depth maps
      
      * prepare_depth_mask -> prepare_depth_map
      
      * fix file name
      
      * fix file name
      
      * quality
      
      * check transformers version
      
      * fix test names
      
      * use skipif
      
      * fix import
      
      * add docs
      
      * skip tests on mps
      
      * correct version
      
      * uP
      
      * Update docs/source/api/pipelines/stable_diffusion_2.mdx
      
      * fix fix-copies
      
      * fix fix-copies
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avataranton- <anton@huggingface.co>
      5383188c
  26. 07 Dec, 2022 1 commit
  27. 06 Dec, 2022 1 commit
    • Anton Lozhkov's avatar
      Standardize fast pipeline tests with PipelineTestMixin (#1526) · 02d83c9f
      Anton Lozhkov authored
      
      
      * [WIP] Standardize fast pipeline tests with PipelineTestMixin
      
      * refactor the sd tests a bit
      
      * add more common tests
      
      * add xformers
      
      * add progressbar test
      
      * cleanup
      
      * upd fp16
      
      * CycleDiffusionPipelineFastTests
      
      * DanceDiffusionPipelineFastTests
      
      * AltDiffusionPipelineFastTests
      
      * StableDiffusion2PipelineFastTests
      
      * StableDiffusion2InpaintPipelineFastTests
      
      * StableDiffusionImageVariationPipelineFastTests
      
      * StableDiffusionImg2ImgPipelineFastTests
      
      * StableDiffusionInpaintPipelineFastTests
      
      * remove unused mixins
      
      * quality
      
      * add missing inits
      
      * try to fix mps tests
      
      * fix mps tests
      
      * add mps warmups
      
      * skip for some pipelines
      
      * style
      
      * Update tests/test_pipelines_common.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      02d83c9f
  28. 05 Dec, 2022 1 commit
  29. 02 Dec, 2022 1 commit
  30. 29 Nov, 2022 2 commits
  31. 28 Nov, 2022 1 commit
    • Patrick von Platen's avatar
      Add 2nd order heun scheduler (#1336) · 4c54519e
      Patrick von Platen authored
      * Add heun
      
      * Finish first version of heun
      
      * remove bogus
      
      * finish
      
      * finish
      
      * improve
      
      * up
      
      * up
      
      * fix more
      
      * change progress bar
      
      * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py
      
      * finish
      
      * up
      
      * up
      
      * up
      4c54519e