1. 27 Jun, 2024 1 commit
  2. 29 May, 2024 1 commit
  3. 15 May, 2024 1 commit
    • Isamu Isozaki's avatar
      Adding VQGAN Training script (#5483) · d27e996c
      Isamu Isozaki authored
      
      
      * Init commit
      
      * Removed einops
      
      * Added default movq config for training
      
      * Update explanation of prompts
      
      * Fixed inheritance of discriminator and init_tracker
      
      * Fixed incompatible api between muse and here
      
      * Fixed output
      
      * Setup init training
      
      * Basic structure done
      
      * Removed attention for quick tests
      
      * Style fixes
      
      * Fixed vae/vqgan styles
      
      * Removed redefinition of wandb
      
      * Fixed log_validation and tqdm
      
      * Nothing commit
      
      * Added commit loss to lookup_from_codebook
      
      * Update src/diffusers/models/vq_model.py
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Adding perliminary README
      
      * Fixed one typo
      
      * Local changes
      
      * Fixed main issues
      
      * Merging
      
      * Update src/diffusers/models/vq_model.py
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Testing+Fixed bugs in training script
      
      * Some style fixes
      
      * Added wandb to docs
      
      * Fixed timm test
      
      * get testing suite ready.
      
      * remove return loss
      
      * remove return_loss
      
      * Remove diffs
      
      * Remove diffs
      
      * fix ruff format
      
      ---------
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      d27e996c
  4. 10 May, 2024 1 commit
    • Mark Van Aken's avatar
      #7535 Update FloatTensor type hints to Tensor (#7883) · be4afa0b
      Mark Van Aken authored
      * find & replace all FloatTensors to Tensor
      
      * apply formatting
      
      * Update torch.FloatTensor to torch.Tensor in the remaining files
      
      * formatting
      
      * Fix the rest of the places where FloatTensor is used as well as in documentation
      
      * formatting
      
      * Update new file from FloatTensor to Tensor
      be4afa0b
  5. 08 Feb, 2024 1 commit
  6. 21 Dec, 2023 1 commit
    • Will Berman's avatar
      open muse (#5437) · 40398152
      Will Berman authored
      
      
      amused
      
      rename
      
      Update docs/source/en/api/pipelines/amused.md
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      AdaLayerNormContinuous default values
      
      custom micro conditioning
      
      micro conditioning docs
      
      put lookup from codebook in constructor
      
      fix conversion script
      
      remove manual fused flash attn kernel
      
      add training script
      
      temp remove training script
      
      add dummy gradient checkpointing func
      
      clarify temperatures is an instance variable by setting it
      
      remove additional SkipFF block args
      
      hardcode norm args
      
      rename tests folder
      
      fix paths and samples
      
      fix tests
      
      add training script
      
      training readme
      
      lora saving and loading
      
      non-lora saving/loading
      
      some readme fixes
      
      guards
      
      Update docs/source/en/api/pipelines/amused.md
      Co-authored-by: default avatarSuraj Patil <surajp815@gmail.com>
      
      Update examples/amused/README.md
      Co-authored-by: default avatarSuraj Patil <surajp815@gmail.com>
      
      Update examples/amused/train_amused.py
      Co-authored-by: default avatarSuraj Patil <surajp815@gmail.com>
      
      vae upcasting
      
      add fp16 integration tests
      
      use tuple for micro cond
      
      copyrights
      
      remove casts
      
      delegate to torch.nn.LayerNorm
      
      move temperature to pipeline call
      
      upsampling/downsampling changes
      40398152
  7. 18 Dec, 2023 1 commit
  8. 16 Nov, 2023 1 commit
  9. 08 Nov, 2023 1 commit
    • Chi's avatar
      Replacing the nn.Mish activation function with a get_activation function. (#5651) · 9ae90593
      Chi authored
      
      
      * I added a new doc string to the class. This is more flexible to understanding other developers what are doing and where it's using.
      
      * Update src/diffusers/models/unet_2d_blocks.py
      
      This changes suggest by maintener.
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Update src/diffusers/models/unet_2d_blocks.py
      
      Add suggested text
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Update unet_2d_blocks.py
      
      I changed the Parameter to Args text.
      
      * Update unet_2d_blocks.py
      
      proper indentation set in this file.
      
      * Update unet_2d_blocks.py
      
      a little bit of change in the act_fun argument line.
      
      * I run the black command to reformat style in the code
      
      * Update unet_2d_blocks.py
      
      similar doc-string add to have in the original diffusion repository.
      
      * I removed the dummy variable defined in both the encoder and decoder.
      
      * Now, I run black package to reformat my file
      
      * Remove the redundant line from the adapter.py file.
      
      * Black package using to reformated my file
      
      * Replacing the nn.Mish activation function with a get_activation function allows developers to more easily choose the right activation function for their task. Additionally, removing redundant variables can improve code readability and maintainability.
      
      * I try to fix this: Fast tests for PRs / Fast PyTorch Models & Schedulers CPU tests (pull_request)
      
      * Update src/diffusers/models/resnet.py
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      
      ---------
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      9ae90593
  10. 25 Oct, 2023 1 commit
    • Aryan V S's avatar
      Improve typehints and docs in `diffusers/models` (#5391) · 0c9f174d
      Aryan V S authored
      
      
      * improvement: add typehints and docs to src/diffusers/models/attention_processor.py
      
      * improvement: add typehints and docs to src/diffusers/models/vae.py
      
      * improvement: add missing docs in src/diffusers/models/vq_model.py
      
      * improvement: add typehints and docs to src/diffusers/models/transformer_temporal.py
      
      * improvement: add typehints and docs to src/diffusers/models/t5_film_transformer.py
      
      * improvement: add type hints to src/diffusers/models/unet_1d_blocks.py
      
      * improvement: add missing type hints to src/diffusers/models/unet_2d_blocks.py
      
      * fix: CI error (make fix-copies required)
      
      * fix: CI error (make fix-copies required again)
      
      ---------
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      0c9f174d
  11. 11 Sep, 2023 1 commit
    • Dhruv Nair's avatar
      Lazy Import for Diffusers (#4829) · b6e0b016
      Dhruv Nair authored
      
      
      * initial commit
      
      * move modules to import struct
      
      * add dummy objects and _LazyModule
      
      * add lazy import to schedulers
      
      * clean up unused imports
      
      * lazy import on models module
      
      * lazy import for schedulers module
      
      * add lazy import to pipelines module
      
      * lazy import altdiffusion
      
      * lazy import audio diffusion
      
      * lazy import audioldm
      
      * lazy import consistency model
      
      * lazy import controlnet
      
      * lazy import dance diffusion ddim ddpm
      
      * lazy import deepfloyd
      
      * lazy import kandinksy
      
      * lazy imports
      
      * lazy import semantic diffusion
      
      * lazy imports
      
      * lazy import stable diffusion
      
      * move sd output to its own module
      
      * clean up
      
      * lazy import t2iadapter
      
      * lazy import unclip
      
      * lazy import versatile and vq diffsuion
      
      * lazy import vq diffusion
      
      * helper to fetch objects from modules
      
      * lazy import sdxl
      
      * lazy import txt2vid
      
      * lazy import stochastic karras
      
      * fix model imports
      
      * fix bug
      
      * lazy import
      
      * clean up
      
      * clean up
      
      * fixes for tests
      
      * fixes for tests
      
      * clean up
      
      * remove import of torch_utils from utils module
      
      * clean up
      
      * clean up
      
      * fix mistake import statement
      
      * dedicated modules for exporting and loading
      
      * remove testing utils from utils module
      
      * fixes from  merge conflicts
      
      * Update src/diffusers/pipelines/kandinsky2_2/__init__.py
      
      * fix docs
      
      * fix alt diffusion copied from
      
      * fix check dummies
      
      * fix more docs
      
      * remove accelerate import from utils module
      
      * add type checking
      
      * make style
      
      * fix check dummies
      
      * remove torch import from xformers check
      
      * clean up error message
      
      * fixes after upstream merges
      
      * dummy objects fix
      
      * fix tests
      
      * remove unused module import
      
      ---------
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      b6e0b016
  12. 06 Sep, 2023 1 commit
    • Kashif Rasul's avatar
      Würstchen model (#3849) · 541bb6ee
      Kashif Rasul authored
      
      
      * initial
      
      * initial
      
      * added initial convert script for paella vqmodel
      
      * initial wuerstchen pipeline
      
      * add LayerNorm2d
      
      * added modules
      
      * fix typo
      
      * use model_v2
      
      * embed clip caption amd negative_caption
      
      * fixed name of var
      
      * initial modules in one place
      
      * WuerstchenPriorPipeline
      
      * inital shape
      
      * initial denoising prior loop
      
      * fix output
      
      * add WuerstchenPriorPipeline to __init__.py
      
      * use the noise ratio in the Prior
      
      * try to save pipeline
      
      * save_pretrained working
      
      * Few additions
      
      * add _execution_device
      
      * shape is int
      
      * fix batch size
      
      * fix shape of ratio
      
      * fix shape of ratio
      
      * fix output dataclass
      
      * tests folder
      
      * fix formatting
      
      * fix float16 + started with generator
      
      * Update pipeline_wuerstchen.py
      
      * removed vqgan code
      
      * add WuerstchenGeneratorPipeline
      
      * fix WuerstchenGeneratorPipeline
      
      * fix docstrings
      
      * fix imports
      
      * convert generator pipeline
      
      * fix convert
      
      * Work on Generator Pipeline. WIP
      
      * Pipeline works with our diffuzz code
      
      * apply scale factor
      
      * removed vqgan.py
      
      * use cosine schedule
      
      * redo the denoising loop
      
      * Update src/diffusers/models/resnet.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * use torch.lerp
      
      * use warp-diffusion org
      
      * clip_sample=False,
      
      * some refactoring
      
      * use model_v3_stage_c
      
      * c_cond size
      
      * use clip-bigG
      
      * allow stage b clip to be None
      
      * add dummy
      
      * würstchen scheduler
      
      * minor changes
      
      * set clip=None in the pipeline
      
      * fix attention mask
      
      * add attention_masks to text_encoder
      
      * make fix-copies
      
      * add back clip
      
      * add text_encoder
      
      * gen_text_encoder and tokenizer
      
      * fix import
      
      * updated pipeline test
      
      * undo changes to pipeline test
      
      * nip
      
      * fix typo
      
      * fix output name
      
      * set guidance_scale=0 and remove diffuze
      
      * fix doc strings
      
      * make style
      
      * nip
      
      * removed unused
      
      * initial docs
      
      * rename
      
      * toc
      
      * cleanup
      
      * remvoe test script
      
      * fix-copies
      
      * fix multi images
      
      * remove dup
      
      * remove unused modules
      
      * undo changes for debugging
      
      * no  new line
      
      * remove dup conversion script
      
      * fix doc string
      
      * cleanup
      
      * pass default args
      
      * dup permute
      
      * fix some tests
      
      * fix prepare_latents
      
      * move Prior class to modules
      
      * offload only the text encoder and vqgan
      
      * fix resolution calculation for prior
      
      * nip
      
      * removed testing script
      
      * fix shape
      
      * fix argument to set_timesteps
      
      * do not change .gitignore
      
      * fix resolution calculations + readme
      
      * resolution calculation fix + readme
      
      * small fixes
      
      * Add combined pipeline
      
      * rename generator -> decoder
      
      * Update .gitignore
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * removed efficient_net
      
      * create combined WuerstchenPipeline
      
      * make arguments consistent with VQ model
      
      * fix var names
      
      * no need to return text_encoder_hidden_states
      
      * add latent_dim_scale to config
      
      * split model into its own file
      
      * add WuerschenPipeline to docs
      
      * remove unused latent_size
      
      * register latent_dim_scale
      
      * update script
      
      * update docstring
      
      * use Attention preprocessor
      
      * concat with normed input
      
      * fix-copies
      
      * add docs
      
      * fix test
      
      * fix style
      
      * add to cpu_offloaded_model
      
      * updated type
      
      * remove 1-line func
      
      * updated type
      
      * initial decoder test
      
      * formatting
      
      * formatting
      
      * fix autodoc link
      
      * num_inference_steps is int
      
      * remove comments
      
      * fix example in docs
      
      * Update src/diffusers/pipelines/wuerstchen/diffnext.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * rename layernorm to WuerstchenLayerNorm
      
      * rename DiffNext to WuerstchenDiffNeXt
      
      * added comment about MixingResidualBlock
      
      * move paella vq-vae to pipelines' folder
      
      * initial decoder test
      
      * increased test_float16_inference expected diff
      
      * self_attn is always true
      
      * more passing decoder tests
      
      * batch image_embeds
      
      * fix failing tests
      
      * set the correct dtype
      
      * relax inference test
      
      * update prior
      
      * added combined pipeline test
      
      * faster test
      
      * faster test
      
      * Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * fix issues from review
      
      * update wuerstchen.md + change generator name
      
      * resolve issues
      
      * fix copied from usage and add back batch_size
      
      * fix API
      
      * fix arguments
      
      * fix combined test
      
      * Added timesteps argument + fixes
      
      * Update tests/pipelines/test_pipelines_common.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update tests/pipelines/wuerstchen/test_wuerstchen_prior.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
      
      * up
      
      * Fix more
      
      * failing tests
      
      * up
      
      * up
      
      * correct naming
      
      * correct docs
      
      * correct docs
      
      * fix test params
      
      * correct docs
      
      * fix classifier free guidance
      
      * fix classifier free guidance
      
      * fix more
      
      * fix all
      
      * make tests faster
      
      ---------
      Co-authored-by: default avatarDominic Rampas <d6582533@gmail.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarDominic Rampas <61938694+dome272@users.noreply.github.com>
      541bb6ee
  13. 06 Jul, 2023 1 commit
    • YiYi Xu's avatar
      Kandinsky_v22_yiyi (#3936) · 74621567
      YiYi Xu authored
      
      
      * Kandinsky2_2
      
      * fix init kandinsky2_2
      
      * kandinsky2_2 fix inpainting
      
      * rename pipelines: remove decoder + 2_2 -> V22
      
      * Update scheduling_unclip.py
      
      * remove text_encoder and tokenizer arguments from doc string
      
      * add test for text2img
      
      * add tests for text2img & img2img
      
      * fix
      
      * add test for inpaint
      
      * add prior tests
      
      * style
      
      * copies
      
      * add controlnet test
      
      * style
      
      * add a test for controlnet_img2img
      
      * update prior_emb2emb api to accept image_embedding or image
      
      * add a test for prior_emb2emb
      
      * style
      
      * remove try except
      
      * example
      
      * fix
      
      * add doc string examples to all kandinsky pipelines
      
      * style
      
      * update doc
      
      * style
      
      * add a top about 2.2
      
      * Apply suggestions from code review
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * vae -> movq
      
      * vae -> movq
      
      * style
      
      * fix the #copied from
      
      * remove decoder from file name
      
      * update doc: add a section for kandinsky 2.2
      
      * fix
      
      * fix-copies
      
      * add coped from
      
      * add copies from for prior
      
      * add copies from for prior emb2emb
      
      * copy from for img2img
      
      * copied from for inpaint
      
      * more copied from
      
      * more copies from
      
      * more copies
      
      * remove the yiyi comments
      
      * Apply suggestions from code review
      
      * Self-contained example, pipeline order
      
      * Import prior output instead of redefining.
      
      * Style
      
      * Make VQModel compatible with model offload.
      
      * Fix copies
      
      ---------
      Co-authored-by: default avatarShahmatov Arseniy <62886550+cene555@users.noreply.github.com>
      Co-authored-by: default avataryiyixuxu <yixu310@gmail,com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      74621567
  14. 30 Jun, 2023 1 commit
    • Steven Liu's avatar
      [docs] Model API (#3562) · 174dcd69
      Steven Liu authored
      * add modelmixin and unets
      
      * remove old model page
      
      * minor fixes
      
      * fix unet2dcondition
      
      * add vqmodel and autoencoderkl
      
      * add rest of models
      
      * fix autoencoderkl path
      
      * fix toctree
      
      * fix toctree again
      
      * apply feedback
      
      * apply feedback
      
      * fix copies
      
      * fix controlnet copy
      
      * fix copies
      174dcd69
  15. 25 May, 2023 1 commit
  16. 01 Mar, 2023 1 commit
  17. 26 Jan, 2023 1 commit
  18. 30 Dec, 2022 1 commit