1. 11 Jul, 2024 1 commit
    • Xin Ma's avatar
      Latte: Latent Diffusion Transformer for Video Generation (#8404) · b8cf84a3
      Xin Ma authored
      
      
      * add Latte to diffusers
      
      * remove print
      
      * remove print
      
      * remove print
      
      * remove unuse codes
      
      * remove layer_norm_latte and add a flag
      
      * remove layer_norm_latte and add a flag
      
      * update latte_pipeline
      
      * update latte_pipeline
      
      * remove unuse squeeze
      
      * add norm_hidden_states.ndim == 2: # for Latte
      
      * fixed test latte pipeline bugs
      
      * fixed test latte pipeline bugs
      
      * delete sh
      
      * add doc for latte
      
      * add licensing
      
      * Move Transformer3DModelOutput to modeling_outputs
      
      * give a default value to sample_size
      
      * remove the einops dependency
      
      * change norm2 for latte
      
      * modify pipeline of latte
      
      * update test for Latte
      
      * modify some codes for latte
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * modify for Latte pipeline
      
      * video_length -> num_frames; update prepare_latents copied from
      
      * make fix-copies
      
      * make style
      
      * typo: videe -> video
      
      * update
      
      * modify for Latte pipeline
      
      * modify latte pipeline
      
      * modify latte pipeline
      
      * modify latte pipeline
      
      * modify latte pipeline
      
      * modify for Latte pipeline
      
      * Delete .vscode directory
      
      * make style
      
      * make fix-copies
      
      * add latte transformer 3d to docs _toctree.yml
      
      * update example
      
      * reduce frames for test
      
      * fixed bug of _text_preprocessing
      
      * set num frame to 1 for testing
      
      * remove unuse print
      
      * add text = self._clean_caption(text) again
      
      ---------
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      Co-authored-by: default avatarAryan <contact.aryanvs@gmail.com>
      Co-authored-by: default avatarAryan <aryan@huggingface.co>
      b8cf84a3
  2. 14 Nov, 2023 1 commit
  3. 15 Mar, 2023 1 commit
    • Henrik Forst茅n's avatar
      Controlnet training (#2545) · 79eb3d07
      Henrik Forst茅n authored
      * Controlnet training code initial commit
      
      Works with circle dataset: https://github.com/lllyasviel/ControlNet/blob/main/docs/train.md
      
      * Script for adding a controlnet to existing model
      
      * Fix control image transform
      
      Control image should be in 0..1 range.
      
      * Add license header and remove more unused configs
      
      * controlnet training readme
      
      * Allow nonlocal model in add_controlnet.py
      
      * Formatting
      
      * Remove unused code
      
      * Code quality
      
      * Initialize controlnet in training script
      
      * Formatting
      
      * Address review comments
      
      * doc style
      
      * explicit constructor args and submodule names
      
      * hub dataset
      
      NOTE -  not tested
      
      * empty prompts
      
      * add conditioning image
      
      * rename
      
      * remove instance data dir
      
      * image_transforms -> -1,1 . conditioning_image_transformers -> 0, 1
      
      * nits
      
      * remove local rank config
      
      I think this isn't necessary in any of our training scripts
      
      * validation images
      
      * proportion_empty_prompts typo
      
      * weight copy...
      79eb3d07
  4. 07 Feb, 2023 1 commit
  5. 15 Dec, 2022 1 commit
  6. 05 Dec, 2022 1 commit
    • Robert Dargavel Smith's avatar
      add AudioDiffusionPipeline and LatentAudioDiffusionPipeline #1334 (#1426) · 48d0123f
      Robert Dargavel Smith authored
      * add AudioDiffusionPipeline and LatentAudioDiffusionPipeline
      
      * add docs to toc
      
      * fix tests
      
      * fix tests
      
      * fix tests
      
      * fix tests
      
      * fix tests
      
      * Update pr_tests.yml
      
      Fix tests
      
      * parent 499ff34b3edc3e0c506313ab48f21514d8f58b09
      author teticio <teticio@gmail.com> 1668765652 +0000
      committer teticio <teticio@gmail.com> 1669041721 +0000
      
      parent 499ff34b3edc3e0c506313ab48f21514d8f58b09
      author teticio <teticio@gmail.com> 1668765652 +0000
      committer teticio <teticio@gmail.com> 1669041704 +0000
      
      add colab notebook
      
      [Flax] Fix loading scheduler from subfolder (#1319)
      
      [FLAX] Fix loading scheduler from subfolder
      
      Fix/Enable all schedulers for in-painting (#1331)
      
      * inpaint fix k lms
      
      * onnox as well
      
      * up
      
      Correct path to schedlure (#1322)
      
      * [Examples] Correct path
      
      * uP
      
      Avoid nested fix-copies (#1332)
      
      * Avoid nested `# Copied from` statements during `make fix-copies`
      
      * style
      
      Fix img2...
      48d0123f
  7. 14 Nov, 2022 1 commit
    • Nathan Lambert's avatar
      Add UNet 1d for RL model for planning + colab (#105) · 7c5fef81
      Nathan Lambert authored
      * re-add RL model code
      
      * match model forward api
      
      * add register_to_config, pass training tests
      
      * fix tests, update forward outputs
      
      * remove unused code, some comments
      
      * add to docs
      
      * remove extra embedding code
      
      * unify time embedding
      
      * remove conv1d output sequential
      
      * remove sequential from conv1dblock
      
      * style and deleting duplicated code
      
      * clean files
      
      * remove unused variables
      
      * clean variables
      
      * add 1d resnet block structure for downsample
      
      * rename as unet1d
      
      * fix renaming
      
      * rename files
      
      * add get_block(...) api
      
      * unify args for model1d like model2d
      
      * minor cleaning
      
      * fix docs
      
      * improve 1d resnet blocks
      
      * fix tests, remove permuts
      
      * fix style
      
      * add output activation
      
      * rename flax blocks file
      
      * Add Value Function and corresponding example script to Diffuser implementation (#884)
      
      * valuefunction code
      
      * start example scripts
      
      * missing imports
      
      * bug fixes and pla...
      7c5fef81
  8. 31 May, 2022 1 commit