Errata - Trim trailing white space in the whole repo (#8575)

* Trim all the trailing white space in the whole repo

* Remove unnecessary empty places

* make style && make quality

* Trim trailing white space

* trim

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

examples/community/README_community_scripts.md

 # Community Scripts
 
-**Community scripts** consist of inference examples using Diffusers pipelines that have been added by the community. 
+**Community scripts** consist of inference examples using Diffusers pipelines that have been added by the community.
 Please have a look at the following table to get an overview of all community examples. Click on the **Code Example** to get a copy-and-paste code example that you can try out.
 If a community script doesn't work as expected, please open an issue and ping the author on it.
 

examples/controlnet/README_sdxl.md

@@ -42,7 +42,7 @@ from accelerate.utils import write_basic_config
 write_basic_config()
 ```
 
-When running `accelerate config`, if we specify torch compile mode to True there can be dramatic speedups. 
+When running `accelerate config`, if we specify torch compile mode to True there can be dramatic speedups.
 
 ## Circle filling dataset
 
@@ -85,7 +85,7 @@ accelerate launch train_controlnet_sdxl.py \
 To better track our training experiments, we're using the following flags in the command above:
 
 * `report_to="wandb` will ensure the training runs are tracked on Weights and Biases. To use it, be sure to install `wandb` with `pip install wandb`.
-* `validation_image`, `validation_prompt`, and `validation_steps` to allow the script to do a few validation inference runs. This allows us to qualitatively check if the training is progressing as expected. 
+* `validation_image`, `validation_prompt`, and `validation_steps` to allow the script to do a few validation inference runs. This allows us to qualitatively check if the training is progressing as expected.
 
 Our experiments were conducted on a single 40GB A100 GPU.
 

examples/custom_diffusion/README.md

-# Custom Diffusion training example 
+# Custom Diffusion training example
 
 [Custom Diffusion](https://arxiv.org/abs/2212.04488) is a method to customize text-to-image models like Stable Diffusion given just a few (4~5) images of a subject.
 The `train_custom_diffusion.py` script shows how to implement the training procedure and adapt it for stable diffusion.
@@ -23,7 +23,7 @@ Then cd in the example folder and run
 
 ```bash
 pip install -r requirements.txt
-pip install clip-retrieval 
+pip install clip-retrieval
 ```
 
 And initialize an [🤗Accelerate](https://github.com/huggingface/accelerate/) environment with:
@@ -46,10 +46,10 @@ write_basic_config()
 ```
 ### Cat example 😺
 
-Now let's get our dataset. Download dataset from [here](https://www.cs.cmu.edu/~custom-diffusion/assets/data.zip) and unzip it. 
+Now let's get our dataset. Download dataset from [here](https://www.cs.cmu.edu/~custom-diffusion/assets/data.zip) and unzip it.
 
-We also collect 200 real images using `clip-retrieval` which are combined with the target images in the training dataset as a regularization. This prevents overfitting to the given target image. The following flags enable the regularization `with_prior_preservation`, `real_prior` with `prior_loss_weight=1.`. 
-The `class_prompt` should be the category name same as target image. The collected real images are with text captions similar to the `class_prompt`. The retrieved image are saved in `class_data_dir`. You can disable `real_prior` to use generated images as regularization. To collect the real images use this command first before training. 
+We also collect 200 real images using `clip-retrieval` which are combined with the target images in the training dataset as a regularization. This prevents overfitting to the given target image. The following flags enable the regularization `with_prior_preservation`, `real_prior` with `prior_loss_weight=1.`.
+The `class_prompt` should be the category name same as target image. The collected real images are with text captions similar to the `class_prompt`. The retrieved image are saved in `class_data_dir`. You can disable `real_prior` to use generated images as regularization. To collect the real images use this command first before training.
 
 ```bash
 pip install clip-retrieval
@@ -77,7 +77,7 @@ accelerate launch train_custom_diffusion.py \
   --lr_warmup_steps=0 \
   --max_train_steps=250 \
   --scale_lr --hflip  \
-  --modifier_token "<new1>" 
+  --modifier_token "<new1>"
 ```
 
 **Use `--enable_xformers_memory_efficient_attention` for faster training with lower VRAM requirement (16GB per GPU). Follow [this guide](https://github.com/facebookresearch/xformers) for installation instructions.**
@@ -85,7 +85,7 @@ accelerate launch train_custom_diffusion.py \
 To track your experiments using Weights and Biases (`wandb`) and to save intermediate results (which we HIGHLY recommend), follow these steps:
 
 * Install `wandb`: `pip install wandb`.
-* Authorize: `wandb login`. 
+* Authorize: `wandb login`.
 * Then specify a `validation_prompt` and set `report_to` to `wandb` while launching training. You can also configure the following related arguments:
     * `num_validation_images`
     * `validation_steps`
@@ -112,7 +112,7 @@ accelerate launch train_custom_diffusion.py \
   --report_to="wandb"
 ```
 
-Here is an example [Weights and Biases page](https://wandb.ai/sayakpaul/custom-diffusion/runs/26ghrcau) where you can check out the intermediate results along with other training details.  
+Here is an example [Weights and Biases page](https://wandb.ai/sayakpaul/custom-diffusion/runs/26ghrcau) where you can check out the intermediate results along with other training details.
 
 If you specify `--push_to_hub`, the learned parameters will be pushed to a repository on the Hugging Face Hub. Here is an [example repository](https://huggingface.co/sayakpaul/custom-diffusion-cat).
 
@@ -120,7 +120,7 @@ If you specify `--push_to_hub`, the learned parameters will be pushed to a repos
 
 Provide a [json](https://github.com/adobe-research/custom-diffusion/blob/main/assets/concept_list.json) file with the info about each concept, similar to [this](https://github.com/ShivamShrirao/diffusers/blob/main/examples/dreambooth/train_dreambooth.py).
 
-To collect the real images run this command for each concept in the json file. 
+To collect the real images run this command for each concept in the json file.
 
 ```bash
 pip install clip-retrieval
@@ -145,16 +145,16 @@ accelerate launch train_custom_diffusion.py \
   --max_train_steps=500 \
   --num_class_images=200 \
   --scale_lr --hflip  \
-  --modifier_token "<new1>+<new2>" 
+  --modifier_token "<new1>+<new2>"
 ```
 
-Here is an example [Weights and Biases page](https://wandb.ai/sayakpaul/custom-diffusion/runs/3990tzkg) where you can check out the intermediate results along with other training details.  
+Here is an example [Weights and Biases page](https://wandb.ai/sayakpaul/custom-diffusion/runs/3990tzkg) where you can check out the intermediate results along with other training details.
 
 ### Training on human faces
 
-For fine-tuning on human faces we found the following configuration to work better: `learning_rate=5e-6`, `max_train_steps=1000 to 2000`, and `freeze_model=crossattn` with at least 15-20 images. 
+For fine-tuning on human faces we found the following configuration to work better: `learning_rate=5e-6`, `max_train_steps=1000 to 2000`, and `freeze_model=crossattn` with at least 15-20 images.
 
-To collect the real images use this command first before training. 
+To collect the real images use this command first before training.
 
 ```bash
 pip install clip-retrieval
@@ -184,7 +184,7 @@ accelerate launch train_custom_diffusion.py \
   --scale_lr --hflip --noaug \
   --freeze_model crossattn \
   --modifier_token "<new1>" \
-  --enable_xformers_memory_efficient_attention 
+  --enable_xformers_memory_efficient_attention
 ```
 
 ## Inference
@@ -267,7 +267,7 @@ Here, `cat` and `wooden pot` refer to the multiple concepts.
 
 ### Inference from a training checkpoint
 
-You can also perform inference from one of the complete checkpoint saved during the training process, if you used the `--checkpointing_steps` argument. 
+You can also perform inference from one of the complete checkpoint saved during the training process, if you used the `--checkpointing_steps` argument.
 
 TODO.
 

examples/dreambooth/README_sd3.md

@@ -2,16 +2,16 @@
 
 [DreamBooth](https://arxiv.org/abs/2208.12242) is a method to personalize text2image models like stable diffusion given just a few (3~5) images of a subject.
 
-The `train_dreambooth_sd3.py` script shows how to implement the training procedure and adapt it for [Stable Diffusion 3](https://huggingface.co/papers/2403.03206). We also provide a LoRA implementation in the `train_dreambooth_lora_sd3.py` script. 
+The `train_dreambooth_sd3.py` script shows how to implement the training procedure and adapt it for [Stable Diffusion 3](https://huggingface.co/papers/2403.03206). We also provide a LoRA implementation in the `train_dreambooth_lora_sd3.py` script.
 
-> [!NOTE]  
-> As the model is gated, before using it with diffusers you first need to go to the [Stable Diffusion 3 Medium Hugging Face page](https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers), fill in the form and accept the gate. Once you are in, you need to log in so that your system knows you’ve accepted the gate. Use the command below to log in: 
+> [!NOTE]
+> As the model is gated, before using it with diffusers you first need to go to the [Stable Diffusion 3 Medium Hugging Face page](https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers), fill in the form and accept the gate. Once you are in, you need to log in so that your system knows you’ve accepted the gate. Use the command below to log in:
 
 ```bash
 huggingface-cli login
 ```
 
-This will also allow us to push the trained model parameters to the Hugging Face Hub platform. 
+This will also allow us to push the trained model parameters to the Hugging Face Hub platform.
 
 ## Running locally with PyTorch
 
@@ -53,7 +53,9 @@ from accelerate.utils import write_basic_config
 write_basic_config()
 ```
 
-When running `accelerate config`, if we specify torch compile mode to True there can be dramatic speedups. 
+When running `accelerate config`, if we specify torch compile mode to True there can be dramatic speedups.
+Note also that we use PEFT library as backend for LoRA training, make sure to have `peft>=0.6.0` installed in your environment.
+
 
 ### Dog toy example
 
@@ -72,6 +74,8 @@ snapshot_download(
 )
 ```
 
+This will also allow us to push the trained LoRA parameters to the Hugging Face Hub platform.
+
 Now, we can launch training using:
 
 ```bash
@@ -102,9 +106,9 @@ accelerate launch train_dreambooth_sd3.py \
 To better track our training experiments, we're using the following flags in the command above:
 
 * `report_to="wandb` will ensure the training runs are tracked on Weights and Biases. To use it, be sure to install `wandb` with `pip install wandb`.
-* `validation_prompt` and `validation_epochs` to allow the script to do a few validation inference runs. This allows us to qualitatively check if the training is progressing as expected. 
+* `validation_prompt` and `validation_epochs` to allow the script to do a few validation inference runs. This allows us to qualitatively check if the training is progressing as expected.
 
-> [!NOTE]  
+> [!NOTE]
 > If you want to train using long prompts with the T5 text encoder, you can use `--max_sequence_length` to set the token limit. The default is 77, but it can be increased to as high as 512. Note that this will use more resources and may slow down the training in some cases.
 
 > [!TIP]
@@ -145,4 +149,4 @@ accelerate launch train_dreambooth_lora_sd3.py \
 
 ## Other notes
 
-We default to the "logit_normal" weighting scheme for the loss following the SD3 paper. Thanks to @bghira for helping us discover that for other weighting schemes supported from the training script, training may incur numerical instabilities. 
\ No newline at end of file
+We default to the "logit_normal" weighting scheme for the loss following the SD3 paper. Thanks to @bghira for helping us discover that for other weighting schemes supported from the training script, training may incur numerical instabilities.
\ No newline at end of file

examples/dreambooth/README_sdxl.md

@@ -4,7 +4,7 @@
 
 The `train_dreambooth_lora_sdxl.py` script shows how to implement the training procedure and adapt it for [Stable Diffusion XL](https://huggingface.co/papers/2307.01952).
 
-> 💡 **Note**: For now, we only allow DreamBooth fine-tuning of the SDXL UNet via LoRA. LoRA is a parameter-efficient fine-tuning technique introduced in [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685) by *Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen*. 
+> 💡 **Note**: For now, we only allow DreamBooth fine-tuning of the SDXL UNet via LoRA. LoRA is a parameter-efficient fine-tuning technique introduced in [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685) by *Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen*.
 
 ## Running locally with PyTorch
 
@@ -46,7 +46,7 @@ from accelerate.utils import write_basic_config
 write_basic_config()
 ```
 
-When running `accelerate config`, if we specify torch compile mode to True there can be dramatic speedups. 
+When running `accelerate config`, if we specify torch compile mode to True there can be dramatic speedups.
 Note also that we use PEFT library as backend for LoRA training, make sure to have `peft>=0.6.0` installed in your environment.
 
 ### Dog toy example
@@ -66,7 +66,7 @@ snapshot_download(
 )
 ```
 
-This will also allow us to push the trained LoRA parameters to the Hugging Face Hub platform. 
+This will also allow us to push the trained LoRA parameters to the Hugging Face Hub platform.
 
 Now, we can launch training using:
 
@@ -100,7 +100,7 @@ accelerate launch train_dreambooth_lora_sdxl.py \
 To better track our training experiments, we're using the following flags in the command above:
 
 * `report_to="wandb` will ensure the training runs are tracked on Weights and Biases. To use it, be sure to install `wandb` with `pip install wandb`.
-* `validation_prompt` and `validation_epochs` to allow the script to do a few validation inference runs. This allows us to qualitatively check if the training is progressing as expected. 
+* `validation_prompt` and `validation_epochs` to allow the script to do a few validation inference runs. This allows us to qualitatively check if the training is progressing as expected.
 
 Our experiments were conducted on a single 40GB A100 GPU.
 
@@ -153,7 +153,7 @@ lora_model_id = <"lora-sdxl-dreambooth-id">
 card = RepoCard.load(lora_model_id)
 base_model_id = card.data.to_dict()["base_model"]
 
-# Load the base pipeline and load the LoRA parameters into it. 
+# Load the base pipeline and load the LoRA parameters into it.
 pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16)
 pipe = pipe.to("cuda")
 pipe.load_lora_weights(lora_model_id)
@@ -205,11 +205,11 @@ You can explore the results from a couple of our internal experiments by checkin
 
 ## Running on a free-tier Colab Notebook
 
-Check out [this notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/SDXL_DreamBooth_LoRA_.ipynb). 
+Check out [this notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/SDXL_DreamBooth_LoRA_.ipynb).
 
 ## Conducting EDM-style training
 
-It's now possible to perform EDM-style training as proposed in [Elucidating the Design Space of Diffusion-Based Generative Models](https://arxiv.org/abs/2206.00364). 
+It's now possible to perform EDM-style training as proposed in [Elucidating the Design Space of Diffusion-Based Generative Models](https://arxiv.org/abs/2206.00364).
 
 For the SDXL model, simple set:
 
@@ -244,22 +244,22 @@ accelerate launch train_dreambooth_lora_sdxl.py \
 > [!CAUTION]
 > Min-SNR gamma is not supported with the EDM-style training yet. When training with the PlaygroundAI model, it's recommended to not pass any "variant".
 
-### DoRA training 
+### DoRA training
 The script now supports DoRA training too!
-> Proposed in [DoRA: Weight-Decomposed Low-Rank Adaptation](https://arxiv.org/abs/2402.09353), 
-**DoRA** is very similar to LoRA, except it decomposes the pre-trained weight into two components, **magnitude** and **direction** and employs LoRA for _directional_ updates to efficiently minimize the number of trainable parameters. 
-The authors found that by using DoRA, both the learning capacity and training stability of LoRA are enhanced without any additional overhead during inference. 
+> Proposed in [DoRA: Weight-Decomposed Low-Rank Adaptation](https://arxiv.org/abs/2402.09353),
+**DoRA** is very similar to LoRA, except it decomposes the pre-trained weight into two components, **magnitude** and **direction** and employs LoRA for _directional_ updates to efficiently minimize the number of trainable parameters.
+The authors found that by using DoRA, both the learning capacity and training stability of LoRA are enhanced without any additional overhead during inference.
 
 > [!NOTE]
-> 💡DoRA training is still _experimental_  
+> 💡DoRA training is still _experimental_
 > and is likely to require different hyperparameter values to perform best compared to a LoRA.
-> Specifically, we've noticed 2 differences to take into account your training: 
+> Specifically, we've noticed 2 differences to take into account your training:
 > 1. **LoRA seem to converge faster than DoRA** (so a set of parameters that may lead to overfitting when training a LoRA may be working well for a DoRA)
-> 2. **DoRA quality superior to LoRA especially in lower ranks** the difference in quality of DoRA of rank 8 and LoRA of rank 8 appears to be more significant than when training ranks of 32 or 64 for example.  
-> This is also aligned with some of the quantitative analysis shown in the paper. 
+> 2. **DoRA quality superior to LoRA especially in lower ranks** the difference in quality of DoRA of rank 8 and LoRA of rank 8 appears to be more significant than when training ranks of 32 or 64 for example.
+> This is also aligned with some of the quantitative analysis shown in the paper.
 
 **Usage**
-1. To use DoRA you need to upgrade the installation of `peft`: 
+1. To use DoRA you need to upgrade the installation of `peft`:
 ```bash
 pip install-U peft
 ```
@@ -267,7 +267,7 @@ pip install-U peft
 ```bash
 --use_dora
 ```
-**Inference** 
+**Inference**
 The inference is the same as if you train a regular LoRA 🤗
 
 ## Format compatibility

examples/instruct_pix2pix/README.md

@@ -58,7 +58,7 @@ write_basic_config()
 
 ### Toy example
 
-As mentioned before, we'll use a [small toy dataset](https://huggingface.co/datasets/fusing/instructpix2pix-1000-samples) for training. The dataset 
+As mentioned before, we'll use a [small toy dataset](https://huggingface.co/datasets/fusing/instructpix2pix-1000-samples) for training. The dataset
 is a smaller version of the [original dataset](https://huggingface.co/datasets/timbrooks/instructpix2pix-clip-filtered) used in the InstructPix2Pix paper.
 
 Configure environment variables such as the dataset identifier and the Stable Diffusion
@@ -109,7 +109,7 @@ accelerate launch --mixed_precision="fp16" train_instruct_pix2pix.py \
     --push_to_hub
  ```
 
- We recommend this type of validation as it can be useful for model debugging. Note that you need `wandb` installed to use this. You can install `wandb` by running `pip install wandb`. 
+ We recommend this type of validation as it can be useful for model debugging. Note that you need `wandb` installed to use this. You can install `wandb` by running `pip install wandb`.
 
  [Here](https://wandb.ai/sayakpaul/instruct-pix2pix/runs/ctr3kovq), you can find an example training run that includes some validation samples and the training hyperparameters.
 
@@ -120,7 +120,7 @@ accelerate launch --mixed_precision="fp16" train_instruct_pix2pix.py \
 `accelerate` allows for seamless multi-GPU training. Follow the instructions [here](https://huggingface.co/docs/accelerate/basic_tutorials/launch)
 for running distributed training with `accelerate`. Here is an example command:
 
-```bash 
+```bash
 accelerate launch --mixed_precision="fp16" --multi_gpu train_instruct_pix2pix.py \
  --pretrained_model_name_or_path=runwayml/stable-diffusion-v1-5 \
  --dataset_name=sayakpaul/instructpix2pix-1000-samples \
@@ -147,7 +147,7 @@ import requests
 import torch
 from diffusers import StableDiffusionInstructPix2PixPipeline
 
-model_id = "your_model_id" # <- replace this 
+model_id = "your_model_id" # <- replace this
 pipe = StableDiffusionInstructPix2PixPipeline.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda")
 generator = torch.Generator("cuda").manual_seed(0)
 
@@ -166,10 +166,10 @@ num_inference_steps = 20
 image_guidance_scale = 1.5
 guidance_scale = 10
 
-edited_image = pipe(prompt, 
-    image=image, 
-    num_inference_steps=num_inference_steps, 
-    image_guidance_scale=image_guidance_scale, 
+edited_image = pipe(prompt,
+    image=image,
+    num_inference_steps=num_inference_steps,
+    image_guidance_scale=image_guidance_scale,
     guidance_scale=guidance_scale,
     generator=generator,
 ).images[0]
@@ -189,7 +189,7 @@ speed and quality during performance:
 Particularly, `image_guidance_scale` and `guidance_scale` can have a profound impact
 on the generated ("edited") image (see [here](https://twitter.com/RisingSayak/status/1628392199196151808?s=20) for an example).
 
-If you're looking for some interesting ways to use the InstructPix2Pix training methodology, we welcome you to check out this blog post: [Instruction-tuning Stable Diffusion with InstructPix2Pix](https://huggingface.co/blog/instruction-tuning-sd). 
+If you're looking for some interesting ways to use the InstructPix2Pix training methodology, we welcome you to check out this blog post: [Instruction-tuning Stable Diffusion with InstructPix2Pix](https://huggingface.co/blog/instruction-tuning-sd).
 
 ## Stable Diffusion XL
 

examples/instruct_pix2pix/README_sdxl.md

@@ -2,7 +2,7 @@
 
 ***This is based on the original InstructPix2Pix training example.***
 
-[Stable Diffusion XL](https://huggingface.co/papers/2307.01952) (or SDXL) is the latest image generation model that is tailored towards more photorealistic outputs with more detailed imagery and composition compared to previous SD models. It leverages a three times larger UNet backbone. The increase of model parameters is mainly due to more attention blocks and a larger cross-attention context as SDXL uses a second text encoder. 
+[Stable Diffusion XL](https://huggingface.co/papers/2307.01952) (or SDXL) is the latest image generation model that is tailored towards more photorealistic outputs with more detailed imagery and composition compared to previous SD models. It leverages a three times larger UNet backbone. The increase of model parameters is mainly due to more attention blocks and a larger cross-attention context as SDXL uses a second text encoder.
 
 The `train_instruct_pix2pix_sdxl.py` script shows how to implement the training procedure and adapt it for Stable Diffusion XL.
 
@@ -15,11 +15,11 @@ training procedure while being faithful to the [original implementation](https:/
 
 Refer to the original InstructPix2Pix training example for installing the dependencies.
 
-You will also need to get access of SDXL by filling the [form](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0). 
+You will also need to get access of SDXL by filling the [form](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0).
 
 ### Toy example
 
-As mentioned before, we'll use a [small toy dataset](https://huggingface.co/datasets/fusing/instructpix2pix-1000-samples) for training. The dataset 
+As mentioned before, we'll use a [small toy dataset](https://huggingface.co/datasets/fusing/instructpix2pix-1000-samples) for training. The dataset
 is a smaller version of the [original dataset](https://huggingface.co/datasets/timbrooks/instructpix2pix-clip-filtered) used in the InstructPix2Pix paper.
 
 Configure environment variables such as the dataset identifier and the Stable Diffusion
@@ -69,7 +69,7 @@ accelerate launch train_instruct_pix2pix_sdxl.py \
     --push_to_hub
  ```
 
- We recommend this type of validation as it can be useful for model debugging. Note that you need `wandb` installed to use this. You can install `wandb` by running `pip install wandb`. 
+ We recommend this type of validation as it can be useful for model debugging. Note that you need `wandb` installed to use this. You can install `wandb` by running `pip install wandb`.
 
  [Here](https://wandb.ai/sayakpaul/instruct-pix2pix-sdxl-new/runs/sw53gxmc), you can find an example training run that includes some validation samples and the training hyperparameters.
 
@@ -80,7 +80,7 @@ accelerate launch train_instruct_pix2pix_sdxl.py \
 `accelerate` allows for seamless multi-GPU training. Follow the instructions [here](https://huggingface.co/docs/accelerate/basic_tutorials/launch)
 for running distributed training with `accelerate`. Here is an example command:
 
-```bash 
+```bash
 accelerate launch --mixed_precision="fp16" --multi_gpu train_instruct_pix2pix_sdxl.py \
     --pretrained_model_name_or_path=stabilityai/stable-diffusion-xl-base-1.0 \
     --dataset_name=$DATASET_ID \
@@ -109,7 +109,7 @@ import requests
 import torch
 from diffusers import StableDiffusionXLInstructPix2PixPipeline
 
-model_id = "your_model_id" # <- replace this 
+model_id = "your_model_id" # <- replace this
 pipe = StableDiffusionXLInstructPix2PixPipeline.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda")
 generator = torch.Generator("cuda").manual_seed(0)
 
@@ -128,10 +128,10 @@ num_inference_steps = 20
 image_guidance_scale = 1.5
 guidance_scale = 10
 
-edited_image = pipe(prompt, 
-    image=image, 
-    num_inference_steps=num_inference_steps, 
-    image_guidance_scale=image_guidance_scale, 
+edited_image = pipe(prompt,
+    image=image,
+    num_inference_steps=num_inference_steps,
+    image_guidance_scale=image_guidance_scale,
     guidance_scale=guidance_scale,
     generator=generator,
 ).images[0]
@@ -148,7 +148,7 @@ speed and quality during performance:
 Particularly, `image_guidance_scale` and `guidance_scale` can have a profound impact
 on the generated ("edited") image (see [here](https://twitter.com/RisingSayak/status/1628392199196151808?s=20) for an example).
 
-If you're looking for some interesting ways to use the InstructPix2Pix training methodology, we welcome you to check out this blog post: [Instruction-tuning Stable Diffusion with InstructPix2Pix](https://huggingface.co/blog/instruction-tuning-sd). 
+If you're looking for some interesting ways to use the InstructPix2Pix training methodology, we welcome you to check out this blog post: [Instruction-tuning Stable Diffusion with InstructPix2Pix](https://huggingface.co/blog/instruction-tuning-sd).
 
 ## Compare between SD and SDXL
 
@@ -178,7 +178,7 @@ accelerate launch train_instruct_pix2pix.py \
 
 We discovered that compared to training with SD-1.5 as the pretrained model, SDXL-0.9 results in a lower training loss value (SD-1.5 yields 0.0599, SDXL scores 0.0254). Moreover, from a visual perspective, the results obtained using SDXL demonstrated fewer artifacts and a richer detail. Notably, SDXL starts to preserve the structure of the original image earlier on.
 
-The following two GIFs provide intuitive visual results. We observed, for each step, what kind of results could be achieved using the image 
+The following two GIFs provide intuitive visual results. We observed, for each step, what kind of results could be achieved using the image
 <p align="center">
     <img src="https://datasets-server.huggingface.co/assets/fusing/instructpix2pix-1000-samples/--/fusing--instructpix2pix-1000-samples/train/23/input_image/image.jpg" alt="input for make it Japan" width=600/>
 </p>

examples/research_projects/diffusion_dpo/README.md

@@ -7,7 +7,7 @@ We provide implementations for both Stable Diffusion (SD) and Stable Diffusion X
 * [mhdang/dpo-sd1.5-text2image-v1](https://huggingface.co/mhdang/dpo-sd1.5-text2image-v1)
 * [mhdang/dpo-sdxl-text2image-v1](https://huggingface.co/mhdang/dpo-sdxl-text2image-v1)
 
-> 💡 Note: The scripts are highly experimental and were only tested on low-data regimes. Proceed with caution. Feel free to let us know about your findings via GitHub issues. 
+> 💡 Note: The scripts are highly experimental and were only tested on low-data regimes. Proceed with caution. Feel free to let us know about your findings via GitHub issues.
 
 ## SD training command
 
@@ -91,4 +91,4 @@ accelerate launch train_diffusion_dpo_sdxl.py \
 
 ## Acknowledgements
 
-This is based on the amazing work done by [Bram](https://github.com/bram-w) here for Diffusion DPO: https://github.com/bram-w/trl/blob/dpo/. 
+This is based on the amazing work done by [Bram](https://github.com/bram-w) here for Diffusion DPO: https://github.com/bram-w/trl/blob/dpo/.

examples/research_projects/dreambooth_inpaint/README.md

@@ -86,7 +86,7 @@ accelerate launch train_dreambooth_inpaint.py \
 
 ### Fine-tune text encoder with the UNet.
 
-The script also allows to fine-tune the `text_encoder` along with the `unet`. It's been observed experimentally that fine-tuning `text_encoder` gives much better results especially on faces. 
+The script also allows to fine-tune the `text_encoder` along with the `unet`. It's been observed experimentally that fine-tuning `text_encoder` gives much better results especially on faces.
 Pass the `--train_text_encoder` argument to the script to enable training `text_encoder`.
 
 ___Note: Training text encoder requires more memory, with this option the training won't fit on 16GB GPU. It needs at least 24GB VRAM.___

examples/research_projects/instructpix2pix_lora/README.md

@@ -28,7 +28,7 @@ accelerate launch finetune_instruct_pix2pix.py \
 ```
 
 ## Inference
-After training the model and the lora weight of the model is stored in the ```$OUTPUT_DIR```. 
+After training the model and the lora weight of the model is stored in the ```$OUTPUT_DIR```.
 
 ```bash
 # load the base model pipeline
@@ -42,12 +42,11 @@ input_image_path = "/path/to/input_image"
 input_image = Image.open(input_image_path)
 edited_images = pipe_lora(num_images_per_prompt=1, prompt=args.edit_prompt, image=input_image, num_inference_steps=1000).images
 edited_images[0].show()
-
 ```
 
 ## Results
 
-Here is an example of using the script to train a instructpix2pix model. 
+Here is an example of using the script to train a instructpix2pix model.
 Trained on google colab T4 GPU
 
 ```bash
@@ -69,7 +68,7 @@ Here are some rough statistics about the training model using this script
     <img src="https://github.com/Aiden-Frost/Efficiently-teaching-counting-and-cartoonization-to-InstructPix2Pix.-/blob/main/diffusers_result_assets/results.png?raw=true" alt="instructpix2pix-inputs" width=600/>
 </p>
 
-## References 
+## References
 
 * InstructPix2Pix - https://github.com/timothybrooks/instruct-pix2pix
 * Dataset and example training script - https://huggingface.co/blog/instruction-tuning-sd

examples/research_projects/intel_opts/README.md

@@ -29,7 +29,6 @@ export MALLOC_CONF="oversize_threshold:1,background_thread:true,metadata_thp:aut
 numactl --membind <node N> -C <cpu list> python python inference_bf16.py
 # Launch with DPMSolverMultistepScheduler
 numactl --membind <node N> -C <cpu list> python python inference_bf16.py --dpm
-
 ```
 
 ## Accelerating the inference for Stable Diffusion using INT8

examples/research_projects/intel_opts/textual_inversion/README.md

@@ -52,7 +52,7 @@ python -m intel_extension_for_pytorch.cpu.launch --distributed \
   --train_data_dir=$DATA_DIR \
   --learnable_property="object" \
   --placeholder_token="<dicoo>" --initializer_token="toy" \
-  --seed=7 \ 
+  --seed=7 \
   --resolution=512 \
   --train_batch_size=1 \
   --gradient_accumulation_steps=1 \

examples/research_projects/multi_subject_dreambooth/README.md

@@ -86,17 +86,17 @@ This example shows training for 2 subjects, but please note that the model can b
 
 Note also that in this script, `sks` and `t@y` were used as tokens to learn the new subjects ([this thread](https://github.com/XavierXiao/Dreambooth-Stable-Diffusion/issues/71) inspired the use of `t@y` as our second identifier). However, there may be better rare tokens to experiment with, and results also seemed to be good when more intuitive words are used.
 
-**Important**: New parameters are added to the script, making possible to validate the progress of the training by 
+**Important**: New parameters are added to the script, making possible to validate the progress of the training by
 generating images at specified steps. Taking also into account that a comma separated list in a text field for a prompt
-it's never a good idea (simply because it is very common in prompts to have them as part of a regular text) we 
-introduce the `concept_list` parameter: allowing to specify a json-like file where you can define the different 
+it's never a good idea (simply because it is very common in prompts to have them as part of a regular text) we
+introduce the `concept_list` parameter: allowing to specify a json-like file where you can define the different
 configuration for each subject that you want to train.
 
 An example of how to generate the file:
 ```python
 import json
 
-# here we are using parameters for prior-preservation and validation as well. 
+# here we are using parameters for prior-preservation and validation as well.
 concepts_list = [
     {
         "instance_prompt":      "drawing of a t@y meme",
@@ -290,7 +290,7 @@ accelerate launch --mixed_precision="fp16" train_dreambooth.py \
 
 ### Fine-tune text encoder with the UNet.
 
-The script also allows to fine-tune the `text_encoder` along with the `unet`. It's been observed experimentally that fine-tuning `text_encoder` gives much better results especially on faces. 
+The script also allows to fine-tune the `text_encoder` along with the `unet`. It's been observed experimentally that fine-tuning `text_encoder` gives much better results especially on faces.
 Pass the `--train_text_encoder` argument to the script to enable training `text_encoder`.
 
 ___Note: Training text encoder requires more memory, with this option the training won't fit on 16GB GPU. It needs at least 24GB VRAM.___

examples/text_to_image/README_sdxl.md

@@ -239,7 +239,6 @@ accelerate launch  --config_file $ACCELERATE_CONFIG_FILE train_text_to_image_lor
   --seed=1234 \
   --output_dir="sd-naruto-model-lora-sdxl" \
   --validation_prompt="cute dragon creature"
-
 ```