"git@developer.sourcefind.cn:OpenDAS/pytorch3d.git" did not exist on "7cbda3ec174f52831ffc4a5677afecc69b446191"
Unverified Commit fddbab79 authored by Sayak Paul's avatar Sayak Paul Committed by GitHub
Browse files

[research_projects] Update README.md to include a note about NF5 T5-xxl (#9775)

Update README.md
parent 298ab6eb
...@@ -5,7 +5,8 @@ ...@@ -5,7 +5,8 @@
This example shows how to fine-tune [Flux.1 Dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) with LoRA and quantization. We show this by using the [`Norod78/Yarn-art-style`](https://huggingface.co/datasets/Norod78/Yarn-art-style) dataset. Steps below summarize the workflow: This example shows how to fine-tune [Flux.1 Dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) with LoRA and quantization. We show this by using the [`Norod78/Yarn-art-style`](https://huggingface.co/datasets/Norod78/Yarn-art-style) dataset. Steps below summarize the workflow:
* We precompute the text embeddings in `compute_embeddings.py` and serialize them into a parquet file. * We precompute the text embeddings in `compute_embeddings.py` and serialize them into a parquet file.
* Even though optional, we load the T5-xxl in NF4 to further reduce the memory foot-print.
* `train_dreambooth_lora_flux_miniature.py` takes care of training: * `train_dreambooth_lora_flux_miniature.py` takes care of training:
* Since we already precomputed the text embeddings, we don't load the text encoders. * Since we already precomputed the text embeddings, we don't load the text encoders.
* We load the VAE and use it to precompute the image latents and we then delete it. * We load the VAE and use it to precompute the image latents and we then delete it.
...@@ -163,4 +164,4 @@ image.save("yarn_merged.png") ...@@ -163,4 +164,4 @@ image.save("yarn_merged.png")
|-------|-------| |-------|-------|
| ![Image A](https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/quantized_flux_training/merged.png) | ![Image B](https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/quantized_flux_training/unmerged.png) | | ![Image A](https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/quantized_flux_training/merged.png) | ![Image B](https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/quantized_flux_training/unmerged.png) |
As we can notice the first column result follows the style more closely. As we can notice the first column result follows the style more closely.
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment