Unverified Commit 2f234376 authored by kabachuha's avatar kabachuha Committed by GitHub
Browse files

Add (Scheduled) Pseudo-Huber Loss training scripts to research projects (#7527)

* add scheduled pseudo-huber loss training scripts

See #7488

* add reduction modes to huber loss

* [DB Lora] *2 multiplier to huber loss cause of 1/2 a^2 conv.

pairing of https://github.com/kohya-ss/sd-scripts/pull/1228/commits/c6495def1fbbaf2a0233110d50f976ed61620e83

* [DB Lora] add option for smooth l1 (huber / delta)

Pairing of https://github.com/kohya-ss/sd-scripts/pull/1228/commits/dd22958caa56e4db885324f76188c13bdf504569

* [DB Lora] unify huber scheduling

Pairing of https://github.com/kohya-ss/sd-scripts/pull/1228/commits/19a834c3ab448614e8887b07f2bb4e0aaabf0805

* [DB Lora] add snr huber scheduler

Pairing of https://github.com/kohya-ss/sd-scripts/pull/1228/commits/47fb1a68547e76f33cd54a3da8d2c35b9489c56e



* fixup examples link

* use snr schedule by default in DB

* update all huber scripts with snr

* code quality

* huber: make style && make quality

---------
Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
parent 2523390c
# Scheduled Pseudo-Huber Loss for Diffusers
These are the modifications of to include the possibility of training text2image models with Scheduled Pseudo Huber loss, introduced in https://arxiv.org/abs/2403.16728. (https://github.com/kabachuha/SPHL-for-stable-diffusion)
## Why this might be useful?
- If you suspect that the part of the training dataset might be corrupted, and you don't want these outliers to distort the model's supposed output
- If you want to improve the aesthetic quality of pictures by helping the model disentangle concepts and be less influenced by another sorts of pictures.
See https://github.com/huggingface/diffusers/issues/7488 for the detailed description.
## Instructions
The same usage as in the case of the corresponding vanilla Diffusers scripts https://github.com/huggingface/diffusers/tree/main/examples
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment