GitLab

Let me know if this breaks anything.

Use fp16 text encoder weights for CPU inference to lower memory usage.

--fp8_e4m3fn-unet and --fp8_e5m2-unet are the two different formats
supported by pytorch.

Pytorch supports two variants of fp8:
--fp8_e4m3fn-text-enc (the one that seems to give better results)
--fp8_e5m2-text-enc

Do inference in fp32 to make sure quality stays the exact same.