[major] lora conversion script released; upgrade the model; release flux.1-redux model

README.md

@@ -4,6 +4,7 @@ Nunchaku is an inference engine designed for 4-bit diffusion models, as demonstr
 
 ### [Paper](http://arxiv.org/abs/2411.05007) | [Project](https://hanlab.mit.edu/projects/svdquant) | [Blog](https://hanlab.mit.edu/blog/svdquant) | [Demo](https://svdquant.mit.edu)
 
+- **[2025-02-14]** 🔥 [LoRA conversion script](nunchaku/convert_lora.py) is now available!
 - **[2025-02-11]** 🎉 **[SVDQuant](http://arxiv.org/abs/2411.05007) has been selected as a ICLR 2025 Spotlight! FLUX.1-tools Gradio demos are now available!** Check [here](#gradio-demos) for the usage details! Our new [depth-to-image demo](https://svdquant.mit.edu/flux1-depth-dev/) is also online—try it out!
 - **[2025-02-04]** **🚀 4-bit [FLUX.1-tools](https://blackforestlabs.ai/flux-1-tools/) is here!** Enjoy a **2-3× speedup** over the original models. Check out the [examples](./examples) for usage. **ComfyUI integration is coming soon!**
 - **[2025-01-23]** 🚀 **4-bit [SANA](https://nvlabs.github.io/Sana/) support is here!** Experience a 2-3× speedup compared to the 16-bit model. Check out the [usage example](./examples/sana_1600m_pag.py) and the [deployment guide](app/sana/t2i) for more details. Explore our live demo at [svdquant.mit.edu](https://svdquant.mit.edu)!
@@ -116,7 +117,7 @@ Please refer to [app/flux/t2i/README.md](app/flux/t2i/README.md) for instruction
 
 - [ ] Easy installation
 - [x] Comfy UI node
-- [ ] Customized LoRA conversion instructions
+- [x] Customized LoRA conversion instructions
 - [ ] Customized model quantization instructions
 - [x] FLUX.1 tools support
 - [ ] Modularization

app/flux.1/depth_canny/assets/description.html

@@ -10,36 +10,36 @@
             SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
         </h2>
         <h3>
-            <a href='https://lmxyy.me'>Muyang Li*</a>,
-            <a href='https://yujunlin.com'>Yujun Lin*</a>,
-            <a href='https://hanlab.mit.edu/team/zhekai-zhang'>Zhekai Zhang*</a>,
-            <a href='https://www.tianle.website/#/'>Tianle Cai</a>,
-            <a href='https://xiuyuli.com'>Xiuyu Li</a>,
+            <a href='https://lmxyy.me' target="_blank">Muyang Li*</a>,
+            <a href='https://yujunlin.com' target="_blank">Yujun Lin*</a>,
+            <a href='https://hanlab.mit.edu/team/zhekai-zhang' target="_blank">Zhekai Zhang*</a>,
+            <a href='https://www.tianle.website/#/' target="_blank">Tianle Cai</a>,
+            <a href='https://xiuyuli.com' target="_blank">Xiuyu Li</a>,
             <br>
-            <a href='https://github.com/JerryGJX'>Junxian Guo</a>,
-            <a href='https://xieenze.github.io'>Enze Xie</a>,
-            <a href='https://cs.stanford.edu/~chenlin/'>Chenlin Meng</a>,
-            <a href='https://www.cs.cmu.edu/~junyanz/'>Jun-Yan Zhu</a>,
-            and <a href='https://hanlab.mit.edu/songhan'>Song Han</a>
+            <a href='https://github.com/JerryGJX' target="_blank">Junxian Guo</a>,
+            <a href='https://xieenze.github.io' target="_blank">Enze Xie</a>,
+            <a href='https://cs.stanford.edu/~chenlin/' target="_blank">Chenlin Meng</a>,
+            <a href='https://www.cs.cmu.edu/~junyanz/' target="_blank">Jun-Yan Zhu</a>,
+            and <a href='https://hanlab.mit.edu/songhan' target="_blank">Song Han</a>
         </h3>
         <div style="display: flex; justify-content: center; align-items: center; text-align: center;">
-            <a href="https://arxiv.org/abs/2411.05007">[Paper]</a>
+            <a href="https://arxiv.org/abs/2411.05007" target="_blank">[Paper]</a>
             &nbsp;
-            <a href='https://github.com/mit-han-lab/nunchaku'>
+            <a href='https://github.com/mit-han-lab/nunchaku' target="_blank">
                 [Code]
             </a>
             &nbsp;
-            <a href='https://hanlab.mit.edu/projects/svdquant'>
+            <a href='https://hanlab.mit.edu/projects/svdquant' target="_blank">
                 [Website]
             </a>
             &nbsp;
-            <a href='https://hanlab.mit.edu/blog/svdquant'>
+            <a href='https://hanlab.mit.edu/blog/svdquant' target="_blank">
                 [Blog]
             </a>
         </div>
         <h4>Quantization Library:
-            <a href='https://github.com/mit-han-lab/deepcompressor'>DeepCompressor</a>&nbsp;
-            Inference Engine: <a href='https://github.com/mit-han-lab/nunchaku'>Nunchaku</a>&nbsp;
+            <a href='https://github.com/mit-han-lab/deepcompressor' target="_blank">DeepCompressor</a>&nbsp;
+            Inference Engine: <a href='https://github.com/mit-han-lab/nunchaku' target="_blank">Nunchaku</a>&nbsp;
         </h4>
         <div style="display: flex; justify-content: center; align-items: center; text-align: center;">
             {device_info}

app/flux.1/depth_canny/assets/style.css

 @import url('https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.1/css/all.min.css');
 
-.gradio-container{max-width: 1200px !important}
-h1{text-align:center}
+.gradio-container {
+    max-width: 1200px !important;
+    margin: auto; /* Centers the element horizontally */
+}
+
+h1 {
+    text-align: center
+}
 
 .wrap.svelte-p4aq0j.svelte-p4aq0j {
     display: none;
@@ -22,8 +28,13 @@ h1{text-align:center}
 
 #accessibility {
     text-align: center; /* Center-aligns the text */
-    margin: auto;       /* Centers the element horizontally */
+    margin: auto; /* Centers the element horizontally */
+}
+
+#random_seed {
+    height: 71px;
 }
 
-#random_seed {height: 71px;}
-#run_button {height: 87px;}
+#run_button {
+    height: 87px;
\ No newline at end of file
+}
\ No newline at end of file

app/flux.1/depth_canny/run_gradio.py

@@ -137,8 +137,8 @@ with gr.Blocks(css_paths="assets/style.css", title=f"SVDQuant Flux.1-{model_name
     def get_header_str():
 
         if args.count_use:
-            if os.path.exists("use_count.txt"):
-                with open("use_count.txt", "r") as f:
+            if os.path.exists(f"{args.model}-use_count.txt"):
+                with open(f"{args.model}-use_count.txt", "r") as f:
                     count = int(f.read())
             else:
                 count = 0

app/flux.1/depth_canny/vars.py

@@ -65,7 +65,7 @@ EXAMPLES = {
         ],
         [
             "https://huggingface.co/mit-han-lab/svdq-int4-flux.1-fill-dev/resolve/main/example.png",
-            "A wooden basked of several individual cartons of strawberries.",
+            "A wooden basket of several individual cartons of strawberries.",
             DEFAULT_STYLE_NAME,
             STYLES[DEFAULT_STYLE_NAME],
             30,

app/flux.1/fill/assets/description.html

@@ -10,36 +10,36 @@
             SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
         </h2>
         <h3>
-            <a href='https://lmxyy.me'>Muyang Li*</a>,
-            <a href='https://yujunlin.com'>Yujun Lin*</a>,
-            <a href='https://hanlab.mit.edu/team/zhekai-zhang'>Zhekai Zhang*</a>,
-            <a href='https://www.tianle.website/#/'>Tianle Cai</a>,
-            <a href='https://xiuyuli.com'>Xiuyu Li</a>,
+            <a href='https://lmxyy.me' target="_blank">Muyang Li*</a>,
+            <a href='https://yujunlin.com' target="_blank">Yujun Lin*</a>,
+            <a href='https://hanlab.mit.edu/team/zhekai-zhang' target="_blank">Zhekai Zhang*</a>,
+            <a href='https://www.tianle.website/#/' target="_blank">Tianle Cai</a>,
+            <a href='https://xiuyuli.com' target="_blank">Xiuyu Li</a>,
             <br>
-            <a href='https://github.com/JerryGJX'>Junxian Guo</a>,
-            <a href='https://xieenze.github.io'>Enze Xie</a>,
-            <a href='https://cs.stanford.edu/~chenlin/'>Chenlin Meng</a>,
-            <a href='https://www.cs.cmu.edu/~junyanz/'>Jun-Yan Zhu</a>,
-            and <a href='https://hanlab.mit.edu/songhan'>Song Han</a>
+            <a href='https://github.com/JerryGJX' target="_blank">Junxian Guo</a>,
+            <a href='https://xieenze.github.io' target="_blank">Enze Xie</a>,
+            <a href='https://cs.stanford.edu/~chenlin/' target="_blank">Chenlin Meng</a>,
+            <a href='https://www.cs.cmu.edu/~junyanz/' target="_blank">Jun-Yan Zhu</a>,
+            and <a href='https://hanlab.mit.edu/songhan' target="_blank">Song Han</a>
         </h3>
         <div style="display: flex; justify-content: center; align-items: center; text-align: center;">
-            <a href="https://arxiv.org/abs/2411.05007">[Paper]</a>
+            <a href="https://arxiv.org/abs/2411.05007" target="_blank">[Paper]</a>
             &nbsp;
-            <a href='https://github.com/mit-han-lab/nunchaku'>
+            <a href='https://github.com/mit-han-lab/nunchaku' target="_blank">
                 [Code]
             </a>
             &nbsp;
-            <a href='https://hanlab.mit.edu/projects/svdquant'>
+            <a href='https://hanlab.mit.edu/projects/svdquant' target="_blank">
                 [Website]
             </a>
             &nbsp;
-            <a href='https://hanlab.mit.edu/blog/svdquant'>
+            <a href='https://hanlab.mit.edu/blog/svdquant' target="_blank">
                 [Blog]
             </a>
         </div>
         <h4>Quantization Library:
-            <a href='https://github.com/mit-han-lab/deepcompressor'>DeepCompressor</a>&nbsp;
-            Inference Engine: <a href='https://github.com/mit-han-lab/nunchaku'>Nunchaku</a>&nbsp;
+            <a href='https://github.com/mit-han-lab/deepcompressor' target="_blank">DeepCompressor</a>&nbsp;
+            Inference Engine: <a href='https://github.com/mit-han-lab/nunchaku' target="_blank">Nunchaku</a>&nbsp;
         </h4>
         <div style="display: flex; justify-content: center; align-items: center; text-align: center;">
             {device_info}

app/flux.1/fill/assets/style.css

 @import url('https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.1/css/all.min.css');
 
-.gradio-container{max-width: 1200px !important}
-h1{text-align:center}
+.gradio-container {
+    max-width: 1200px !important;
+    margin: auto; /* Centers the element horizontally */
+}
+
+h1 {
+    text-align: center
+}
 
 .wrap.svelte-p4aq0j.svelte-p4aq0j {
     display: none;
@@ -22,8 +28,13 @@ h1{text-align:center}
 
 #accessibility {
     text-align: center; /* Center-aligns the text */
-    margin: auto;       /* Centers the element horizontally */
+    margin: auto; /* Centers the element horizontally */
+}
+
+#random_seed {
+    height: 71px;
 }
 
-#random_seed {height: 71px;}
-#run_button {height: 87px;}
+#run_button {
+    height: 87px;
\ No newline at end of file
+}
\ No newline at end of file

app/flux.1/redux/run_gradio.py

@@ -23,17 +23,13 @@ pipe_prior_redux = FluxPriorReduxPipeline.from_pretrained(
 ).to("cuda")
 
 if args.precision == "bf16":
-    pipeline = FluxPipeline.from_pretrained(
-        "black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16
-    )
+    pipeline = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16)
     pipeline = pipeline.to("cuda")
     pipeline.precision = "bf16"
 else:
     assert args.precision == "int4"
     pipeline_init_kwargs = {}
-    transformer = NunchakuFluxTransformer2dModel.from_pretrained(
-        "mit-han-lab/svdq-int4-flux.1-dev"
-    )
+    transformer = NunchakuFluxTransformer2dModel.from_pretrained("mit-han-lab/svdq-int4-flux.1-dev")
     pipeline = FluxPipeline.from_pretrained(
         "black-forest-labs/FLUX.1-dev",
         text_encoder=None,
@@ -45,9 +41,7 @@ else:
     pipeline.precision = "int4"
 
 
-def run(
-    image, num_inference_steps: int, guidance_scale: float, seed: int
-) -> tuple[Image, str]:
+def run(image, num_inference_steps: int, guidance_scale: float, seed: int) -> tuple[Image, str]:
     pipe_prior_output = pipe_prior_redux(image["composite"])
 
     start_time = time.time()
@@ -82,9 +76,7 @@ def run(
     return result_image, latency_str
 
 
-with gr.Blocks(
-    css_paths="assets/style.css", title=f"SVDQuant Flux.1-redux-dev Demo"
-) as demo:
+with gr.Blocks(css_paths="assets/style.css", title=f"SVDQuant Flux.1-redux-dev Demo") as demo:
     with open("assets/description.html", "r") as f:
         DESCRIPTION = f.read()
     gpus = GPUtil.getGPUs()
@@ -148,9 +140,7 @@ with gr.Blocks(
                     step=1,
                     scale=4,
                 )
-                randomize_seed = gr.Button(
-                    "Random Seed", scale=1, min_width=50, elem_id="random_seed"
-                )
+                randomize_seed = gr.Button("Random Seed", scale=1, min_width=50, elem_id="random_seed")
             with gr.Accordion("Advanced options", open=False):
                 with gr.Group():
                     num_inference_steps = gr.Slider(
@@ -204,17 +194,9 @@ with gr.Blocks(
         queue=False,
     ).then(run, inputs=run_inputs, outputs=run_outputs, api_name=False)
 
-    gr.on(
-        triggers=[run_button.click],
-        fn=run,
-        inputs=run_inputs,
-        outputs=run_outputs,
-        api_name=False,
-    )
+    gr.on(triggers=[run_button.click], fn=run, inputs=run_inputs, outputs=run_outputs, api_name=False)
 
-    gr.Markdown(
-        "MIT Accessibility: https://accessibility.mit.edu/", elem_id="accessibility"
-    )
+    gr.Markdown("MIT Accessibility: https://accessibility.mit.edu/", elem_id="accessibility")
 
 
 if __name__ == "__main__":

app/flux.1/redux/vars.py

@@ -3,10 +3,16 @@ DEFAULT_GUIDANCE = 2.5
 DEFAULT_INFERENCE_STEP = 50
 
 EXAMPLES = [
+    [
+        "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/robot.png",
+        DEFAULT_INFERENCE_STEP,
+        DEFAULT_GUIDANCE,
+        1,
+    ],
     [
         "https://huggingface.co/mit-han-lab/svdq-int4-flux.1-fill-dev/resolve/main/example.png",
         DEFAULT_INFERENCE_STEP,
         DEFAULT_GUIDANCE,
         1,
-    ]
+    ],
 ]

app/flux.1/sketch/assets/description.html

@@ -10,37 +10,37 @@
             SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
         </h2>
         <h3>
-            <a href='https://lmxyy.me'>Muyang Li*</a>,
-            <a href='https://yujunlin.com'>Yujun Lin*</a>,
-            <a href='https://hanlab.mit.edu/team/zhekai-zhang'>Zhekai Zhang*</a>,
-            <a href='https://www.tianle.website/#/'>Tianle Cai</a>,
-            <a href='https://xiuyuli.com'>Xiuyu Li</a>,
+            <a href='https://lmxyy.me' target="_blank">Muyang Li*</a>,
+            <a href='https://yujunlin.com' target="_blank">Yujun Lin*</a>,
+            <a href='https://hanlab.mit.edu/team/zhekai-zhang' target="_blank">Zhekai Zhang*</a>,
+            <a href='https://www.tianle.website/#/' target="_blank">Tianle Cai</a>,
+            <a href='https://xiuyuli.com' target="_blank">Xiuyu Li</a>,
             <br>
-            <a href='https://github.com/JerryGJX'>Junxian Guo</a>,
-            <a href='https://xieenze.github.io'>Enze Xie</a>,
-            <a href='https://cs.stanford.edu/~chenlin/'>Chenlin Meng</a>,
-            <a href='https://www.cs.cmu.edu/~junyanz/'>Jun-Yan Zhu</a>,
-            and <a href='https://hanlab.mit.edu/songhan'>Song Han</a>
+            <a href='https://github.com/JerryGJX' target="_blank">Junxian Guo</a>,
+            <a href='https://xieenze.github.io' target="_blank">Enze Xie</a>,
+            <a href='https://cs.stanford.edu/~chenlin/' target="_blank">Chenlin Meng</a>,
+            <a href='https://www.cs.cmu.edu/~junyanz/' target="_blank">Jun-Yan Zhu</a>,
+            and <a href='https://hanlab.mit.edu/songhan' target="_blank">Song Han</a>
         </h3>
         <div style="display: flex; justify-content: center; align-items: center; text-align: center;">
-            <a href="https://arxiv.org/abs/2411.05007">[Paper]</a>
+            <a href="https://arxiv.org/abs/2411.05007" target="_blank">[Paper]</a>
             &nbsp;
-            <a href='https://github.com/mit-han-lab/nunchaku'>
+            <a href='https://github.com/mit-han-lab/nunchaku' target="_blank">
                 [Code]
             </a>
             &nbsp;
-            <a href='https://hanlab.mit.edu/projects/svdquant'>
+            <a href='https://hanlab.mit.edu/projects/svdquant' target="_blank">
                 [Website]
             </a>
             &nbsp;
-            <a href='https://hanlab.mit.edu/blog/svdquant'>
+            <a href='https://hanlab.mit.edu/blog/svdquant' target="_blank">
                 [Blog]
             </a>
         </div>
         <h4>Quantization Library:
-            <a href='https://github.com/mit-han-lab/deepcompressor'>DeepCompressor</a>&nbsp;
-            Inference Engine: <a href='https://github.com/mit-han-lab/nunchaku'>Nunchaku</a>&nbsp;
-            Image Control: <a href="https://github.com/GaParmar/img2img-turbo">img2img-turbo</a>
+            <a href='https://github.com/mit-han-lab/deepcompressor' target="_blank">DeepCompressor</a>&nbsp;
+            Inference Engine: <a href='https://github.com/mit-han-lab/nunchaku' target="_blank">Nunchaku</a>&nbsp;
+            Image Control: <a href="https://github.com/GaParmar/img2img-turbo" target="_blank">img2img-turbo</a>
         </h4>
         <div style="display: flex; justify-content: center; align-items: center; text-align: center;">
             {device_info}

app/flux.1/sketch/assets/style.css

 @import url('https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.1/css/all.min.css');
 
-.gradio-container{max-width: 1200px !important}
-h1{text-align:center}
+.gradio-container {
+    max-width: 1200px !important;
+    margin: auto; /* Centers the element horizontally */
+}
+
+h1 {
+    text-align: center
+}
 
 .wrap.svelte-p4aq0j.svelte-p4aq0j {
     display: none;
@@ -22,8 +28,13 @@ h1{text-align:center}
 
 #accessibility {
     text-align: center; /* Center-aligns the text */
-    margin: auto;       /* Centers the element horizontally */
+    margin: auto; /* Centers the element horizontally */
+}
+
+#random_seed {
+    height: 71px;
 }
 
-#random_seed {height: 71px;}
-#run_button {height: 87px;}
+#run_button {
+    height: 87px;
\ No newline at end of file
+}
\ No newline at end of file

app/flux.1/sketch/flux_pix2pix_pipeline.py

@@ -139,10 +139,8 @@ class FluxPix2pixTurboPipeline(FluxPipeline):
             erosion_kernel = torch.ones(1, 1, kernel_size, kernel_size, dtype=self.dtype, device=device)
             self.erosion_kernel = erosion_kernel
 
-        torchvision.utils.save_image(image_t[0], "before.png")
         image_t = nn.functional.conv2d(image_t[:, :1], erosion_kernel, padding=kernel_size // 2) > kernel_size**2 - 0.1
         image_t = torch.concat([image_t, image_t, image_t], dim=1).to(self.dtype)
-        torchvision.utils.save_image(image_t[0], "after.png")
 
         image_t = (image_t - 0.5) * 2
 

app/flux.1/sketch/run_gradio.py

 # Changed from https://github.com/GaParmar/img2img-turbo/blob/main/gradio_sketch2image.py
-import logging
 import os
 import random
 import tempfile
@@ -29,7 +28,7 @@ if args.precision == "bf16":
     pipeline = pipeline.to("cuda")
     pipeline.precision = "bf16"
     pipeline.load_control_module(
-        "mit-han-lab/svdquant-models", "flux.1-pix2pix-turbo-sketch2image.safetensors", alpha=DEFAULT_SKETCH_GUIDANCE
+        "mit-han-lab/svdq-flux.1-schnell-pix2pix-turbo", "sketch.safetensors", alpha=DEFAULT_SKETCH_GUIDANCE
     )
 else:
     assert args.precision == "int4"
@@ -48,9 +47,9 @@ else:
     pipeline = pipeline.to("cuda")
     pipeline.precision = "int4"
     pipeline.load_control_module(
-        "mit-han-lab/svdquant-models",
-        "flux.1-pix2pix-turbo-sketch2image.safetensors",
-        svdq_lora_path="mit-han-lab/svdquant-models/svdq-flux.1-pix2pix-turbo-sketch2image.safetensors",
+        "mit-han-lab/svdq-flux.1-schnell-pix2pix-turbo",
+        "sketch.safetensors",
+        svdq_lora_path="mit-han-lab/svdq-flux.1-schnell-pix2pix-turbo/svdq-int4-sketch.safetensors",
         alpha=DEFAULT_SKETCH_GUIDANCE,
     )
 safety_checker = SafetyChecker("cuda", disabled=args.no_safety_checker)
@@ -223,13 +222,7 @@ with gr.Blocks(css_paths="assets/style.css", title=f"SVDQuant Sketch-to-Image De
         queue=False,
     ).then(run, inputs=run_inputs, outputs=run_outputs, api_name=False)
 
-    style.change(
-        lambda x: STYLES[x],
-        inputs=[style],
-        outputs=[prompt_template],
-        api_name=False,
-        queue=False,
-    )
+    style.change(lambda x: STYLES[x], inputs=[style], outputs=[prompt_template], api_name=False, queue=False)
     gr.on(
         triggers=[prompt.submit, run_button.click, canvas.change],
         fn=run,

app/flux.1/t2i/assets/description.html

@@ -10,37 +10,37 @@
             SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
         </h2>
         <h3>
-            <a href='https://lmxyy.me'>Muyang Li*</a>,
-            <a href='https://yujunlin.com'>Yujun Lin*</a>,
-            <a href='https://hanlab.mit.edu/team/zhekai-zhang'>Zhekai Zhang*</a>,
-            <a href='https://www.tianle.website/#/'>Tianle Cai</a>,
-            <a href='https://xiuyuli.com'>Xiuyu Li</a>,
+            <a href='https://lmxyy.me' target="_blank">Muyang Li*</a>,
+            <a href='https://yujunlin.com' target="_blank">Yujun Lin*</a>,
+            <a href='https://hanlab.mit.edu/team/zhekai-zhang' target="_blank">Zhekai Zhang*</a>,
+            <a href='https://www.tianle.website/#/' target="_blank">Tianle Cai</a>,
+            <a href='https://xiuyuli.com' target="_blank">Xiuyu Li</a>,
             <br>
-            <a href='https://github.com/JerryGJX'>Junxian Guo</a>,
-            <a href='https://xieenze.github.io'>Enze Xie</a>,
-            <a href='https://cs.stanford.edu/~chenlin/'>Chenlin Meng</a>,
-            <a href='https://www.cs.cmu.edu/~junyanz/'>Jun-Yan Zhu</a>,
-            and <a href='https://hanlab.mit.edu/songhan'>Song Han</a>
+            <a href='https://github.com/JerryGJX' target="_blank">Junxian Guo</a>,
+            <a href='https://xieenze.github.io' target="_blank">Enze Xie</a>,
+            <a href='https://cs.stanford.edu/~chenlin/' target="_blank">Chenlin Meng</a>,
+            <a href='https://www.cs.cmu.edu/~junyanz/' target="_blank">Jun-Yan Zhu</a>,
+            and <a href='https://hanlab.mit.edu/songhan' target="_blank">Song Han</a>
         </h3>
         <div style="display: flex; justify-content: center; align-items: center; text-align: center;">
-            <a href="https://arxiv.org/abs/2411.05007">[Paper]</a>
+            <a href="https://arxiv.org/abs/2411.05007" target="_blank">[Paper]</a>
             &nbsp;
-            <a href='https://github.com/mit-han-lab/nunchaku'>
+            <a href='https://github.com/mit-han-lab/nunchaku' target="_blank">
                 [Code]
             </a>
             &nbsp;
-            <a href='https://hanlab.mit.edu/projects/svdquant'>
+            <a href='https://hanlab.mit.edu/projects/svdquant' target="_blank">
                 [Website]
             </a>
             &nbsp;
-            <a href='https://hanlab.mit.edu/blog/svdquant'>
+            <a href='https://hanlab.mit.edu/blog/svdquant' target="_blank">
                 [Blog]
             </a>
         </div>
         <h4>Quantization Library:
-            <a href='https://github.com/mit-han-lab/deepcompressor'>DeepCompressor</a>
+            <a href='https://github.com/mit-han-lab/deepcompressor' target="_blank">DeepCompressor</a>
             &nbsp;
-            Inference Engine: <a href='https://github.com/mit-han-lab/nunchaku'>Nunchaku</a>
+            Inference Engine: <a href='https://github.com/mit-han-lab/nunchaku' target="_blank">Nunchaku</a>
         </h4>
         <div style="display: flex; justify-content: center; align-items: center; text-align: center;">
             {device_info}

app/flux.1/t2i/assets/frame1.css

-.gradio-container{max-width: 560px !important}
+.gradio-container {
+    max-width: 640px !important;
+    margin: auto; /* Centers the element horizontally */
+}

app/flux.1/t2i/assets/frame2.css

-.gradio-container{max-width: 1200px !important}
+.gradio-container {
+    max-width: 1200px !important;
+    margin: auto; /* Centers the element horizontally */
+}

app/flux.1/t2i/run_gradio.py

@@ -276,8 +276,6 @@ with gr.Blocks(
             outputs=[prompt_template],
             api_name=False,
             queue=False,
-        ).then(
-            fn=generate_func, inputs=input_args, outputs=[*image_results, *latency_results], api_name=False, queue=False
         )
     gr.Markdown("MIT Accessibility: https://accessibility.mit.edu/", elem_id="accessibility")
 

app/flux.1/t2i/vars.py

@@ -36,7 +36,7 @@ LORA_PATHS = {
     },
 }
 
-SVDQ_LORA_PATH_FORMAT = "mit-han-lab/svdquant-models/svdq-flux.1-dev-lora-{name}.safetensors"
+SVDQ_LORA_PATH_FORMAT = "mit-han-lab/svdquant-lora-collection/svdq-int4-flux.1-dev-{name}.safetensors"
 SVDQ_LORA_PATHS = {
     "Anime": SVDQ_LORA_PATH_FORMAT.format(name="anime"),
     "GHIBSKY Illustration": SVDQ_LORA_PATH_FORMAT.format(name="ghibsky"),

app/sana/t2i/assets/description.html

@@ -4,43 +4,43 @@
             <img src="https://github.com/mit-han-lab/nunchaku/raw/refs/heads/main/assets/logo.svg"
                  alt="logo"
                  style="height: 40px; width: auto; display: block; margin: auto;"/>
-            <a href='https://nvlabs.github.io/Sana/'>SANA-1600M</a> Demo
+            <a href='https://nvlabs.github.io/Sana/' target="_blank">SANA-1.6B</a> Demo
         </h1>
         <h2>
             SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
         </h2>
         <h3>
-            <a href='https://lmxyy.me'>Muyang Li*</a>,
-            <a href='https://yujunlin.com'>Yujun Lin*</a>,
-            <a href='https://hanlab.mit.edu/team/zhekai-zhang'>Zhekai Zhang*</a>,
-            <a href='https://www.tianle.website/#/'>Tianle Cai</a>,
-            <a href='https://xiuyuli.com'>Xiuyu Li</a>,
+            <a href='https://lmxyy.me' target="_blank">Muyang Li*</a>,
+            <a href='https://yujunlin.com' target="_blank">Yujun Lin*</a>,
+            <a href='https://hanlab.mit.edu/team/zhekai-zhang' target="_blank">Zhekai Zhang*</a>,
+            <a href='https://www.tianle.website/#/' target="_blank">Tianle Cai</a>,
+            <a href='https://xiuyuli.com' target="_blank">Xiuyu Li</a>,
             <br>
-            <a href='https://github.com/JerryGJX'>Junxian Guo</a>,
-            <a href='https://xieenze.github.io'>Enze Xie</a>,
-            <a href='https://cs.stanford.edu/~chenlin/'>Chenlin Meng</a>,
-            <a href='https://www.cs.cmu.edu/~junyanz/'>Jun-Yan Zhu</a>,
-            and <a href='https://hanlab.mit.edu/songhan'>Song Han</a>
+            <a href='https://github.com/JerryGJX' target="_blank">Junxian Guo</a>,
+            <a href='https://xieenze.github.io' target="_blank">Enze Xie</a>,
+            <a href='https://cs.stanford.edu/~chenlin/' target="_blank">Chenlin Meng</a>,
+            <a href='https://www.cs.cmu.edu/~junyanz/' target="_blank">Jun-Yan Zhu</a>,
+            and <a href='https://hanlab.mit.edu/songhan' target="_blank">Song Han</a>
         </h3>
         <div style="display: flex; justify-content: center; align-items: center; text-align: center;">
-            <a href="https://arxiv.org/abs/2411.05007">[Paper]</a>
+            <a href="https://arxiv.org/abs/2411.05007" target="_blank">[Paper]</a>
             &nbsp;
-            <a href='https://github.com/mit-han-lab/nunchaku'>
+            <a href='https://github.com/mit-han-lab/nunchaku' target="_blank">
                 [Code]
             </a>
             &nbsp;
-            <a href='https://hanlab.mit.edu/projects/svdquant'>
+            <a href='https://hanlab.mit.edu/projects/svdquant' target="_blank">
                 [Website]
             </a>
             &nbsp;
-            <a href='https://hanlab.mit.edu/blog/svdquant'>
+            <a href='https://hanlab.mit.edu/blog/svdquant' target="_blank">
                 [Blog]
             </a>
         </div>
         <h4>Quantization Library:
-            <a href='https://github.com/mit-han-lab/deepcompressor'>DeepCompressor</a>
+            <a href='https://github.com/mit-han-lab/deepcompressor' target="_blank">DeepCompressor</a>
             &nbsp;
-            Inference Engine: <a href='https://github.com/mit-han-lab/nunchaku'>Nunchaku</a>
+            Inference Engine: <a href='https://github.com/mit-han-lab/nunchaku' target="_blank">Nunchaku</a>
         </h4>
         <div style="display: flex; justify-content: center; align-items: center; text-align: center;">
             {device_info}

app/sana/t2i/assets/frame1.css

-.gradio-container{max-width: 560px !important}
+.gradio-container {
+    max-width: 640px !important;
+    margin: auto; /* Centers the element horizontally */
+}