[`bnb`] Fix bnb slow tests (#28788)

fix bnb slow tests

[`bnb`] Fix bnb slow tests (#28788)
fix bnb slow tests
d703eaae · Younes Belkada · GitHub · 74c9cfea · d703eaae
Unverified Commit d703eaae authored Jan 31, 2024 by Younes Belkada Committed by GitHub Jan 31, 2024
Show whitespace changes
Inline Side-by-side

Showing with 2 additions and 0 deletions

src/transformers/utils/quantization_config.py src/transformers/utils/quantization_config.py +2 -0

No files found.
--- a/src/transformers/utils/quantization_config.py
+++ b/src/transformers/utils/quantization_config.py
@@ -342,6 +342,8 @@ class BitsAndBytesConfig(QuantizationConfigMixin):
        """
        output = copy.deepcopy(self.__dict__)
        output["bnb_4bit_compute_dtype"] = str(output["bnb_4bit_compute_dtype"]).split(".")[1]
+        output["load_in_4bit"] = self.load_in_4bit
+        output["load_in_8bit"] = self.load_in_8bit

        return output