Unverified Commit edea08a6 authored by Younes Belkada's avatar Younes Belkada Committed by GitHub
Browse files

[`bnb`] Fix bnb error message (#22026)

* fix error message

* make style
parent dfe9a319
...@@ -2578,8 +2578,11 @@ class PreTrainedModel(nn.Module, ModuleUtilsMixin, GenerationMixin, PushToHubMix ...@@ -2578,8 +2578,11 @@ class PreTrainedModel(nn.Module, ModuleUtilsMixin, GenerationMixin, PushToHubMix
raise ValueError( raise ValueError(
""" """
Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit
the quantized model. If you have set a value for `max_memory` you should increase that. To have the quantized model. If you want to dispatch the model on the CPU or the disk while keeping
an idea of the modules that are set on the CPU or RAM you can print model.hf_device_map. these modules in 32-bit, you need to set `load_in_8bit_fp32_cpu_offload=True` and pass a custom
`device_map` to `from_pretrained`. Check
https://huggingface.co/docs/transformers/main/en/main_classes/quantization#offload-between-cpu-and-gpu
for more details.
""" """
) )
del device_map_without_lm_head del device_map_without_lm_head
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment