Unverified Commit 372baec2 authored by Vallepu Vamsi Krishna's avatar Vallepu Vamsi Krishna Committed by GitHub
Browse files

Added description of quantization_config (#31133)

* Description of quantization_config

Added missing description about quantization_config in replace_with_bnb_linear for better readability.

* Removed trailing spaces
parent cdc81311
......@@ -243,6 +243,10 @@ def replace_with_bnb_linear(model, modules_to_not_convert=None, current_key_name
An array to track the current key of the recursion. This is used to check whether the current key (part of
it) is not in the list of modules to not convert (for instances modules that are offloaded to `cpu` or
`disk`).
quantization_config ('transformers.utils.quantization_config.BitsAndBytesConfig'):
To configure and manage settings related to quantization, a technique used to compress neural network models
by reducing the precision of the weights and activations, thus making models more efficient in terms of both
storage and computation.
"""
modules_to_not_convert = ["lm_head"] if modules_to_not_convert is None else modules_to_not_convert
model, has_been_replaced = _replace_with_bnb_linear(
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment