Skip to content

GitLab

"git@developer.sourcefind.cn:renzhc/diffusers_dcu.git" did not exist on "fb29132b98abdd218bacb6dbaab372f5bb177a2e"
Commit aa6497cd authored by Casper Hansen's avatar Casper Hansen
Browse files

Add low_cpu_mem_usage=True in example

parent 1c5ccc79
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
examples/basic_quant.py
View file @ aa6497cd
...@@ -7,7 +7,7 @@ quant_config = { "zero_point": True, "q_group_size": 128, "w_bit": 4, "version": ...@@ -7,7 +7,7 @@ quant_config = { "zero_point": True, "q_group_size": 128, "w_bit": 4, "version":
# Load model # Load model
# NOTE: pass safetensors=True to load safetensors # NOTE: pass safetensors=True to load safetensors
model = AutoAWQForCausalLM.from_pretrained(model_path) model = AutoAWQForCausalLM.from_pretrained(model_path, **{"low_cpu_mem_usage": True})
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True) tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
# Quantize # Quantize
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment