Skip to content

GitLab

Commit affd1906 authored by Casper Hansen's avatar Casper Hansen
Browse files

Correct example

parent 219ccb33
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
examples/basic_generate_safetensors.py
View file @ affd1906
...@@ -4,7 +4,7 @@ from transformers import AutoTokenizer, TextStreamer ...@@ -4,7 +4,7 @@ from transformers import AutoTokenizer, TextStreamer
quant_path = "casperhansen/opt-125m-awq" quant_path = "casperhansen/opt-125m-awq"
# Load model # Load model
model = AutoAWQForCausalLM.from_quantized(quant_path, fuse_layers=True, use_safetensors=True) model = AutoAWQForCausalLM.from_quantized(quant_path, fuse_layers=True, safetensors=True)
tokenizer = AutoTokenizer.from_pretrained(quant_path, trust_remote_code=True) tokenizer = AutoTokenizer.from_pretrained(quant_path, trust_remote_code=True)
streamer = TextStreamer(tokenizer, skip_special_tokens=True) streamer = TextStreamer(tokenizer, skip_special_tokens=True)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment