"git@developer.sourcefind.cn:wangsen/mineru.git" did not exist on "3f87f83fdad308e927c043a9053ebfe47e3a4394"
Commit e7b070b5 authored by dcuai's avatar dcuai
Browse files

Update README.md

parent 931561f0
......@@ -148,6 +148,9 @@ HIP_VISIBLE_DEVICES=0 swift export --model_type telechat-7b \
--quant_bits 8 --quant_method gptq --model_id_or_path /path/to/telechat-7b \
--quant_output_dir ./quant_out
```
如果GPTQ量化遇到:RuntimeError: The weights trying to be saved containhed shared tensors [{'lm_head.weight', 'transformer.word_embeddings.weight'}] that are mismatching the transformers base configuration. Try saving using 'safe serialization=False` or remove this ternsor sharing,
可尝试找到文件/usr/local/lib/python3.10/site-packages/optimum/gptq/quantizer.py,并修改676行为safe_serialization=False
### 量化后推理
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment