"src/git@developer.sourcefind.cn:chenpangpang/open-webui.git" did not exist on "2bece7b4c5b670d3d4a0d985c54abdcca4868815"
Commit e7b070b5 authored by dcuai's avatar dcuai
Browse files

Update README.md

parent 931561f0
...@@ -148,6 +148,9 @@ HIP_VISIBLE_DEVICES=0 swift export --model_type telechat-7b \ ...@@ -148,6 +148,9 @@ HIP_VISIBLE_DEVICES=0 swift export --model_type telechat-7b \
--quant_bits 8 --quant_method gptq --model_id_or_path /path/to/telechat-7b \ --quant_bits 8 --quant_method gptq --model_id_or_path /path/to/telechat-7b \
--quant_output_dir ./quant_out --quant_output_dir ./quant_out
``` ```
如果GPTQ量化遇到:RuntimeError: The weights trying to be saved containhed shared tensors [{'lm_head.weight', 'transformer.word_embeddings.weight'}] that are mismatching the transformers base configuration. Try saving using 'safe serialization=False` or remove this ternsor sharing,
可尝试找到文件/usr/local/lib/python3.10/site-packages/optimum/gptq/quantizer.py,并修改676行为safe_serialization=False
### 量化后推理 ### 量化后推理
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment