"...git@developer.sourcefind.cn:chenpangpang/open-webui.git" did not exist on "3de4c6189b9b829a19304c71d5ec3e01e40fe463"
Commit ce813068 authored by dcuai's avatar dcuai
Browse files

Update README.md

parent e7b070b5
......@@ -148,8 +148,6 @@ HIP_VISIBLE_DEVICES=0 swift export --model_type telechat-7b \
--quant_bits 8 --quant_method gptq --model_id_or_path /path/to/telechat-7b \
--quant_output_dir ./quant_out
```
如果GPTQ量化遇到:RuntimeError: The weights trying to be saved containhed shared tensors [{'lm_head.weight', 'transformer.word_embeddings.weight'}] that are mismatching the transformers base configuration. Try saving using 'safe serialization=False` or remove this ternsor sharing,
可尝试找到文件/usr/local/lib/python3.10/site-packages/optimum/gptq/quantizer.py,并修改676行为safe_serialization=False
### 量化后推理
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment