Commit 360fd436 authored by weishb's avatar weishb
Browse files

update README.md

parent 1c72076a
......@@ -62,7 +62,7 @@ pip install numpy==1.25.0
### vllm
#### 单机推理
**注意**:使用`K100 AI` 启动服务时需要添加`--disable-custom-all-reduce`参数,加载8W8A模型启动服务时需要添加`-cc.mode=3``-cc.inductor_compile_config='{"combo_kernels": false, "benchmark_combo_kernel": false}'`
**注意**:使用`K100 AI` 启动服务时需要添加`--disable-custom-all-reduce`参数,加载W8A8模型启动服务时需要添加`-cc.mode=3``-cc.inductor_compile_config='{"combo_kernels": false, "benchmark_combo_kernel": false}'`
```bash
## serve启动
......@@ -176,7 +176,7 @@ DCU与GPU精度一致,推理框架:vllm。
| 模型名称 | 权重大小 | DCU型号 | 最低卡数需求 | 下载地址 |
|:------:|:----:|:----------:|:------:|:---------------------:|
| Qwen3.5-397B-A17B | 397B | K100AI,BW1000 | 16 | [Hugging Face](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) |
| Qwen3.5-397B-A17B-W8A8 | 397B | K100AI,BW1000 | 8 | [Modelscope](https://www.modelscope.cn/models/metax-tech/Qwen3.5-397B-A17B-W8A8) |
| Qwen3.5-397B-A17B-INT8 | 397B | K100AI,BW1000 | 8 | [Modelscope](https://www.modelscope.cn/models/metax-tech/Qwen3.5-397B-A17B-W8A8) |
| Qwen3.5-122B-A10B | 122B | K100AI,BW1000 | 8 | [Hugging Face](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) |
| Qwen3.5-35B-A3B | 35B | K100AI,BW1000 | 2 | [Hugging Face](https://huggingface.co/Qwen/Qwen3.5-35B-A3B) |
| Qwen3.5-27B | 27B | K100AI,BW1000 | 2 | [Hugging Face](https://huggingface.co/Qwen/Qwen3.5-27B) |
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment