Commit 8e8e10b2 authored by raojy's avatar raojy 💬
Browse files

Update README.md

parent bfb39f34
......@@ -72,13 +72,14 @@ docker run -it \
> 如果出现`ImportError: librocm_smi64.so.2: cannot open shaned object file: No such file or directory`报错,系机器hyhal版本较低所致,请进行升级。
### vLLM
#### 单机推理
##### BF16
**mtp**功能需添加以下参数:
```
--speculative-config.method mtp \
--speculative-config.num_speculative_tokens 2
```
#### 单机推理
```bash
# serve启动
vllm serve Qwen/Qwen3.5-35B-A3B \
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment