Commit c27b0339 authored by raojy's avatar raojy
Browse files

tow

parent c93161f8
......@@ -63,13 +63,6 @@ pip install -e .
```bash
python run.py
```
### vllm
#### 单机推理
**注意**:使用`K100 AI` 启动服务时需要添加`--disable-custom-all-reduce`参数,加载W8A8模型启动服务时需要添加`-cc.mode=3``-cc.inductor_compile_config='{"combo_kernels": false, "benchmark_combo_kernel": false}'`
```bash
## 效果展示
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment