Commit c27b0339 authored by raojy's avatar raojy
Browse files

tow

parent c93161f8
...@@ -63,13 +63,6 @@ pip install -e . ...@@ -63,13 +63,6 @@ pip install -e .
```bash ```bash
python run.py python run.py
``` ```
### vllm
#### 单机推理
**注意**:使用`K100 AI` 启动服务时需要添加`--disable-custom-all-reduce`参数,加载W8A8模型启动服务时需要添加`-cc.mode=3``-cc.inductor_compile_config='{"combo_kernels": false, "benchmark_combo_kernel": false}'`
```bash
## 效果展示 ## 效果展示
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment