"vscode:/vscode.git/clone" did not exist on "e6f7e946666db8311bcf5131e81319b78f0419e8"
Commit ea3cd106 authored by dengjb's avatar dengjb
Browse files

update

parent 8faca65b
......@@ -52,6 +52,7 @@ docker run -it \
需要单独安装:
```
pip uninstall vllm
pip install vllm-0.15.1+das.opt1.alpha.dtk2604-cp310-cp310-linux_x86_64.whl
pip install -r requirements.txt
```
......@@ -63,9 +64,9 @@ pip install -r requirements.txt
## 推理
### vllm
#### 单机单卡推理
#### 单机推理
vllm推理脚本
推理脚本参考
```
export HIP_VISIBLE_DEVICES=0
vllm serve ZhipuAI/GLM-OCR/ \
......@@ -109,7 +110,7 @@ DCU与GPU精度一致,推理框架:vllm。
## 预训练权重
| 模型名称 | 权重大小 | DCU型号 | 最低卡数需求 | 下载地址 |
|:------:|:----:|:----------:|:------:|:---------------------:|
| GLM-OCR | 1.33B | BW1000 | 1 | [Hugging Face](https://huggingface.co/moonshotai/Kimi-K2.5) |
| GLM-OCR | 1.33B | BW1000 | 1 | [Model Scope](https://www.modelscope.cn/models/ZhipuAI/GLM-OCR/summary) |
## 源码仓库及问题反馈
- https://developer.sourcefind.cn/codes/modelzoo/glm-ocr_vllm
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment