Skip to content

GitLab

Commit 5c6c5987 authored by raojy's avatar raojy
Browse files

updata

parent 986dd49f
Hide whitespace changes
Inline Side-by-side
Showing with 9 additions and 15 deletions
README.md
View file @ 5c6c5987
...@@ -16,8 +16,7 @@ ...@@ -16,8 +16,7 @@
<div align=center> <div align=center>
<img src="./images/arch.png"/> <img src="./images/arch.png"/>
</div> </div>
## 环境依赖
### 环境依赖
| 软件 | 版本 | | 软件 | 版本 |
| :----------: | :--------------------------------------------: | | :----------: | :--------------------------------------------: |
...@@ -212,22 +211,19 @@ torchrun ./LLaMA-Factory/src/train.py \ ...@@ -212,22 +211,19 @@ torchrun ./LLaMA-Factory/src/train.py \
### transformers ### transformers
### 单机单卡 ### 单机推理
``` ```
#单卡推理
python inference.py python inference.py
``` #多卡推理
### 单机多卡
```
CUDA_VISIBLE_DEVICES=0,1,2,3 python inference.py CUDA_VISIBLE_DEVICES=0,1,2,3 python inference.py
``` ```
### vllm ## vllm
#### 单卡推理 #### 单机推理
``` ```
vllm推理需要确认qwen_vl_utils库，若不存在，请执行 vllm推理需要确认qwen_vl_utils库，若不存在，请执行
...@@ -297,16 +293,14 @@ curl http://localhost:8000/v1/chat/completions \ ...@@ -297,16 +293,14 @@ curl http://localhost:8000/v1/chat/completions \
}' }'
``` ```
### 效果展示 ## 效果展示
<div align=center> <div align=center>
<img src="./images/result1.png"/> <img src="./images/result1.png"/>
</div> </div>
## 精度
`DCU与GPU精度一致，推理框架：transformers、vllm。`
### 精度
DCU与GPU精度一致，推理框架：transformers、vllm。
## 预训练权重 ## 预训练权重
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment