Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
Qwen_lmdeploy
Commits
6afe24b1
Commit
6afe24b1
authored
Dec 22, 2023
by
zhouxiang
Browse files
新增qwen72b支持
parent
f3f9a9a3
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
25 additions
and
5 deletions
+25
-5
.gitmodules
.gitmodules
+1
-1
README.md
README.md
+23
-3
lmdeploy
lmdeploy
+1
-1
No files found.
.gitmodules
View file @
6afe24b1
[submodule "lmdeploy"]
[submodule "lmdeploy"]
path = lmdeploy
path = lmdeploy
url = http://developer.hpccube.com/codes/aicomponent/lmdeploy.git
url = http://developer.hpccube.com/codes/aicomponent/lmdeploy.git
tag = dtk23.04-v0.0.13
branch = dtk23.10-v0.0.13-qwen
\ No newline at end of file
\ No newline at end of file
README.md
View file @
6afe24b1
...
@@ -55,6 +55,8 @@ cd .. && python3 setup.py install
...
@@ -55,6 +55,8 @@ cd .. && python3 setup.py install
[
Qwen-14B-chat
](
https://huggingface.co/Qwen/Qwen-14B-Chat/tree/main
)
[
Qwen-14B-chat
](
https://huggingface.co/Qwen/Qwen-14B-Chat/tree/main
)
[
Qwen-72B-Chat
](
https://huggingface.co/Qwen/Qwen-72B-Chat
)
### 运行 Qwen-7B-chat
### 运行 Qwen-7B-chat
```
```
# 模型转换
# 模型转换
...
@@ -89,19 +91,37 @@ lmdeploy serve gradio --model_path_or_server ./workspace_qwe7b --server_name {ip
...
@@ -89,19 +91,37 @@ lmdeploy serve gradio --model_path_or_server ./workspace_qwe7b --server_name {ip
### 运行 Qwen-14B-chat
### 运行 Qwen-14B-chat
```
```
# 模型转换
# 模型转换
mdeploy convert --model_name qwen-
7
b --model_path /path/to/model --model_format qwen --tokenizer_path None --dst_path ./workspace_qwe
7
b --tp 2 --quant_path None --group_size 0
mdeploy convert --model_name qwen-
14
b --model_path /path/to/model --model_format qwen --tokenizer_path None --dst_path ./workspace_qwe
n14
b --tp 2 --quant_path None --group_size 0
# bash界面运行
# bash界面运行
lmdeploy chat turbomind --model_path ./workspace_qwe
7
b --tp 2
lmdeploy chat turbomind --model_path ./workspace_qwe
n14
b --tp 2
# 服务器网页端运行
# 服务器网页端运行
在bash端运行:
在bash端运行:
lmdeploy serve gradio --model_path_or_server ./workspace_qwe7b --server_name {ip} --server_port {pord} --batch_size 32 --tp 1 --restful_api False
lmdeploy serve gradio --model_path_or_server ./workspace_qwen14b --server_name {ip} --server_port {pord} --batch_size 32 --tp 2 --restful_api False
在网页上输入{ip}:{pord}即可进行对话
```
### 运行 Qwen-72B-chat
```
# 模型转换
mdeploy convert --model_name qwen-72b --model_path /path/to/model --model_format qwen --tokenizer_path None --dst_path ./workspace_qwen72b --tp 8 --quant_path None --group_size 0
# bash界面运行
lmdeploy chat turbomind --model_path ./workspace_qwen72b --tp 8
# 服务器网页端运行
在bash端运行:
lmdeploy serve gradio --model_path_or_server ./workspace_qwen72b --server_name {ip} --server_port {pord} --batch_size 32 --tp 8 --restful_api False
在网页上输入{ip}:{pord}即可进行对话
在网页上输入{ip}:{pord}即可进行对话
```
```
## result
## result


### 精度
### 精度
...
...
lmdeploy
@
0189f17c
Subproject commit
e432dbb0e56caaf319b9c9d7b79eb8106852dc91
Subproject commit
0189f17c859b879781235bd57163aae1b00f1e72
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment