Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
Qwen_lmdeploy
Commits
31d86193
Commit
31d86193
authored
Mar 05, 2024
by
zhouxiang
Browse files
Merge branch 'master' of
http://developer.hpccube.com/codes/modelzoo/qwen_lmdeploy
parents
a1308778
1ad40dd2
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
8 additions
and
5 deletions
+8
-5
LICENSE
LICENSE
+3
-0
README.md
README.md
+5
-5
No files found.
LICENSE
0 → 100644
View file @
31d86193
None LICENSE Currently
README.md
View file @
31d86193
...
@@ -56,11 +56,11 @@ cd .. && python3 setup.py install
...
@@ -56,11 +56,11 @@ cd .. && python3 setup.py install
### 模型下载
### 模型下载
[
Qwen-7B-chat
](
https://
huggingface.co
/Qwen/Qwen-7B-Chat
/tree/main
)
[
Qwen-7B-chat
](
https://
modelscope.cn/models
/Qwen/Qwen-7B-Chat
)
[
Qwen-14B-chat
](
https://
huggingface.co
/Qwen/Qwen-14B-Chat
/tree/main
)
[
Qwen-14B-chat
](
https://
modelscope.cn/models
/Qwen/Qwen-14B-Chat
)
[
Qwen-72B-Chat
](
https://
huggingface.co/Q
wen/Qwen-72B-Chat
)
[
Qwen-72B-Chat
](
https://
modelscope.cn/models/q
wen/Qwen-72B-Chat
)
### 运行 Qwen-7B-chat
### 运行 Qwen-7B-chat
```
```
...
@@ -72,7 +72,7 @@ cd .. && python3 setup.py install
...
@@ -72,7 +72,7 @@ cd .. && python3 setup.py install
# <model_format> 保存输出的目标路径(默认./workspace)
# <model_format> 保存输出的目标路径(默认./workspace)
# <tp> 用于张量并行的GPU数量应该是2^n
# <tp> 用于张量并行的GPU数量应该是2^n
mdeploy convert --model_name qwen-7b --model_path /path/to/model --dst_path ./workspace_qwe7b --tp 1
l
mdeploy convert --model_name qwen-7b --model_path /path/to/model --dst_path ./workspace_qwe7b --tp 1
# bash界面运行
# bash界面运行
lmdeploy chat turbomind --model_path ./workspace_qwe7b --tp 1 # 输入问题后执行2次回车进行推理
lmdeploy chat turbomind --model_path ./workspace_qwe7b --tp 1 # 输入问题后执行2次回车进行推理
...
@@ -94,7 +94,7 @@ lmdeploy serve gradio --model_path_or_server ./workspace_qwe7b --server_name {ip
...
@@ -94,7 +94,7 @@ lmdeploy serve gradio --model_path_or_server ./workspace_qwe7b --server_name {ip
### 运行 Qwen-14B-chat
### 运行 Qwen-14B-chat
```
```
# 模型转换
# 模型转换
mdeploy convert --model_name qwen-14b --model_path /path/to/model --dst_path ./workspace_qwen14b --tp 2
l
mdeploy convert --model_name qwen-14b --model_path /path/to/model --dst_path ./workspace_qwen14b --tp 2
# bash界面运行
# bash界面运行
lmdeploy chat turbomind --model_path ./workspace_qwen14b --tp 2
lmdeploy chat turbomind --model_path ./workspace_qwen14b --tp 2
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment