Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
Lmdeploy
Commits
21c2a9d4
Commit
21c2a9d4
authored
Nov 24, 2023
by
xiabo
Browse files
Update README.md
parent
b1ebc450
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
23 additions
and
6 deletions
+23
-6
README.md
README.md
+23
-6
No files found.
README.md
View file @
21c2a9d4
...
@@ -44,6 +44,15 @@ pip3 install transformers==4.33.2
...
@@ -44,6 +44,15 @@ pip3 install transformers==4.33.2
pip3
install
urllib3
==
1.24
pip3
install
urllib3
==
1.24
pip3
install
wheel
pip3
install
wheel
yum
install
rapidjson
yum
install
rapidjson
# 执行dtk环境变量
source
{
DTK_PATH
}
/env.sh
source
{
DTK_PATH
}
/cuda/env.sh
# 升级gcc版本到9
yum
install
-y
centos-release-scl
yum
install
-y
devtoolset-9
scl
enable
devtoolset-9 bash
```
```
3.
基于现有python环境:安装pytorch,pytorch whl包下载目录:
[
https://cancon.hpccube.com:65024/4/main/pytorch/dtk23.04
](
https://cancon.hpccube.com:65024/4/main/pytorch/dtk23.04
)
,根据python、dtk版本,下载对应pytorch的whl包。安装命令如下:
3.
基于现有python环境:安装pytorch,pytorch whl包下载目录:
[
https://cancon.hpccube.com:65024/4/main/pytorch/dtk23.04
](
https://cancon.hpccube.com:65024/4/main/pytorch/dtk23.04
)
,根据python、dtk版本,下载对应pytorch的whl包。安装命令如下:
...
@@ -54,6 +63,14 @@ pip3 install transformers==4.33.2
...
@@ -54,6 +63,14 @@ pip3 install transformers==4.33.2
pip3
install
urllib3
==
1.24
pip3
install
urllib3
==
1.24
pip3
install
wheel
pip3
install
wheel
yum
install
rapidjson
yum
install
rapidjson
# 执行dtk环境变量
source
{
DTK_PATH
}
/env.sh
source
{
DTK_PATH
}
/cuda/env.sh
# 升级gcc版本到9
yum
install
-y
centos-release-scl
yum
install
-y
devtoolset-9
scl
enable
devtoolset-9 bash
```
```
注:需要GCC版本>=9.0
注:需要GCC版本>=9.0
...
@@ -91,7 +108,7 @@ cd dist && pip3 install lmdeploy*
...
@@ -91,7 +108,7 @@ cd dist && pip3 install lmdeploy*
# <model_format> 保存输出的目标路径(默认./workspace)
# <model_format> 保存输出的目标路径(默认./workspace)
# <tp> 用于张量并行的GPU数量应该是2^n
# <tp> 用于张量并行的GPU数量应该是2^n
mdeploy convert --model_name llama --model_path /path/to/model --model_format
llama
--tokenizer_path None --dst_path ./workspace_llama --tp 1
ll
mdeploy convert --model_name llama --model_path /path/to/model --model_format
hf
--tokenizer_path None --dst_path ./workspace_llama --tp 1
2、运行
2、运行
# bash界面运行
# bash界面运行
...
@@ -116,7 +133,7 @@ lmdeploy serve gradio --model_path_or_server ./workspace_llama --server_name {ip
...
@@ -116,7 +133,7 @@ lmdeploy serve gradio --model_path_or_server ./workspace_llama --server_name {ip
以7B为例:
以7B为例:
```
```
1、模型转换
1、模型转换
mdeploy convert --model_name llama2 --model_path /path/to/model --model_format hf --tokenizer_path None --dst_path ./workspace_llama2 --tp 1 #
l
mdeploy convert --model_name llama2 --model_path /path/to/model --model_format hf --tokenizer_path None --dst_path ./workspace_llama2 --tp 1 #
2、运行
2、运行
# bash界面运行
# bash界面运行
lmdeploy chat turbomind --model_path ./workspace_llama2 --tp 1
lmdeploy chat turbomind --model_path ./workspace_llama2 --tp 1
...
@@ -131,7 +148,7 @@ lmdeploy serve gradio --model_path_or_server ./workspace_llama2 --server_name {i
...
@@ -131,7 +148,7 @@ lmdeploy serve gradio --model_path_or_server ./workspace_llama2 --server_name {i
以7B为例:
以7B为例:
```
```
1、模型转换
1、模型转换
mdeploy convert --model_name model_name --model_path /path/to/model --model_format hf --tokenizer_path None --dst_path ./workspace_intern --tp 1 # 根据模型的类型选择model_name是internlm-chat还是internlm
l
mdeploy convert --model_name model_name --model_path /path/to/model --model_format hf --tokenizer_path None --dst_path ./workspace_intern --tp 1 # 根据模型的类型选择model_name是internlm-chat还是internlm
2、运行
2、运行
# bash界面运行
# bash界面运行
lmdeploy chat turbomind --model_path ./workspace_intern --tp 1
lmdeploy chat turbomind --model_path ./workspace_intern --tp 1
...
@@ -146,7 +163,7 @@ lmdeploy serve gradio --model_path_or_server ./workspace_intern --server_name {i
...
@@ -146,7 +163,7 @@ lmdeploy serve gradio --model_path_or_server ./workspace_intern --server_name {i
以7B为例:
以7B为例:
```
```
1、模型转换
1、模型转换
mdeploy convert --model_name baichuan-7b --model_path /path/to/model --model_format hf --tokenizer_path None --dst_path ./workspace_baichuan --tp 1
l
mdeploy convert --model_name baichuan-7b --model_path /path/to/model --model_format hf --tokenizer_path None --dst_path ./workspace_baichuan --tp 1
2、运行
2、运行
# bash界面运行
# bash界面运行
lmdeploy chat turbomind --model_path ./workspace_baichuan --tp 1
lmdeploy chat turbomind --model_path ./workspace_baichuan --tp 1
...
@@ -162,7 +179,7 @@ lmdeploy serve gradio --model_path_or_server ./workspace_baichuan --server_name
...
@@ -162,7 +179,7 @@ lmdeploy serve gradio --model_path_or_server ./workspace_baichuan --server_name
以7B为例:
以7B为例:
```
```
1、模型转换
1、模型转换
mdeploy convert --model_name baichuan2-7b --model_path /path/to/model --model_format hf --tokenizer_path None --dst_path ./workspace_baichuan2 --tp 1
l
mdeploy convert --model_name baichuan2-7b --model_path /path/to/model --model_format hf --tokenizer_path None --dst_path ./workspace_baichuan2 --tp 1
2、运行
2、运行
# bash界面运行
# bash界面运行
lmdeploy chat turbomind --model_path ./workspace_baichuan2 --tp 1
lmdeploy chat turbomind --model_path ./workspace_baichuan2 --tp 1
...
@@ -178,7 +195,7 @@ lmdeploy serve gradio --model_path_or_server ./workspace_baichuan2 --server_name
...
@@ -178,7 +195,7 @@ lmdeploy serve gradio --model_path_or_server ./workspace_baichuan2 --server_name
以7B为例:
以7B为例:
```
```
1、模型转换
1、模型转换
mdeploy convert --model_name qwen-7b --model_path /path/to/model --model_format qwen --tokenizer_path None --dst_path ./workspace_qwen --tp 1
l
mdeploy convert --model_name qwen-7b --model_path /path/to/model --model_format qwen --tokenizer_path None --dst_path ./workspace_qwen --tp 1
2、运行
2、运行
# bash界面运行
# bash界面运行
lmdeploy chat turbomind --model_path ./workspace_qwen --tp 1
lmdeploy chat turbomind --model_path ./workspace_qwen --tp 1
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment