Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
opencompass
Commits
b7d2607e
Commit
b7d2607e
authored
Jun 04, 2024
by
huangwb
Browse files
format readme
parent
e6288ec7
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
2 deletions
+3
-2
README.md
README.md
+3
-2
No files found.
README.md
View file @
b7d2607e
...
@@ -105,7 +105,7 @@ python run.py configs/lmdeploy/eval_llama2_lmdeploy.py
...
@@ -105,7 +105,7 @@ python run.py configs/lmdeploy/eval_llama2_lmdeploy.py
run所有的opencompass测试都需要先开启TGI的服务,TGI服务的环境及使用参考:
[
https://developer.hpccube.com/codes/OpenDAS/text-generation-inference
](
https://developer.hpccube.com/codes/OpenDAS/text-generation-inference
)
run所有的opencompass测试都需要先开启TGI的服务,TGI服务的环境及使用参考:
[
https://developer.hpccube.com/codes/OpenDAS/text-generation-inference
](
https://developer.hpccube.com/codes/OpenDAS/text-generation-inference
)
**评测base模型**
**评测base模型**
启动服务example:
启动服务example:
```
shell
```
shell
HIP_VISIBLE_DEVICES
=
3 text-generation-launcher
--dtype
=
float16
--model-id
/data/models/Llama-2-7b-chat-hf
--port
3001
HIP_VISIBLE_DEVICES
=
3 text-generation-launcher
--dtype
=
float16
--model-id
/data/models/Llama-2-7b-chat-hf
--port
3001
...
@@ -114,7 +114,7 @@ HIP_VISIBLE_DEVICES=3 text-generation-launcher --dtype=float16 --model-id /data/
...
@@ -114,7 +114,7 @@ HIP_VISIBLE_DEVICES=3 text-generation-launcher --dtype=float16 --model-id /data/
```
shell
```
shell
python run.py configs/tgi/eval_llama2_tgi.py
--debug
python run.py configs/tgi/eval_llama2_tgi.py
--debug
```
```
**评测chat模型**
**评测chat模型**
评测chat模型需要在模型路径里的
`tokenizer_config.json`
配置文件中提供
`chat_template`
,不同模型的
`chat_template`
可以参考
[
https://github.com/chujiezheng/chat_templates/tree/main/chat_templates
](
https://github.com/chujiezheng/chat_templates/tree/main/chat_templates
)
。
评测chat模型需要在模型路径里的
`tokenizer_config.json`
配置文件中提供
`chat_template`
,不同模型的
`chat_template`
可以参考
[
https://github.com/chujiezheng/chat_templates/tree/main/chat_templates
](
https://github.com/chujiezheng/chat_templates/tree/main/chat_templates
)
。
具体操作的时候可以从模型路径下copy一份
`tokenizer_config.json`
到其他目录下,比如copy后的文件为
`tokenizer_config_llama2_7b_chat.json`
,在该文件里添加
`chat_template`
,然后在起服务的时候指定
`--tokenizer-config-path`
参数为修改后的文件。以llama为列,修改后的config例子见
[
tokenizer_config_llama2_7b_chat.json
](
./configs/tgi/tokenizer_config_llama2_7b_chat.json
)
具体操作的时候可以从模型路径下copy一份
`tokenizer_config.json`
到其他目录下,比如copy后的文件为
`tokenizer_config_llama2_7b_chat.json`
,在该文件里添加
`chat_template`
,然后在起服务的时候指定
`--tokenizer-config-path`
参数为修改后的文件。以llama为列,修改后的config例子见
[
tokenizer_config_llama2_7b_chat.json
](
./configs/tgi/tokenizer_config_llama2_7b_chat.json
)
...
@@ -123,6 +123,7 @@ python run.py configs/tgi/eval_llama2_tgi.py --debug
...
@@ -123,6 +123,7 @@ python run.py configs/tgi/eval_llama2_tgi.py --debug
HIP_VISIBLE_DEVICES
=
3 text-generation-launcher
--dtype
=
float16
--model-id
/data/models/Llama-2-7b-chat-hf
--port
3001
--tokenizer-config-path
/path/to/tokenizer_config_llama2_7b_chat.json
HIP_VISIBLE_DEVICES
=
3 text-generation-launcher
--dtype
=
float16
--model-id
/data/models/Llama-2-7b-chat-hf
--port
3001
--tokenizer-config-path
/path/to/tokenizer_config_llama2_7b_chat.json
```
```
注意:和base模型比多了
`--tokenizer-config-path`
参数。
注意:和base模型比多了
`--tokenizer-config-path`
参数。
运行评测example:
运行评测example:
```
shell
```
shell
python run.py configs/tgi/eval_llama2_7b_chat_tgi.py.py
--debug
python run.py configs/tgi/eval_llama2_7b_chat_tgi.py.py
--debug
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment