Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
opencompass
Commits
8b3db123
Commit
8b3db123
authored
May 29, 2024
by
gaoqiong
Browse files
修改lmdeploy相关的config
parent
fb53af18
Changes
4
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
38 additions
and
1 deletion
+38
-1
README.md
README.md
+1
-1
configs/eval_llama2_chat_lmdeploy.py
configs/eval_llama2_chat_lmdeploy.py
+0
-0
configs/eval_llama2_lmdelpoy.py
configs/eval_llama2_lmdelpoy.py
+0
-0
configs/eval_llama2_lmdeploy.py
configs/eval_llama2_lmdeploy.py
+37
-0
No files found.
README.md
View file @
8b3db123
...
@@ -75,7 +75,7 @@ python run.py configs/eval_llama2_vllm.py
...
@@ -75,7 +75,7 @@ python run.py configs/eval_llama2_vllm.py
2、使用lmdeploy推理验证
2、使用lmdeploy推理验证
```
shell
```
shell
python run.py configs/eval_llama2_lmde
l
poy.py
python run.py configs/eval_llama2_lmdep
l
oy.py
```
```
3、使用tgi推理验证
3、使用tgi推理验证
...
...
configs/eval_llama2_chat_lmde
l
poy.py
→
configs/eval_llama2_chat_lmdep
l
oy.py
View file @
8b3db123
File moved
configs/eval_llama2_lmdelpoy.py
deleted
100755 → 0
View file @
fb53af18
configs/eval_llama2_lmdeploy.py
0 → 100755
View file @
8b3db123
from
mmengine.config
import
read_base
from
opencompass.models.turbomind
import
TurboMindModel
with
read_base
():
from
.datasets.ARC_c.ARC_c_gen_1e0de5
import
ARC_c_datasets
from
.datasets.ARC_e.ARC_e_gen_1e0de5
import
ARC_e_datasets
from
.summarizers.example
import
summarizer
datasets
=
sum
((
v
for
k
,
v
in
locals
().
items
()
if
k
.
endswith
(
'_datasets'
)),
[])
work_dir
=
'./outputs/llama2/'
llama_chat_meta_template
=
dict
(
round
=
[
dict
(
role
=
"HUMAN"
,
begin
=
'[INST] '
,
end
=
' [/INST]'
),
dict
(
role
=
"BOT"
,
begin
=
' '
,
end
=
' '
,
generate
=
True
),
],
)
models
=
[
dict
(
type
=
TurboMindModel
,
abbr
=
'llama-2-7b-hf-lmdeploy'
,
path
=
"Llama-2-7b-hf"
,
engine_config
=
dict
(
session_len
=
4096
,
max_batch_size
=
32
,
rope_scaling_factor
=
1.0
),
gen_config
=
dict
(
top_k
=
1
,
top_p
=
0.8
,
temperature
=
1.0
,
max_new_tokens
=
100
),
max_out_len
=
100
,
max_seq_len
=
2048
,
batch_size
=
1
,
concurrency
=
1
,
run_cfg
=
dict
(
num_gpus
=
1
,
num_procs
=
1
)
)
]
\ No newline at end of file
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment