Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
jerrrrry
infinilm
Commits
764f75fe
Commit
764f75fe
authored
Aug 14, 2025
by
PanZezhong
Browse files
update readme
parent
b4e3f49f
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
15 additions
and
5 deletions
+15
-5
README.md
README.md
+14
-4
scripts/jiuge_ppl.py
scripts/jiuge_ppl.py
+1
-1
No files found.
README.md
View file @
764f75fe
...
...
@@ -15,13 +15,23 @@ xmake && xmake install
-
运行模型推理测试
```
bash
python jiuge.py
[
--cpu
|
--nvidia
|
--cambricon
|
--ascend
|
--metax
|
--moore
]
<
path/to/model_dir
>
[
n_device]
python
scripts/
jiuge.py
[
--cpu
|
--nvidia
|
--cambricon
|
--ascend
|
--metax
|
--moore
]
path/to/model_dir
[
n_device]
```
-
部署模型推理服务
```
bash
launch_server.py
[
-h
]
[
--dev
{
cpu,nvidia,cambricon,ascend,metax,moore
}]
[
--model-path
MODEL_PATH]
[
--ndev
NDEV]
[
--max-batch
MAX_BATCH]
[
--max-tokens
MAX_TOKENS]
python scripts/launch_server.py
--model-path
MODEL_PATH
[
-h
]
[
--dev
{
cpu,nvidia,cambricon,ascend,metax,moore
}]
[
--ndev
NDEV]
[
--max-batch
MAX_BATCH]
[
--max-tokens
MAX_TOKENS]
```
-
测试模型推理服务性能
```
bash
python scripts/test_perf.py
```
-
使用推理服务测试模型困惑度(Perplexity)
```
bash
python scripts/test_ppl.py
--model-path
MODEL_PATH
[
--ndev
NDEV]
[
--max-batch
MAX_BATCH]
[
--max-tokens
MAX_TOKENS]
```
scripts/jiuge_ppl.py
View file @
764f75fe
...
...
@@ -90,7 +90,7 @@ if __name__ == "__main__":
dataset
=
load_dataset
(
"wikitext"
,
"wikitext-2-raw-v1"
,
split
=
"test"
)
texts
=
dataset
[
"text"
]
texts
=
[
t
for
t
in
texts
if
len
(
t
.
strip
())
>
0
]
texts
=
[
t
.
strip
()
for
t
in
texts
if
len
(
t
.
strip
())
>
0
]
input_ids_list
=
[]
for
text
in
texts
:
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment