Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
jerrrrry
vllm_test_tools
Commits
3e7e5c3e
Commit
3e7e5c3e
authored
May 29, 2025
by
jerrrrry
Browse files
Update test.sh
parent
b2895ed2
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
085-offline/test.sh
085-offline/test.sh
+2
-2
No files found.
085-offline/test.sh
View file @
3e7e5c3e
...
...
@@ -5,5 +5,5 @@ model_path=/llm-models/qwen1.5/Qwen1.5-0.5B-Chat
batch
=
"1 2"
prompt_tokens
=
"16 64"
completion_tokens
=
"128 256"
python benchmark_throughput_0.7.2.py
--model
${
model_path
}
--tensor-parallel-size
${
tp
}
--num-prompts
${
batch
}
--input-len
${
prompt_tokens
}
--output-len
${
completion_tokens
}
\
--dtype
float16
--trust-remote-code
--max-model-len
32768
--output-json
./test_0.5B-0.7.2.txt
\ No newline at end of file
python benchmark_throughput_0.8.5.py
--model
${
model_path
}
--tensor-parallel-size
${
tp
}
--num-prompts
${
batch
}
--input-len
${
prompt_tokens
}
--output-len
${
completion_tokens
}
\
--dtype
float16
--trust-remote-code
--max-model-len
32768
--output-json
./test_0.5B-0.7.2.txt
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment