Skip to content

GitLab

Commit 97f8f43f authored by jerrrrry's avatar jerrrrry
Browse files

Update test.sh

parent 249823a5
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
084-offline/test.sh
View file @ 97f8f43f
...@@ -5,5 +5,5 @@ model_path=/llm-models/qwen1.5/Qwen1.5-0.5B-Chat ...@@ -5,5 +5,5 @@ model_path=/llm-models/qwen1.5/Qwen1.5-0.5B-Chat
batch="1 2" batch="1 2"
prompt_tokens="16 64" prompt_tokens="16 64"
completion_tokens="128 256" completion_tokens="128 256"
python benchmark_throughput_0.7.2.py --model ${model_path} --tensor-parallel-size ${tp} --num-prompts ${batch} --input-len ${prompt_tokens} --output-len ${completion_tokens} \ python benchmark_throughput_0.8.4.py --model ${model_path} --tensor-parallel-size ${tp} --num-prompts ${batch} --input-len ${prompt_tokens} --output-len ${completion_tokens} \
--dtype float16 --trust-remote-code --max-model-len 32768 --output-json ./test_0.5B-0.7.2.txt --dtype float16 --trust-remote-code --max-model-len 32768 --output-json ./test_0.5B-0.7.2.txt
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment