Skip to content

GitLab

Unverified Commit 24e82227 authored by Weida Hong's avatar Weida Hong Committed by GitHub
Browse files

[Misc] Reduce initialization time of auto_tune (#23682) 


Signed-off-by: default avatarWeida Hong <wdhongtw@google.com>
parent 100b630a
Show whitespace changes
Inline Side-by-side
Showing with 6 additions and 1 deletion
benchmarks/auto_tune/auto_tune.sh
View file @ 24e82227
...@@ -103,10 +103,15 @@ start_server() { ...@@ -103,10 +103,15 @@ start_server() {
VLLM_USE_V1=1 VLLM_SERVER_DEV_MODE=1 \ VLLM_USE_V1=1 VLLM_SERVER_DEV_MODE=1 \
vllm serve "${common_args_array[@]}" > "$vllm_log" 2>&1 & vllm serve "${common_args_array[@]}" > "$vllm_log" 2>&1 &
fi fi
local server_pid=$!
# wait for 10 minutes... # wait for 10 minutes...
server_started=0 server_started=0
for i in {1..60}; do for i in {1..60}; do
# This line checks whether the server is still alive or not,
# since that we should always have permission to send signal to the server process.
kill -0 $server_pid 2> /dev/null || break
RESPONSE=$(curl -s -X GET "http://0.0.0.0:8004/health" -w "%{http_code}" -o /dev/stdout) RESPONSE=$(curl -s -X GET "http://0.0.0.0:8004/health" -w "%{http_code}" -o /dev/stdout)
STATUS_CODE=$(echo "$RESPONSE" | tail -n 1) STATUS_CODE=$(echo "$RESPONSE" | tail -n 1)
if [[ "$STATUS_CODE" -eq 200 ]]; then if [[ "$STATUS_CODE" -eq 200 ]]; then
...@@ -118,7 +123,7 @@ start_server() { ...@@ -118,7 +123,7 @@ start_server() {
done done
if (( ! server_started )); then if (( ! server_started )); then
echo "server did not start within 10 minutes. Please check server log at $vllm_log". echo "server did not start within 10 minutes or crashed. Please check server log at $vllm_log".
return 1 return 1
else else
return 0 return 0
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment