"vllm/vscode:/vscode.git/clone" did not exist on "efa6bed264b2dbb4c5d7a28e49fab60f6c69aef2"
Unverified Commit 03d4235f authored by Ruixiang Tan's avatar Ruixiang Tan Committed by GitHub
Browse files

[Misc] Fix the benchmark's README and improve the error messages for the...


[Misc] Fix the benchmark's README and improve the error messages for the benchmark's argument checks (#22654)
Signed-off-by: default avatartanruixiang <tanruixiang0104@gmail.com>
parent d6a1a209
...@@ -194,6 +194,7 @@ vllm serve Qwen/Qwen2-VL-7B-Instruct ...@@ -194,6 +194,7 @@ vllm serve Qwen/Qwen2-VL-7B-Instruct
```bash ```bash
vllm bench serve \ vllm bench serve \
--backend openai-chat \ --backend openai-chat \
--endpoint-type openai-chat \
--model Qwen/Qwen2-VL-7B-Instruct \ --model Qwen/Qwen2-VL-7B-Instruct \
--endpoint /v1/chat/completions \ --endpoint /v1/chat/completions \
--dataset-name hf \ --dataset-name hf \
...@@ -230,6 +231,7 @@ vllm serve Qwen/Qwen2-VL-7B-Instruct ...@@ -230,6 +231,7 @@ vllm serve Qwen/Qwen2-VL-7B-Instruct
```bash ```bash
vllm bench serve \ vllm bench serve \
--backend openai-chat \ --backend openai-chat \
--endpoint-type openai-chat \
--model Qwen/Qwen2-VL-7B-Instruct \ --model Qwen/Qwen2-VL-7B-Instruct \
--endpoint /v1/chat/completions \ --endpoint /v1/chat/completions \
--dataset-name hf \ --dataset-name hf \
...@@ -244,6 +246,7 @@ vllm bench serve \ ...@@ -244,6 +246,7 @@ vllm bench serve \
```bash ```bash
vllm bench serve \ vllm bench serve \
--backend openai-chat \ --backend openai-chat \
--endpoint-type openai-chat \
--model Qwen/Qwen2-VL-7B-Instruct \ --model Qwen/Qwen2-VL-7B-Instruct \
--endpoint /v1/chat/completions \ --endpoint /v1/chat/completions \
--dataset-name hf \ --dataset-name hf \
......
...@@ -740,10 +740,11 @@ def get_samples(args, tokenizer) -> list[SampleRequest]: ...@@ -740,10 +740,11 @@ def get_samples(args, tokenizer) -> list[SampleRequest]:
"openai-chat", "openai-chat",
"openai-audio", "openai-audio",
]: ]:
# multi-modal benchmark is only available on OpenAI Chat backend. # multi-modal benchmark is only available on OpenAI Chat
# endpoint-type.
raise ValueError( raise ValueError(
"Multi-modal content is only supported on 'openai-chat' and " "Multi-modal content is only supported on 'openai-chat' and "
"'openai-audio' backend.") "'openai-audio' endpoint-type.")
input_requests = dataset_class( input_requests = dataset_class(
dataset_path=args.dataset_path, dataset_path=args.dataset_path,
dataset_subset=args.hf_subset, dataset_subset=args.hf_subset,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment