Unverified Commit e6e4d022 authored by Ravi Theja's avatar Ravi Theja Committed by GitHub
Browse files

Update MMMU Benchmark instructions (#4694)

parent 188105a2
...@@ -2,12 +2,16 @@ ...@@ -2,12 +2,16 @@
### Evaluate sglang ### Evaluate sglang
Host the VLM:
``` ```
python -m sglang.launch_server --model-path Qwen/Qwen2-VL-7B-Instruct --port 30000 python -m sglang.launch_server --model-path Qwen/Qwen2-VL-7B-Instruct --chat-template qwen2-vl --port 30000
``` ```
Benchmark:
``` ```
python benchmark/mmmu/bench_sglang.py --model-path Qwen/Qwen2-VL-7B-Instruct --chat-template qwen2-vl --port 30000 python benchmark/mmmu/bench_sglang.py --port 30000
``` ```
It's recommended to reduce the memory usage by appending something ike `--mem-fraction-static 0.6` to the command above. It's recommended to reduce the memory usage by appending something ike `--mem-fraction-static 0.6` to the command above.
......
...@@ -2,7 +2,9 @@ ...@@ -2,7 +2,9 @@
Bench the sglang-hosted vLM with benchmark MMMU Bench the sglang-hosted vLM with benchmark MMMU
Usage: Usage:
python benchmark/mmmu/bench_sglang.py --model-path Qwen/Qwen2-VL-7B-Instruct --chat-template qwen2-vl Host the VLM: python -m sglang.launch_server --model-path Qwen/Qwen2-VL-7B-Instruct --chat-template qwen2-vl --port 30000
Benchmark: python benchmark/mmmu/bench_sglang.py --port 30000
The eval output will be logged The eval output will be logged
""" """
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment