Unverified Commit ee275cbf authored by Yan Ru Pei's avatar Yan Ru Pei Committed by GitHub
Browse files

fix: add the proper flag for mooncake benchmarking on router (#3878)


Signed-off-by: default avatarPeaBrane <yanrpei@gmail.com>
parent 9e5407f2
...@@ -184,7 +184,15 @@ python prefix_ratio_benchmark.py --output-dir results/experiment1 ...@@ -184,7 +184,15 @@ python prefix_ratio_benchmark.py --output-dir results/experiment1
### Step 4 (Alternative): Run Benchmarks with Real Trace Data ### Step 4 (Alternative): Run Benchmarks with Real Trace Data
Instead of synthetic benchmarks with controlled prefix ratios, you can benchmark using real trace data in [mooncake-style format](https://github.com/kvcache-ai/Mooncake/blob/d21da178bae8db9651cf18a76824c084145fc725/mooncake_trace.jsonl). This approach uses actual request patterns from production traces, potentially modified with synthesis parameters. Instead of synthetic benchmarks with controlled prefix ratios, you can benchmark using real trace data. This approach uses actual request patterns from production traces, potentially modified with synthesis parameters.
First, download the mooncake trace dataset:
```bash
wget https://raw.githubusercontent.com/kvcache-ai/Mooncake/d21da178bae8db9651cf18a76824c084145fc725/mooncake_trace.jsonl
```
Then run the benchmark:
```bash ```bash
python real_data_benchmark.py --input-dataset mooncake_trace.jsonl python real_data_benchmark.py --input-dataset mooncake_trace.jsonl
......
...@@ -48,6 +48,8 @@ def get_aiperf_cmd_for_trace( ...@@ -48,6 +48,8 @@ def get_aiperf_cmd_for_trace(
url, url,
"--input-file", "--input-file",
f"{input_dataset}", f"{input_dataset}",
"--custom-dataset-type",
"mooncake_trace",
"--fixed-schedule-auto-offset", "--fixed-schedule-auto-offset",
"--random-seed", "--random-seed",
str(seed), str(seed),
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment