Support penalty in overlap mode; return logprob with chunked prefill; improve...

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988) Co-authored-by: SangBin Cho <rkooo567@gmail.com> Co-authored-by: dhou-xai <dhou@x.ai> Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>

Support penalty in overlap mode; return logprob with chunked prefill; improve...
Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988) Co-authored-by: SangBin Cho <rkooo567@gmail.com> Co-authored-by: dhou-xai <dhou@x.ai> Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>
ac238727 · Lianmin Zheng · 0194948f · ac238727 · ac238727 · ac238727
Commit ac238727 authored Mar 03, 2025 by Lianmin Zheng
20 changed files
--- a/python/sglang/test/runners.py
+++ b/python/sglang/test/runners.py
--- a/python/sglang/test/send_one.py
+++ b/python/sglang/test/send_one.py
--- a/python/sglang/test/test_utils.py
+++ b/python/sglang/test/test_utils.py
--- a/scripts/killall_sglang.sh
+++ b/scripts/killall_sglang.sh
--- a/scripts/playground/bench_speculative.py
+++ b/scripts/playground/bench_speculative.py
--- a/sgl-kernel/src/sgl-kernel/__init__.py
+++ b/sgl-kernel/src/sgl-kernel/__init__.py
--- a/test/lang/test_srt_backend.py
+++ b/test/lang/test_srt_backend.py
--- a/test/srt/run_suite.py
+++ b/test/srt/run_suite.py
--- a/test/srt/sampling/penaltylib/penalizers/test_frequency_penalty.py
+++ b/test/srt/sampling/penaltylib/penalizers/test_frequency_penalty.py
--- a/test/srt/sampling/penaltylib/penalizers/test_min_new_tokens.py
+++ b/test/srt/sampling/penaltylib/penalizers/test_min_new_tokens.py
--- a/test/srt/sampling/penaltylib/penalizers/test_presence_penalty.py
+++ b/test/srt/sampling/penaltylib/penalizers/test_presence_penalty.py
--- a/test/srt/sampling/penaltylib/penalizers/test_repetition_penalty.py
+++ b/test/srt/sampling/penaltylib/penalizers/test_repetition_penalty.py
--- a/test/srt/test_bench_serving.py
+++ b/test/srt/test_bench_serving.py
--- a/test/srt/test_eval_accuracy_large.py
+++ b/test/srt/test_eval_accuracy_large.py
--- a/test/srt/test_health_check.py
+++ b/test/srt/test_health_check.py
--- a/test/srt/test_hidden_states.py
+++ b/test/srt/test_hidden_states.py
--- a/test/srt/test_metrics.py
+++ b/test/srt/test_metrics.py
--- a/test/srt/test_mla.py
+++ b/test/srt/test_mla.py
--- a/test/srt/sampling/penaltylib/test_srt_endpoint_with_penalizers.py
+++ b/test/srt/sampling/penaltylib/test_srt_endpoint_with_penalizers.py
--- a/test/srt/test_session_control.py
+++ b/test/srt/test_session_control.py