Support penalty in overlap mode; return logprob with chunked prefill; improve...

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988) Co-authored-by: SangBin Cho <rkooo567@gmail.com> Co-authored-by: dhou-xai <dhou@x.ai> Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>

Support penalty in overlap mode; return logprob with chunked prefill; improve...
Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988) Co-authored-by: SangBin Cho <rkooo567@gmail.com> Co-authored-by: dhou-xai <dhou@x.ai> Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>
ac238727 · Lianmin Zheng · 0194948f · ac238727 · ac238727 · ac238727
Commit ac238727 authored Mar 03, 2025 by Lianmin Zheng
Showing with 200 additions and 115 deletions

test/srt/test_skip_tokenizer_init.py test/srt/test_skip_tokenizer_init.py +89 -40

test/srt/test_srt_endpoint.py test/srt/test_srt_endpoint.py +110 -74

test/srt/test_verl_engine.py test/srt/test_verl_engine.py +1 -1

No files found.
--- a/test/srt/test_skip_tokenizer_init.py
+++ b/test/srt/test_skip_tokenizer_init.py
--- a/test/srt/test_srt_endpoint.py
+++ b/test/srt/test_srt_endpoint.py
--- a/test/srt/test_verl_engine.py
+++ b/test/srt/test_verl_engine.py