GitLab

Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>

Co-authored-by: yych0745 <1398089567@qq.com>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: b0urnee <2769086541@qq.com>

Signed-off-by: ZX-ModelCloud <zx@modelcloud.ai>
Co-authored-by: ZX-ModelCloud <zx@modelcloud.ai>

Co-authored-by: Lucas Pickup <lupickup@microsoft.com>

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>

Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>

Co-authored-by: HandH1998 <1335248067@qq.com>

Co-authored-by: ShenAo1111 <1377693092@qq.com>
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>

Co-authored-by: Chayenne <zhaochen20@outlook.com>

Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>

Co-authored-by: Ying Sheng <sqy1415@gmail.com>

Co-authored-by: bjmsong <bjmsong@126.com>