GitLab

Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Ying1123 <sqy1415@gmail.com>
Co-authored-by: merrymercy <lianminzheng@gmail.com>
Co-authored-by: makro
Co-authored-by: dhou-xai

Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>

Co-authored-by: Edenzzzz <wtan45@wisc.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Yi Zhang <1109276519@qq.com>

Co-authored-by: yinfan98 <1106310035@qq.com>

[Fix] use `torch.cat` instead of `torch.concat` to prevent entering the `Autograd` backends. (#4466)

Co-authored-by: Ke Bao <ISPObaoke@163.com>

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>

Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>

Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>

Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>

Co-authored-by: Chayenne <zhaochen20@outlook.com>

Co-authored-by: josephyou <josephyou@tencent.com>

Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>

Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>

Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>

Co-authored-by: dhou-xai <dhou@x.ai>