GitLab

Co-authored-by: Edenzzzz <wtan45@wisc.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Yi Zhang <1109276519@qq.com>

Co-authored-by: Alex Kirillov <alex@iterationlab.org>

Co-authored-by: yinfan98 <1106310035@qq.com>

Co-authored-by: Sehoon Kim <sehoon@x.ai>

[Fix] use `torch.cat` instead of `torch.concat` to prevent entering the `Autograd` backends. (#4466)

Signed-off-by: wangyu <wangyu.steph@bytedance.com>

Signed-off-by: Changqi Lu <luchangqi.123@bytedance.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>

Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>

Co-authored-by: wenfeng.wf <wenfeng.wf@alibaba-inc.com>

Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>

Co-authored-by: Ke Bao <ISPObaoke@163.com>

Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>

Co-authored-by: Lucas Pickup <lupickup@microsoft.com>

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>

Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>