GitLab

[fix]1.优化mtp代码，解决prefix-caching不兼容问题; 2.修复v1 engine从tokenizer config中读取max_model_len导致长文本输入报错

See merge request dcutoolkit/deeplearing/vllm!158

tbo debug fix error

See merge request dcutoolkit/deeplearing/vllm!156

[fix]修复091 v0 engine mtp报错问题

See merge request dcutoolkit/deeplearing/vllm!155

1.优化v1 engine mtp;2.添加v1 engine chunked prefill开关

See merge request dcutoolkit/deeplearing/vllm!152

This reverts commit e202dd27.

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>

Signed-off-by: reidliu41 <reid201711@gmail.com>

[Benchmark] Add support for multiple batch size benchmark through CLI in `benchmark_moe.py` (#20516)
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>