GitLab

[V1][Sampler] Improve performance of FlashInfer sampling by sampling logits instead of probs (#18608)

[V1] Fix _pickle.PicklingError: Can't pickle <class 'transformers_modules.deepseek-ai.DeepSeek-V2-Lite... (#18640)
Signed-off-by: Seiji Eicher <seiji@anyscale.com>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: qizixi <qizixi@meta.com>

Signed-off-by: qizixi <qizixi@meta.com>

Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>

Signed-off-by: Crucifixion-Fxl <xmufxl@gmail.com>
Co-authored-by: Crucifixion-Fxl <xmufxl@gmail.com>

Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

Signed-off-by: youkaichao <youkaichao@gmail.com>

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Co-authored-by: simon-mo <xmo@berkeley.edu>

Signed-off-by: Ronald Xu <ronaldxu@amazon.com>

Signed-off-by: Nick Hill <nhill@redhat.com>

Signed-off-by: Mark McLoughlin <markmc@redhat.com>

[Frontend] [Core] Add Tensorizer support for V1, LoRA adapter serialization and deserialization (#17926)
Signed-off-by: Sanger Steel <sangersteel@gmail.com>

Signed-off-by: Bowen Wang <abmfy@icloud.com>

Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: kourosh hakhamaneshi <kourosh@anyscale.com>

Signed-off-by: rabi <ramishra@redhat.com>

Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

Signed-off-by: Mark McLoughlin <markmc@redhat.com>

Signed-off-by: Yong Hoon Shin <yhshin@meta.com>

Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>

Signed-off-by: cascade812 <cascade812@outlook.com>

Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: Carol Zheng <cazheng@google.com>
Co-authored-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: Hongmin Fan <fanhongmin@google.com>

Signed-off-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

Signed-off-by: Seiji Eicher <seiji@anyscale.com>

Signed-off-by: Lucia Fang <fanglu@fb.com>

Signed-off-by: lisiqi23 <lisiqi23@xiaomi.com>
Signed-off-by: skylee-01 <497627264@qq.com>
Co-authored-by: lisiqi23 <lisiqi23@xiaomi.com>

Signed-off-by: kf <kuanfu.liu@embeddedllm.com>

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>

Signed-off-by: omahs <73983677+omahs@users.noreply.github.com>

Signed-off-by: Mark McLoughlin <markmc@redhat.com>

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

[V1][Spec Decode] Share input embedding of target model with EAGLE draft model to free ~1GB for llama 3 model (#17326)
Co-authored-by: root <root@ekagra-8xh100.us-east5-a.c.serving-efficiency-poc.internal>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>