Commits · dafb4e504ad1b83320bc2800675b52be0324ba12 · OpenDAS / vllm_cscc · GitLab

19 Mar, 2025 7 commits
- [V1][Bugfix] Fix oracle for device checking (#15104) · dafb4e50
  Roger Wang authored Mar 19, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  dafb4e50
- [Bugfix] Re-enable Gemma3 for V1 (#14980) · 61f41218
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61f41218
- [V1] Ensure using int64 for sampled token ids (#15065) · 05ccd0aa
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  05ccd0aa
- [Core] Update dtype detection and defaults (#14858) · f690372b
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f690372b
- [Model] Remove duplicated message check in Mistral chat completion request (#15069) · 8b3e94a3
  Brayden Zhong authored Mar 19, 2025
```
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  8b3e94a3
- [Model] Pixtral: Remove layer instantiation duplication (#15053) · 437f9162
  Julien Denize authored Mar 19, 2025
```
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
```
  437f9162
- [Misc][V1] Skip device checking if not available (#15061) · 4f065f12
  Cody Yu authored Mar 18, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  4f065f12
18 Mar, 2025 14 commits
- fix long dtype in topk sampling (#15049) · 027827cc
  Chujie Zheng authored Mar 19, 2025
  
  027827cc
- [V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels (#14930) · 99abb8b6
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  99abb8b6
- [V1] Refactor Structured Output for multiple backends (#14694) · 3a1e6481
  Russell Bryant authored Mar 18, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  3a1e6481
- [Bugfix] Fix LoRA extra vocab size (#15047) · 46c759c1
  Jee Jee Li authored Mar 19, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  46c759c1
- [Bugfix] Fix broken CPU quantization due to triton import (#15038) · 179a619c
  Isotr0py authored Mar 18, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  179a619c
- [MODEL] Add support for Zamba2 models (#13185) · 452e8fd9
  yury-tokpanov authored Mar 18, 2025
```
Signed-off-by: Yury Tokpanov <yury@zyphra.com>
Signed-off-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  452e8fd9
- MI325 configs, fused_moe_kernel bugfix (#14987) · 8b793f7e
  ekuznetsov139 authored Mar 18, 2025
```
Signed-off-by: Eugene Kuznetsov <eugene.kuznetsov@amd.com>
```
  8b793f7e
- [TPU][V1][Bugfix] Fix chunked prefill with padding (#15037) · af35d3a3
  Nicolò Lucchesi authored Mar 18, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  af35d3a3
- [Bugfix] Register serializers for V0 MQ Engine (#15009) · 3b457143
  Simon Mo authored Mar 18, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  3b457143
- [Bugfix] Loosen type check to avoid errors in V1 (#15021) · ab656f2c
  Cyrus Leung authored Mar 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ab656f2c
- [Bugfix][Frontend] Fix validation of `logprobs` in `ChatCompletionRequest` (#14352) · dd732028
  Sebastian Schoennenbeck authored Mar 18, 2025
```
Signed-off-by: Sebastian Schönnenbeck <sebastian.schoennenbeck@comma-soft.com>
```
  dd732028
- [Bugfix] torchrun compatibility (#14899) · 41491913
  hoshi-hiyouga authored Mar 18, 2025
```
Signed-off-by: hiyouga <hiyouga@buaa.edu.cn>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  41491913
- [Misc] Embedding model support LoRA (#14935) · db7c8ca9
  Jee Jee Li authored Mar 18, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  db7c8ca9
- [Kernels] LoRA - Retire SGMV and BGMV Kernels (#14685) · 400d483e
  Varun Sundar Rabindranath authored Mar 18, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  400d483e
17 Mar, 2025 18 commits
- [Bugfix] Fix bnb quantization for models with both HF-format and Mistral-format weights (#14950) · 5eeabc2a
  Tristan Leclercq authored Mar 18, 2025
  
  5eeabc2a
- [V1] Guard Against Main Thread Usage (#14972) · e41e1602
  Robert Shaw authored Mar 17, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  e41e1602
- [Bugfix] Fix interface for Olmo2 on V1 (#14976) · 5340b0e2
  Roger Wang authored Mar 17, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  5340b0e2
- [Bugfix] Make Gemma3 MM V0 only for now (#14971) · 37e38061
  Roger Wang authored Mar 17, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  37e38061
- [Fix][Structured Output] using vocab_size to construct matcher (#14868) · c0efdd65
  Aaron Pham authored Mar 17, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
```
  c0efdd65
- [Bugfix][Model] Mixtral: use unused head_dim config argument (#14961) · aaaec52a
  Quentin authored Mar 17, 2025
```
Signed-off-by: Quentin Torroba <quentin.torroba@mistral.ai>
```
  aaaec52a
- [Bugfix] Fix precommit - line too long in pixtral.py (#14960) · e1eb45d3
  Tyler Michael Smith authored Mar 17, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e1eb45d3
- [V1] Default MLA to V1 (#14921) · 89fca671
  Simon Mo authored Mar 17, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  89fca671
- Add patch merger (#14957) · d20b0c13
  Patrick von Platen authored Mar 17, 2025
  
  d20b0c13
- [Doc] Fix misleading log during multi-modal profiling (#14955) · 166a168b
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  166a168b
- [Bugfix] Fix Ultravox on V1 (#14929) · 868a8c5b
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  868a8c5b
- [V1][TPU] Apply the ragged paged attention kernel fix and remove the padding. (#14846) · b4ad56c1
  iefgnoix authored Mar 17, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  b4ad56c1
- [Misc][XPU] Use None as device capacity for XPU (#14932) · 9b87a579
  Yan Ma authored Mar 17, 2025
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  9b87a579
- [V1] Remove input cache client (#14864) · b539222d
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  b539222d
- [V1] [Spec Decode] Support random sampling for spec decode (#13933) · 8d6cf895
  Lily Liu authored Mar 16, 2025
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8d6cf895
- [BugFix] Fix MLA + V1 + TP==1 causing reinitialization of cuda context (#14910) · 1e799b7e
  Lucas Wilkinson authored Mar 16, 2025
  
  1e799b7e
- [V1][Minor] Add __repr__ to ConstantList (#14907) · 7f6c5ee0
  Woosuk Kwon authored Mar 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7f6c5ee0
- [V1] Optimize the overhead of rewinding (#14905) · faa02757
  Woosuk Kwon authored Mar 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  faa02757
16 Mar, 2025 1 commit
- [V1][BugFix] Detect interleaved sliding window attention (#14896) · 31060b27
  Woosuk Kwon authored Mar 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  31060b27