Commits · f690372b6803c43870aefa631ab207ea3d163b1d · OpenDAS / vllm_cscc · GitLab

19 Mar, 2025 4 commits
- [Core] Update dtype detection and defaults (#14858) · f690372b
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f690372b
- [Model] Remove duplicated message check in Mistral chat completion request (#15069) · 8b3e94a3
  Brayden Zhong authored Mar 19, 2025
```
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  8b3e94a3
- [Model] Pixtral: Remove layer instantiation duplication (#15053) · 437f9162
  Julien Denize authored Mar 19, 2025
```
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
```
  437f9162
- [Misc][V1] Skip device checking if not available (#15061) · 4f065f12
  Cody Yu authored Mar 18, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  4f065f12
18 Mar, 2025 20 commits
- [Doc] Minor v1_user_guide update (#15064) · 228b768d
  Jennifer Zhao authored Mar 18, 2025
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  228b768d
- fix long dtype in topk sampling (#15049) · 027827cc
  Chujie Zheng authored Mar 19, 2025
  
  027827cc
- [V1] TPU - CI/CD use smaller model (#15054) · 72a8639b
  Alexander Matveev authored Mar 18, 2025
```
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
```
  72a8639b
- [V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels (#14930) · 99abb8b6
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  99abb8b6
- [V1] Refactor Structured Output for multiple backends (#14694) · 3a1e6481
  Russell Bryant authored Mar 18, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  3a1e6481
- [Bugfix] Fix LoRA extra vocab size (#15047) · 46c759c1
  Jee Jee Li authored Mar 19, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  46c759c1
- [Bugfix] Fix broken CPU quantization due to triton import (#15038) · 179a619c
  Isotr0py authored Mar 18, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  179a619c
- [MODEL] Add support for Zamba2 models (#13185) · 452e8fd9
  yury-tokpanov authored Mar 18, 2025
```
Signed-off-by: Yury Tokpanov <yury@zyphra.com>
Signed-off-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  452e8fd9
- MI325 configs, fused_moe_kernel bugfix (#14987) · 8b793f7e
  ekuznetsov139 authored Mar 18, 2025
```
Signed-off-by: Eugene Kuznetsov <eugene.kuznetsov@amd.com>
```
  8b793f7e
- [TPU][V1][Bugfix] Fix chunked prefill with padding (#15037) · af35d3a3
  Nicolò Lucchesi authored Mar 18, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  af35d3a3
- [Bugfix] Register serializers for V0 MQ Engine (#15009) · 3b457143
  Simon Mo authored Mar 18, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  3b457143
- [Bugfix] Loosen type check to avoid errors in V1 (#15021) · ab656f2c
  Cyrus Leung authored Mar 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ab656f2c
- [Misc][Docs] fix the comments of KV_T and CACHE_T in CALL_RESHAPE_AND_CACHE_XX macros (#14347) · 64fc2193
  Serena authored Mar 18, 2025
  
  64fc2193
- [Bugfix][Frontend] Fix validation of `logprobs` in `ChatCompletionRequest` (#14352) · dd732028
  Sebastian Schoennenbeck authored Mar 18, 2025
```
Signed-off-by: Sebastian Schönnenbeck <sebastian.schoennenbeck@comma-soft.com>
```
  dd732028
- [Bugfix] torchrun compatibility (#14899) · 41491913
  hoshi-hiyouga authored Mar 18, 2025
```
Signed-off-by: hiyouga <hiyouga@buaa.edu.cn>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  41491913
- [Misc] Embedding model support LoRA (#14935) · db7c8ca9
  Jee Jee Li authored Mar 18, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  db7c8ca9
- [Mistral-Small 3.1] Update docs and tests (#14977) · f863ffc9
  Patrick von Platen authored Mar 18, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  f863ffc9
- [Kernels] LoRA - Retire SGMV and BGMV Kernels (#14685) · 400d483e
  Varun Sundar Rabindranath authored Mar 18, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  400d483e
- [Doc][V1] Fix V1 APC doc (#14920) · d1695758
  Shanshan Shen authored Mar 18, 2025
  
  d1695758
- [Neuron] trim attention kernel tests to fit trn1.2x instance (#14988) · 53a0cf8b
  Liangfu Chen authored Mar 18, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
  53a0cf8b
17 Mar, 2025 16 commits
- [Bugfix] Fix bnb quantization for models with both HF-format and Mistral-format weights (#14950) · 5eeabc2a
  Tristan Leclercq authored Mar 18, 2025
  
  5eeabc2a
- [V1] TPU - Fix CI/CD runner (#14974) · 18551e82
  Alexander Matveev authored Mar 17, 2025
  
  18551e82
- [V1] Guard Against Main Thread Usage (#14972) · e41e1602
  Robert Shaw authored Mar 17, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  e41e1602
- [CI/Build] Use `AutoModelForImageTextToText` to load VLMs in tests (#14945) · b89fb2a4
  Cyrus Leung authored Mar 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b89fb2a4
- [Bugfix] Fix interface for Olmo2 on V1 (#14976) · 5340b0e2
  Roger Wang authored Mar 17, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  5340b0e2
- [Bugfix] Make Gemma3 MM V0 only for now (#14971) · 37e38061
  Roger Wang authored Mar 17, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  37e38061
- [Fix][Structured Output] using vocab_size to construct matcher (#14868) · c0efdd65
  Aaron Pham authored Mar 17, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
```
  c0efdd65
- [Bugfix][Model] Mixtral: use unused head_dim config argument (#14961) · aaaec52a
  Quentin authored Mar 17, 2025
```
Signed-off-by: Quentin Torroba <quentin.torroba@mistral.ai>
```
  aaaec52a
- [Bugfix] Fix precommit - line too long in pixtral.py (#14960) · e1eb45d3
  Tyler Michael Smith authored Mar 17, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e1eb45d3
- [V1] Default MLA to V1 (#14921) · 89fca671
  Simon Mo authored Mar 17, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  89fca671
- Add patch merger (#14957) · d20b0c13
  Patrick von Platen authored Mar 17, 2025
  
  d20b0c13
- [Doc] Fix misleading log during multi-modal profiling (#14955) · 166a168b
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  166a168b
- [Bugfix][ROCm] running new process using spawn method for rocm in tests. (#14810) · 2bb0e1a7
  vllmellm authored Mar 17, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  2bb0e1a7
- [Misc] Add `--seed` option to offline multi-modal examples (#14934) · 6eaf1e5c
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6eaf1e5c
- [Bugfix] Fix Ultravox on V1 (#14929) · 868a8c5b
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  868a8c5b
- [V1][TPU] Apply the ragged paged attention kernel fix and remove the padding. (#14846) · b4ad56c1
  iefgnoix authored Mar 17, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  b4ad56c1