Commits · 04149cce2775a5d7ba4ec5dec693003f81e58cba · OpenDAS / vllm_cscc

09 Apr, 2025 10 commits
- [BugFix] fix some typos found by typos. (#16314) · 04149cce
  yihong authored Apr 09, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  04149cce
- update neuron config (#16289) · 24834f48
  ajayvohra2005 authored Apr 09, 2025
```
Signed-off-by: Ajay Vohra <ajayvohr@amazon.com>
```
  24834f48
- [BugFix] llama4 qknorm should be not shared across head (#16311) · ec7da6fc
  Lucia Fang authored Apr 09, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  ec7da6fc
- [BugFix] logger is not callable (#16312) · 819d548e
  yihong authored Apr 09, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  819d548e
- [Bugfix] Avoid transferring cached multi-modal items from P0 to P1 (#16273) · e484e028
  Cyrus Leung authored Apr 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e484e028
- [Core] Upgrade to xgrammar 0.1.18, add cache size limit (#16283) · cb84e45a
  Russell Bryant authored Apr 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  cb84e45a
- [Feature] Estimate max-model-len use available KV cache memory (#16168) · 4716377f
  rongfu.leng authored Apr 09, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  4716377f
- [Bug] [ROCm] Fix Llama 4 Enablement Bug on ROCm: V0 ROCmFlashAttentionImpl and... · 2976dc27
  TJian authored Apr 09, 2025
```
[Bug] [ROCm] Fix Llama 4 Enablement Bug on ROCm: V0 ROCmFlashAttentionImpl and Triton Fused MoE bugs (#16198)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: kliuae <kuanfu.liu@embeddedllm.com>
```
  2976dc27
- [Model] Add smolvlm support (#16017) · 102bf967
  Chauncey authored Apr 09, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  102bf967
- Add support to modelopt quantization of Mixtral model (#15961) · 1f4b09b5
  yueshen2016 authored Apr 08, 2025
```
Signed-off-by: Yue <yueshen@nvidia.com>
```
  1f4b09b5
08 Apr, 2025 16 commits
- [Bugfix] fix deepseek fp16 scale bug (#14809) · db104221
  Jinzhen Lin authored Apr 09, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  db104221
- [BugFix] Fix Llama4 - Index Error When Single Request Near Max Context (#16209) · e1a2c699
  Lucas Wilkinson authored Apr 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  e1a2c699
- [Bugfix] Handle `process_weights_after_loading` for `QKVCrossParallelLinear` (#15328) · 40b4284f
  Isotr0py authored Apr 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  40b4284f
- [Bugfix] Proper input validation for multi-modal encoder-decoder models (#16156) · 4ebc0b96
  Cyrus Leung authored Apr 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  4ebc0b96
- [New Model]: jinaai/jina-embeddings-v3 (#16120) · 1f5d13ab
  wang.yuqi authored Apr 08, 2025
  
  1f5d13ab
- [Model] use AutoWeightsLoader for phimoe,qwen2_moe,qwen3_moe (#16203) · 5a1e1c83
  rongfu.leng authored Apr 08, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  5a1e1c83
- [Misc] Add warning for multimodal data in LLM.beam_search (#16241) · 69ecaa7c
  Alex Brooks authored Apr 08, 2025
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  69ecaa7c
- [Misc] Merge the logs of pp layers partitions (#16225) · b4ac449a
  Kebe authored Apr 08, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
  b4ac449a
- [V1] Add `disable_chunked_mm_input` arg to disable partial mm input prefill (#15837) · 8e5314a4
  Michael Goin authored Apr 08, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  8e5314a4
- [torch.compile][TPU] Make @support_torch_compile work for XLA backend (#15782) · 87918e40
  Siyuan Liu authored Apr 07, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  87918e40
- [Bugfix] Do not skip "empty" parts of chats that are parsable (#16219) · b99733d0
  Michael Goin authored Apr 07, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b99733d0
- Add warning for Attention backends that do not support irope yet (#16212) · 05a015d6
  Yong Hoon Shin authored Apr 07, 2025
  
  05a015d6
- [Bugfix] fix use-ep bug to enable ep by dp/tp size > 1 (#16161) · ad971af8
  zxfan-cpu authored Apr 08, 2025
  
  ad971af8
- [V1] Scatter and gather placeholders in the model runner (#16076) · f2ebb6f5
  Roger Wang authored Apr 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
```
  f2ebb6f5
- [core] do not send error across process (#16174) · a865bc1c
  youkaichao authored Apr 08, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  a865bc1c
- [ROCm][Bugfix][FP8] Make fp8 quant respect fused modules mapping (#16031) · 21802c4b
  Michael Goin authored Apr 07, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  21802c4b
07 Apr, 2025 14 commits

Torchao (#14231) · 652907b3
Driss Guessous authored Apr 07, 2025
```
Signed-off-by: drisspg <drisspguessous@gmail.com>
```
652907b3
[Bugfix][V0] XGrammar structured output supports Enum (#15878) · 24f1c01e
leon-seidel authored Apr 08, 2025
```
Signed-off-by: Leon Seidel <leon.seidel@fau.de>
```
24f1c01e

[Misc] add description attribute in CLI (#15921) · fad6e253

Reid authored Apr 08, 2025


Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>

fad6e253

[V1][BugFix] Exit properly if engine core fails during startup (#16137) · 7f6d47c1
Nick Hill authored Apr 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
7f6d47c1
[Bugfix] Fix guidance backend for Qwen models (#16210) · 3147586e
Benjamin Chislett authored Apr 07, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
3147586e
[Misc] Move Llama 4 projector call into encoder execution (#16201) · ed636d99
Roger Wang authored Apr 07, 2025

ed636d99

[Misc] Human-readable `max-model-len` cli arg (#16181) · 090c856d

Nicolò Lucchesi authored Apr 07, 2025


Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

090c856d

Print the warning only once (#16193) · ad434d4c
Gregory Shtrasberg authored Apr 07, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
ad434d4c
[V1] Revert the default `max_num_seqs` to V0 values for most hardware (#16158) · 66d433b9
Cyrus Leung authored Apr 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
66d433b9
[Bugfix] Re-enable support for `ChatGLMForConditionalGeneration` (#16187) · 027b204f
Cyrus Leung authored Apr 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
027b204f

Upstream Llama4 Support to Main (#16113) · 55dcce91

Lu Fang authored Apr 07, 2025


Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

55dcce91

[Model] Add Qwen3 and Qwen3MoE (#15289) · 7699258e

YamPengLi authored Apr 07, 2025


Signed-off-by: YamPengLi <yampayne.lyp@alibaba-inc.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

7699258e

[V1][Structured Output] Add `supports_structured_output()` method to Platform (#16148) · e9ba99f2
Shanshan Shen authored Apr 07, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
e9ba99f2
[VLM] Florence-2 supports online serving (#16164) · 7c803687
Isotr0py authored Apr 07, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
7c803687