Commits · 55880ca2fbfc446449f08d8db0eca92470f09c57 · OpenDAS / vllm_cscc

21 Apr, 2025 1 commit
- update test_flash_mla.py · 3357c074
  zhuwenwen authored Apr 21, 2025
  
  3357c074
19 Apr, 2025 1 commit
- [Bugfix] Fix GLM4 model · 9e14887f
  zhuwenwen authored Apr 19, 2025
  
  9e14887f
14 Apr, 2025 1 commit
- [Core][V0] Enable regex support with xgrammar (#13228) · dc1b4a6f
  Russell Bryant authored Apr 13, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  dc1b4a6f
13 Apr, 2025 1 commit
- [V1][Spec Decode] KV cache slots for eagle heads (#16370) · f49e5aff
  Lily Liu authored Apr 12, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  f49e5aff
12 Apr, 2025 6 commits
- [Bugfix] Validate logit biases to prevent out of vocab ids crashing engine (#16529) · 6c11ecf8
  Ryan McConville authored Apr 12, 2025
```
Signed-off-by: Ryan McConville <ryan@ryanmcconville.com>
```
  6c11ecf8
- [V1] Enable multi-input by default (#15799) · d9fc8cd9
  Cyrus Leung authored Apr 12, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d9fc8cd9
- [Misc] Update chat utils tests (#16520) · c5bc0e7f
  Cyrus Leung authored Apr 12, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c5bc0e7f
- [Frontend] support matryoshka representation / support embedding API dimensions (#16331) · fbf722c6
  wang.yuqi authored Apr 12, 2025
  
  fbf722c6
- [Feature][V1] Add xgrammar to support minLength, maxLength with test (#16516) · e92d7085
  leon-seidel authored Apr 12, 2025
```
Signed-off-by: Leon Seidel <leon.seidel@fau.de>
```
  e92d7085
- [BugFix] Handle non-contiguous tensors properly when serializing (#16492) · 41cc883c
  Nick Hill authored Apr 11, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  41cc883c
11 Apr, 2025 7 commits
- [Frontend] Added chat templates for LLaMa4 pythonic tool calling (#16463) · 16eda8c4
  Ye (Charlotte) Qi authored Apr 11, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Co-authored-by: Kai Wu <kaiwu@meta.com>
```
  16eda8c4
- [Bugfix] handle alignment of encoder_seq_lens in mllama.py (#14784) · 71b9cde0
  Travis Johnson authored Apr 11, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  71b9cde0
- [Kernel] Support W8A8 channel-wise weights and per-token activations in triton... · f41647ee
  Michael Goin authored Apr 11, 2025
```
[Kernel] Support W8A8 channel-wise weights and per-token activations in triton fused_moe_kernel (#16366)
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f41647ee
- [Bugfix] Fix bugs of running Quark quantized models (#16236) · 9e90c9f7
  chaow-amd authored Apr 11, 2025
```
Signed-off-by: chaow <chaow@amd.com>
```
  9e90c9f7
- [Kernel] support merge_attn_states CUDA kernel, 3x speedup (#16173) · e9528f6d
  DefTruth authored Apr 11, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  e9528f6d
- Enforce valid max_num_batched_tokens when disable_chunked_mm_input=True (#16447) · aa3b3d76
  Michael Goin authored Apr 11, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  aa3b3d76
- [Bugfix][VLM] Fix failing Phi-4-MM multi-images tests and add vision-speech test (#16424) · 93195146
  Isotr0py authored Apr 11, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  93195146
10 Apr, 2025 8 commits

[TPU][V1] Disable per-request seed/Generator (#16172) · 3cc9af88
Nicolò Lucchesi authored Apr 10, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
3cc9af88
[V1] Zero-copy tensor/ndarray serialization/transmission (#13790) · dd143ef5
Nick Hill authored Apr 10, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
dd143ef5
[V1][Spec Decode] Eagle Model loading (#16035) · e8224f3d
Lily Liu authored Apr 10, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
e8224f3d
[V1] Set structured output backend to `auto` by default (#15724) · 9665313c
Russell Bryant authored Apr 10, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
9665313c
[VLM] Remove `BaseProcessingInfo.get_mm_max_tokens_per_item` (#16408) · 83b824c8
Cyrus Leung authored Apr 11, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
83b824c8
[Misc] Update transformers version limits of multi-modal tests (#16381) · 3d4c8775
Cyrus Leung authored Apr 10, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
3d4c8775

Add GLM-4-0414 support (#16338) · 1e44ffc3

Yuxuan Zhang authored Apr 10, 2025


Signed-off-by: lvfei.lv <lvfei.lv@alibaba-inc.com>
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Ajay Vohra <ajayvohr@amazon.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
Co-authored-by: Accelerator1996 <lvfei.lv@alibaba-inc.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: yihong <zouzou0208@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: ajayvohra2005 <ajayvohr@amazon.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Guillaume Calmettes <gcalmettes@scaleway.com>

1e44ffc3

[TPU][V1] Refine tpu_model_runner to mitigate future recompilation issues (#16275) · a4547485
Chengji Yao authored Apr 09, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
a4547485

09 Apr, 2025 8 commits
- [Bugfix][Frontend] respect provided default guided decoding backend (#15476) · 98d01d3c
  Guillaume Calmettes authored Apr 09, 2025
```
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  98d01d3c
- [Misc] Fix test_sharded_state_loader.py(#16004) (#16005) · 24f6b9a7
  Accelerator1996 authored Apr 09, 2025
```
Signed-off-by: lvfei.lv <lvfei.lv@alibaba-inc.com>
```
  24f6b9a7
- [BugFix] Fix fusion test and add them to CI (#16287) · 9cdde472
  Luka Govedič authored Apr 09, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  9cdde472
- [TPU] Update PyTorch/XLA (#16288) · b1eb4ca1
  Chengji Yao authored Apr 08, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  b1eb4ca1
- [CI][Bugfix] Fix bad tolerance for test_batch_base64_embedding (#16221) · 87b4ac56
  Michael Goin authored Apr 08, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  87b4ac56
- [Feature] Estimate max-model-len use available KV cache memory (#16168) · 4716377f
  rongfu.leng authored Apr 09, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  4716377f
- [Model] Add smolvlm support (#16017) · 102bf967
  Chauncey authored Apr 09, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  102bf967
- [CI/Build] Fix CI LoRA failure (#16270) · 86c3369e
  Jee Jee Li authored Apr 09, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  86c3369e
08 Apr, 2025 7 commits
- [Bugfix] Proper input validation for multi-modal encoder-decoder models (#16156) · 4ebc0b96
  Cyrus Leung authored Apr 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  4ebc0b96
- [New Model]: jinaai/jina-embeddings-v3 (#16120) · 1f5d13ab
  wang.yuqi authored Apr 08, 2025
  
  1f5d13ab
- [Bugfix] Remove triton do_bench fast_flush arg (#16256) · e11880de
  Kebe authored Apr 08, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
  e11880de
- [V1] Add `disable_chunked_mm_input` arg to disable partial mm input prefill (#15837) · 8e5314a4
  Michael Goin authored Apr 08, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  8e5314a4
- [Bugfix] Fix and reorganize broken GGUF tests and bump gguf version (#16194) · f6b32efb
  Isotr0py authored Apr 08, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  f6b32efb
- [Bugfix] Do not skip "empty" parts of chats that are parsable (#16219) · b99733d0
  Michael Goin authored Apr 07, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b99733d0
- [V1] Scatter and gather placeholders in the model runner (#16076) · f2ebb6f5
  Roger Wang authored Apr 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
```
  f2ebb6f5