Commits · f40f763f12c0ffbbc781b0ea23f4b9868db70d21 · OpenDAS / vllm_cscc

16 Jun, 2025 4 commits

[CI] Add mteb testing for rerank models (#19344) · f40f763f
wang.yuqi authored Jun 16, 2025

f40f763f
[TPU] support attention head dim smaller than 128 (#19620) · a77aea59
Chengji Yao authored Jun 15, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
a77aea59

[Misc] Fix skipped max-model-len validation when deriving max model length... · b692e9cd

Ye (Charlotte) Qi authored Jun 15, 2025


[Misc] Fix skipped max-model-len validation when deriving max model length from tokenizer config (#19660)
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>

b692e9cd

[Bugfix][Core] Prefix caching causes incorrect outputs due to outdated... · 92183b41

quanliu authored Jun 16, 2025


[Bugfix][Core] Prefix caching causes incorrect outputs due to outdated ComputedBlocksTracker (#18957)
Signed-off-by: 刘全 <quan.liu2@dbappsecurity.com.cn>
Co-authored-by: 刘全 <quan.liu2@dbappsecurity.com.cn>

92183b41

15 Jun, 2025 2 commits
- [Kernel] Raise verbose error and consolidate `num_heads/num_kv_heads` divisibility check (#19339) · 0b73736a
  22quinn authored Jun 14, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  0b73736a
- [Bugfix][2/n] Fix speculative decoding CI - Fix test_ngram_e2e_greedy_correctness (#19644) · ee1531bc
  Lu Fang authored Jun 15, 2025
  
  ee1531bc
14 Jun, 2025 3 commits
- [Bugfix] Fix auto dtype casting for BatchFeature (#19316) · 2db9044a
  Isotr0py authored Jun 14, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  2db9044a
- [Bugfix] Fix the speculative decoding test by setting the target dtype (#19633) · 06be8588
  Lu Fang authored Jun 14, 2025
  
  06be8588
- Adding "AMD: Multi-step Tests" to amdproduction. (#19508) · d65668b4
  Concurrensee authored Jun 13, 2025
```
Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d65668b4
13 Jun, 2025 1 commit
- [CUDA] Enable full cudagraph for FlashMLA (#18581) · 3597b06a
  Luka Govedič authored Jun 13, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  3597b06a
12 Jun, 2025 7 commits
- [Spec Decode][Benchmark] Generalize spec decode offline benchmark to more... · 017ef648
  Ekagra Ranjan authored Jun 12, 2025
```
[Spec Decode][Benchmark] Generalize spec decode offline benchmark to more methods and datasets (#18847)
```
  017ef648
- [torch.compile][ROCm] Fuse quantization onto attention using a torch.compile pass (#16756) · f98548b9
  Luka Govedič authored Jun 12, 2025
```
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
```
  f98548b9
- Fix TorchAOConfig skip layers (#19265) · 96846bb3
  mobicham authored Jun 12, 2025
```
Signed-off-by: mobicham <hicham@mobiuslabs.com>
```
  96846bb3
- [Perf] Vectorize static / dynamic INT8 quant kernels (#19233) · b6efafd9
  Wentao Ye authored Jun 12, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  b6efafd9
- [Bugfix] Respect num-gpu-blocks-override in v1 (#19503) · c9280e63
  jmswen authored Jun 12, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  c9280e63
- [BugFix] Work-around incremental detokenization edge case error (#19449) · d5bdf899
  Nick Hill authored Jun 11, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  d5bdf899
- [CI] change spell checker from codespell to typos (#18711) · 2f1c19b2
  Ning Xie authored Jun 12, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  2f1c19b2
11 Jun, 2025 6 commits
- [Kernels] Add activation chunking logic to FusedMoEModularKernel (#19168) · 29fa5cac
  bnellnm authored Jun 11, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  29fa5cac
- Support non-string values in JSON keys from CLI (#19471) · a2142f01
  Cyrus Leung authored Jun 11, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a2142f01
- Fix Typo in Documentation and Function Name (#19442) · 7c644ab6
  leopardracer authored Jun 11, 2025
  
  7c644ab6
- [CI] Disable failing GGUF model test (#19454) · 1e473b30
  Michael Goin authored Jun 11, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  1e473b30
- Fix test_max_model_len in tests/entrypoints/llm/test_generate.py (#19451) · 2b1e2111
  Lu Fang authored Jun 11, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  2b1e2111
- [New Model]: Support Qwen3 Embedding & Reranker (#19260) · 3952731e
  wang.yuqi authored Jun 11, 2025
  
  3952731e
10 Jun, 2025 3 commits
- [BugFix] Allow use_cudagraph to work with dynamic VLLM_USE_V1 (#19390) · 77f0d465
  Richard Zou authored Jun 10, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  77f0d465
- Revert "[v1] Add fp32 support to v1 engine through flex attn" (#19404) · 5f1ac1e1
  Isotr0py authored Jun 10, 2025
  
  5f1ac1e1
- [Core] Use tuple for kv cache group block ids (#19175) · 646d62f6
  Nick Hill authored Jun 09, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  646d62f6
09 Jun, 2025 7 commits
- [TPU]Fix KV cache sharing tests (#19371) · 7d44c469
  Siyuan Liu authored Jun 09, 2025
  
  7d44c469
- [Frontend] Make TIMEOUT_KEEP_ALIVE configurable through env var (#18472) · 31f58be9
  liusiqian-tal authored Jun 10, 2025
```
Signed-off-by: liusiqian <liusiqian@tal.com>
```
  31f58be9
- [Bugfix][Core] Prevent token lengths exceeding `max_model_len` in V0 (#19348) · c1c7dbbe
  22quinn authored Jun 09, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  c1c7dbbe
- [Misc] Fixes and Optimizations for DeepEP + DeepGEMM combination. (#19298) · 5cf2daea
  Varun Sundar Rabindranath authored Jun 09, 2025
```
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
```
  5cf2daea
- [v1] Add fp32 support to v1 engine through flex attn (#19319) · b8089195
  Isotr0py authored Jun 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  b8089195
- [CI/Build] Fix LoRA test (#19350) · 95a6568b
  Jee Jee Li authored Jun 09, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  95a6568b
- [Misc] Cleanup compilation tests (#19343) · 3a4d4177
  Richard Zou authored Jun 09, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  3a4d4177
08 Jun, 2025 3 commits
- [Quantization] Add compressed-tensors NVFP4 support (#18312) · c123bc33
  Dipika Sikka authored Jun 08, 2025
  
  c123bc33
- [Misc] Change tests/compile to use VLLM_V1 by default (#19302) · 3d64d366
  Richard Zou authored Jun 08, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  3d64d366
- [Bugfix] Re-enable use_cudagraph in vLLM v1 (#19299) · eaa2e510
  Richard Zou authored Jun 07, 2025
```
Signed-off-by: Richard Zou <zou3519@gmail.com>
```
  eaa2e510
07 Jun, 2025 4 commits
- [BugFix][V1] Fix memory profiling bug (#18974) · 2d8476e4
  Luka Govedič authored Jun 07, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  2d8476e4
- [CI/Build] Improve Llama GGUF test robustness (#19287) · d2f0e7e6
  Isotr0py authored Jun 07, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  d2f0e7e6
- Add FlexAttention to V1 (#16078) · cf02f9b2
  Driss Guessous authored Jun 07, 2025
```
Signed-off-by: drisspg <drisspguessous@gmail.com>
```
  cf02f9b2
- [Kernel] Integrate CUTLASS MoE kernel with PPLX (#18762) · 84166fee
  ElizaWszola authored Jun 07, 2025
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  84166fee