Commits · aa49f148322a39727be110da51a6782d43a2f5d8 · OpenDAS / vllm_cscc · GitLab

05 Jun, 2025 18 commits
- [Quantization] Skip Fp4 Test for `compressed-tensors` (#19217) · aa49f148
  Dipika Sikka authored Jun 05, 2025
  
  aa49f148
- [P/D][NixlConnector] Enable FlashInfer backend (#19090) · 9ef9173c
  Nicolò Lucchesi authored Jun 05, 2025
  
  9ef9173c
- [MISC][Bugfix] Use less CPU when message queue has been empty for some time (#16226) · 85e2b7bb
  Povilas Kanapickas authored Jun 05, 2025
```
Signed-off-by: Povilas Kanapickas <povilas@radix.lt>
```
  85e2b7bb
- [Hardware][NVIDIA] FP4 MoE kernel optimization (#19110) · 61059bee
  Chiyue Wei authored Jun 05, 2025
```
Signed-off-by: Chiyue Wei <chiyuew@nvidia.com>
Co-authored-by: Chiyue Wei <chiyuew@nvidia.com>
```
  61059bee
- Add H20-3e fused MoE kernel tuning configs for DeepSeek-R1/V3 (#19205) · ec89524f
  Xu Wenqing authored Jun 06, 2025
  
  ec89524f
- [mistral_common] Add v11 tokenizer (#19193) · f20f9f06
  Patrick von Platen authored Jun 05, 2025
```
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
```
  f20f9f06
- [Bugfix] properly catch PIL-related errors for vision models when incorrect... · 9bc8bb07
  Guillaume Calmettes authored Jun 05, 2025
```
[Bugfix] properly catch PIL-related errors for vision models when incorrect data urls are provided (#19202)
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  9bc8bb07
- [Frontend] improve vllm run-batch --help display (#19187) · 1aeb925f
  Reid authored Jun 05, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  1aeb925f
- [Misc] Do not override NCCL_CUMEM_ENABLE if set explicitly (#19105) · 188a4590
  22quinn authored Jun 05, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  188a4590
- [Misc] Remove unnecessary fallback to prefill-decode attention (#19138) · 18093084
  vllmellm authored Jun 05, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  18093084
- [Build] Annotate wheel and container path for release workflow (#19162) · da403802
  Simon Mo authored Jun 04, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  da403802
- [Bugfix]: Fix the incompatibility issue with stream when Thinking is disabled (#19135) · 8fc57501
  Chauncey authored Jun 05, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  8fc57501
- [BugFix][Minor] Fix full cuda graph bug when max_num_seqs < 512 (#19171) · af7fc84f
  Woosuk Kwon authored Jun 04, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  af7fc84f
- Handle non-serializable objects when dumping benchmark results (#19114) · 0678b522
  Huy Do authored Jun 04, 2025
  
  0678b522
- [Torch Nightly]add missing dependency (#18770) · 25b918ee
  Yang Wang authored Jun 04, 2025
```
Signed-off-by: Yang Wang <elainewy@meta.com>
```
  25b918ee
- [Bugfix] Fix port handling in make_zmq_path (#19117) · a408820f
  Michael Goin authored Jun 04, 2025
  
  a408820f
- [Bugfix][Nixl] Fix full prefix cache hit bug (#18632) · c56ed8bb
  Robert Shaw authored Jun 04, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  c56ed8bb
- [doc] small fix (#19167) · 78dcf56c
  Reid authored Jun 05, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  78dcf56c
04 Jun, 2025 22 commits
- [P/D] Heterogeneous TP (#18833) · b2fac671
  Nicolò Lucchesi authored Jun 05, 2025
```
Signed-off-by: nicklucche <nlucches@redhat.com>
```
  b2fac671
- [Misc] refactor: simplify EngineCoreClient.make_async_mp_client in AsyncLLM (#18817) · 23027e2d
  CYJiang authored Jun 05, 2025
```
Signed-off-by: googs1025 <googs1025@gmail.com>
```
  23027e2d
- [Kernel] Integrate batched/masked deepgemm kernel (#19111) · c3fd4d66
  Varun Sundar Rabindranath authored Jun 04, 2025
```
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
```
  c3fd4d66
- [Bugfix] fix v1 cpu worker fails on macOS (#19121) · ef3f98b5
  Kebe authored Jun 05, 2025
  
  ef3f98b5
- [TPU] Update dynamo dump file name in compilation test (#19108) · 7ee25904
  Siyuan Liu authored Jun 04, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  7ee25904
- [Perf] Tunings for SM100 FP8 CUTLASS kernel (#18778) · 53a5a0ce
  Michael Goin authored Jun 04, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  53a5a0ce
- [Bugfix][EP+DP] Fix internode check (#19112) · d459fae0
  Tyler Michael Smith authored Jun 04, 2025
```
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
```
  d459fae0
- Allow AsyncLLMEngine.generate to target a specific DP rank (#19102) · c8dcc159
  jmswen authored Jun 04, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  c8dcc159
- [Doc] Update V1 Guide for embedding models (#19141) · 8f4ffbd3
  Cyrus Leung authored Jun 04, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f4ffbd3
- Sm100 blockwise fp8 swap ab (#18564) · 5f2cd251
  Lain authored Jun 04, 2025
  
  5f2cd251
- Add DeepSeek-R1-0528 function call chat template (#18874) · 02658c2d
  Xu Wenqing authored Jun 04, 2025
```
Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
```
  02658c2d
- [CI/Build][Bugfix] Ensure compatibility with transformers 4.52 (#18678) · 01dc9a76
  Cyrus Leung authored Jun 04, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  01dc9a76
- Improve the output precision of embedding models (#19092) · 35cf32df
  wang.yuqi authored Jun 04, 2025
  
  35cf32df
- [Misc] Add packages for benchmark as extra dependency (#19089) · 8711bc5e
  Isotr0py authored Jun 04, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  8711bc5e
- Fix ValueError: Missing value for tag key(s): model_name,engine. (#19113) · 2669a0d7
  Seiji Eicher authored Jun 04, 2025
```
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
```
  2669a0d7
- [TPU] Skip hanging tests (#19115) · 8e972d9c
  Siyuan Liu authored Jun 04, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  8e972d9c
- Fix #19130 (#19132) · 3336c8cf
  汪志鹏 authored Jun 04, 2025
```
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
```
  3336c8cf
- [Bugfix] Fix FA3 full cuda graph correctness (#19106) · b124e108
  Woosuk Kwon authored Jun 03, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b124e108
- [NVIDIA] Add Cutlass MLA backend (#17625) · 41aa5784
  Kaixi Hou authored Jun 04, 2025
  
  41aa5784
- [Cleanup][v1]:remote guided-decoding-backend for example (#19059) · 8d646c2e
  Calvin Chen authored Jun 04, 2025
```
Signed-off-by: calvin chen <120380290@qq.com>
```
  8d646c2e
- [KERNEL] Sampler. CUDA kernel for applying repetition penalty (#18437) · 5d6d1adf
  Vadim Gimpelson authored Jun 04, 2025
  
  5d6d1adf
- [Core] Cast multimodal input in hf processor (#18862) · 1409ef91
  Lukas Geiger authored Jun 04, 2025
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
```
  1409ef91