Commits · 33f8dba7c68a0dfae762f902da5e894cd27e2c3a · OpenDAS / vllm_cscc · GitLab

10 Jun, 2025 12 commits
- [Model] use AutoWeightsLoader for commandr (#19399) · 33f8dba7
  py-andy-c authored Jun 10, 2025
```
Signed-off-by: py-andy-c <pychen1017@gmail.com>
```
  33f8dba7
- [ROCm][V1] Adding ROCm to the list of plaforms using V1 by default (#19440) · 5241ca50
  Gregory Shtrasberg authored Jun 10, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  5241ca50
- [Misc] Slight improvement of the BNB (#19418) · b6553be1
  Jee Jee Li authored Jun 10, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  b6553be1
- [BugFix][FlashInfer] Fix attention backend interface mismatch with unexpected... · 467bef18
  Rachel Guo authored Jun 10, 2025
```
[BugFix][FlashInfer] Fix attention backend interface mismatch with unexpected keyword `use_irope` (#19134)
Signed-off-by: Yunqiu Guo <guorachel@meta.com>
```
  467bef18
- Revert "[v1] Add fp32 support to v1 engine through flex attn" (#19404) · 5f1ac1e1
  Isotr0py authored Jun 10, 2025
  
  5f1ac1e1
- Automatically bind CPU OMP Threads of a rank to CPU ids of a NUMA node. (#17930) · 9368cc90
  Louie Tsai authored Jun 09, 2025
```
Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Co-authored-by: Li, Jiang <bigpyj64@gmail.com>
```
  9368cc90
- [Core] Batch multi modal input using pinned memory (#19169) · 319cb1e3
  Lukas Geiger authored Jun 10, 2025
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
```
  319cb1e3
- [Bugfix] Fix modelscope token passed in (#19389) · 1efef716
  Li Wang authored Jun 10, 2025
```
Signed-off-by: wangli <wangli858794774@gmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1efef716
- [Core] Use tuple for kv cache group block ids (#19175) · 646d62f6
  Nick Hill authored Jun 09, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  646d62f6
- [Frontend] Add tqdm_leave_pbar to control progress bar visibility (#19357) · 6cd4ae8a
  Reid authored Jun 10, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  6cd4ae8a
- Use xla flag to improve the quantized model performance (#19303) · 9af6d22e
  XiongfeiWei authored Jun 09, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  9af6d22e
- [V1] Reuse V0's memory_profiling util for gpu worker memory profiling (#19312) · cc867be1
  Ye (Charlotte) Qi authored Jun 09, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  cc867be1
09 Jun, 2025 9 commits
- [Misc] Fix a config typo in disable_hybrid_kv_cache_manager configuration (#19383) · 3a7cd627
  Siyuan Liu authored Jun 09, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  3a7cd627
- [HOT-FIX] Add `kv_sharing_target_layer_name` argument to cutlass_mla backend (#19374) · 8058c911
  Pavani Majety authored Jun 09, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  8058c911
- [Frontend] Make TIMEOUT_KEEP_ALIVE configurable through env var (#18472) · 31f58be9
  liusiqian-tal authored Jun 10, 2025
```
Signed-off-by: liusiqian <liusiqian@tal.com>
```
  31f58be9
- [Bugfix][Core] Prevent token lengths exceeding `max_model_len` in V0 (#19348) · c1c7dbbe
  22quinn authored Jun 09, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  c1c7dbbe
- [Misc] Fixes and Optimizations for DeepEP + DeepGEMM combination. (#19298) · 5cf2daea
  Varun Sundar Rabindranath authored Jun 09, 2025
```
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
```
  5cf2daea
- [v1] Add fp32 support to v1 engine through flex attn (#19319) · b8089195
  Isotr0py authored Jun 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  b8089195
- [full_graph] Fix query_start_loc padding (#19321) · 770e5dcd
  Yinghai Lu authored Jun 09, 2025
```
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
```
  770e5dcd
- [Frontend] Remove unreachable code from llm.py (#19288) · 8335667c
  Kseniya Parkhamchuk authored Jun 08, 2025
```
Signed-off-by: KsuParkhamchuk <k.parkhamchuk@gmail.com>
```
  8335667c
- [Deprecation] Remove `inputs` arg fallback in Engine classes (#18799) · e31ae3de
  Cyrus Leung authored Jun 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e31ae3de
08 Jun, 2025 6 commits
- [Bugfix] model_max_length should consider max_model_len in tokenizer_config (#19201) · 2ffb9b6e
  wang.yuqi authored Jun 08, 2025
  
  2ffb9b6e
- [Multi Modal] Add an env var for message queue max chunk bytes (#19242) · cda10fa3
  jennyyyyzhen authored Jun 08, 2025
```
Signed-off-by: yZhen <yZhen@fb.com>
Co-authored-by: yZhen <yZhen@fb.com>
```
  cda10fa3
- [Quantization] Add compressed-tensors NVFP4 support (#18312) · c123bc33
  Dipika Sikka authored Jun 08, 2025
  
  c123bc33
- Add H20-3e fused MoE kernel tuning configs for Qwen3-235B-A22B (#19315) · 989dcee9
  Xu Wenqing authored Jun 08, 2025
```
Signed-off-by: Xu Wenqing <xuwq1993@qq.com>
```
  989dcee9
- [Bugfix] Re-enable use_cudagraph in vLLM v1 (#19299) · eaa2e510
  Richard Zou authored Jun 07, 2025
```
Signed-off-by: Richard Zou <zou3519@gmail.com>
```
  eaa2e510
- [Bugfix]: Fix TypeError: 'float' object cannot be interpreted as an integer (#19283) · d77f7fb8
  Chauncey authored Jun 08, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  d77f7fb8
07 Jun, 2025 3 commits
- [BugFix][V1] Fix memory profiling bug (#18974) · 2d8476e4
  Luka Govedič authored Jun 07, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  2d8476e4
- Add FlexAttention to V1 (#16078) · cf02f9b2
  Driss Guessous authored Jun 07, 2025
```
Signed-off-by: drisspg <drisspguessous@gmail.com>
```
  cf02f9b2
- [Kernel] Integrate CUTLASS MoE kernel with PPLX (#18762) · 84166fee
  ElizaWszola authored Jun 07, 2025
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  84166fee
06 Jun, 2025 10 commits
- [BugFix] Fix tpu_model_runner block_id concatenation (#19228) · 46ecc579
  Nick Hill authored Jun 06, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  46ecc579
- [Core] Fix abrupt request abort (#18485) · b6a3a9f7
  Nicolò Lucchesi authored Jun 07, 2025
```
Signed-off-by: nicklucche <nlucches@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  b6a3a9f7
- [Bugfix][Core] Update cancellation logic in `generate()` to handle Generator exits (#19225) · ca27f0f9
  Adolfo Victoria authored Jun 06, 2025
```
Co-authored-by: Adolfo Victoria <adovi@meta.com>
```
  ca27f0f9
- [BugFix] Fix MultiConnector test after HMA changes (#19291) · aad30bd3
  Nick Hill authored Jun 06, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  aad30bd3
- improve logits bias (#19041) · 8267f991
  Yu Guo authored Jun 06, 2025
  
  8267f991
- [Core] Raise when non-multi-instance DP clients target a DP rank (#19227) · 7353492a
  jmswen authored Jun 06, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  7353492a
- [Model] Optimize nemotron_h implementation (#19249) · 7661e92e
  Jee Jee Li authored Jun 06, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  7661e92e
- Fix CompilationConfig repr (#19091) · da511d54
  Richard Zou authored Jun 06, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  da511d54
- [Docs] Improve V1 KVConnector interface documentation (#19172) · 65c69444
  Nick Hill authored Jun 06, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  65c69444
- [Quantization] Bump compressed-tensors version; update NVFP4A16 test model (#19224) · 94870359
  Dipika Sikka authored Jun 06, 2025
```
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
```
  94870359