Commits · 980385f8c1839f98e04928cb07c643d20b140cf7 · OpenDAS / vllm_cscc · GitLab

08 Mar, 2025 4 commits
- [Bugfix][Disaggregated] Add a check in send_kv_caches_and_hidden_states and... · 980385f8
  Mathis Felardos authored Mar 08, 2025
```
[Bugfix][Disaggregated] Add a check in send_kv_caches_and_hidden_states and fix the reshape of the KVCache (#14369)
Signed-off-by: Mathis Felardos <mathis@mistral.ai>
```
  980385f8
- Revert "[Perf] Reduce MLA CPU overheads in V1 (#14384)" (#14471) · ca7a2d5f
  Tyler Michael Smith authored Mar 08, 2025
  
  ca7a2d5f
- [Bugfix][V1] Handle MLA in kv_cache_interface (#14462) · 33368140
  Tyler Michael Smith authored Mar 08, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  33368140
- [V1] Prompt logprobs + APC compatibility; prompt logprobs reqs cannot fill APC (#13949) · ef640440
  afeldman-nm authored Mar 07, 2025
  
  ef640440
07 Mar, 2025 25 commits
- [Bugfix] Fix torch_xla which can't handle None seed introduced in #14274 (#14459) · 66e16a03
  yarongmu-google authored Mar 07, 2025
```
Signed-off-by: Yarong Mu <ymu@google.com>
```
  66e16a03
- [V1][Metrics] Fix traceback with preemptions+LoRA (#14220) · e1f0835a
  Mark McLoughlin authored Mar 07, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  e1f0835a
- [V1] Eagerly remove finished requests from the batch (#14388) · 8ed5421a
  Nick Hill authored Mar 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  8ed5421a
- [v1] torch.compile integration explanation (#14437) · c6359e8c
  youkaichao authored Mar 08, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c6359e8c
- [Misc] Add Phi4-MM example (#14343) · 952a0749
  Jee Jee Li authored Mar 08, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  952a0749
- [Kernel] optimize performance of gptq marlin kernel when n is small (#14138) · d0feea31
  Jinzhen Lin authored Mar 08, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  d0feea31
- [Benchmarks] Make detokenization optional in benchmark scripts (#11697) · 58abe354
  Jeremy Arnold authored Mar 07, 2025
```
Signed-off-by: Jeremy Arnold <Jeremy.Arnold@amd.com>
```
  58abe354
- [Doc] Update prefix_caching.md to match the example image (#14420) · f7ebad23
  York-RDWang authored Mar 07, 2025
  
  f7ebad23
- [V1][Core] Support for Structured Outputs (#12388) · 80e9afb5
  Aaron Pham authored Mar 07, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  80e9afb5
- Use the optimized block sizes after tuning the kernel. (#14329) · 1e3598ed
  iefgnoix authored Mar 07, 2025
  
  1e3598ed
- Fix missing `kv_caches` and `attn_metadata` in `OpenVINOCausalLM` (#14271) · f7a6bd0f
  Harry Mellor authored Mar 07, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  f7a6bd0f
- [BUGFIX] Skip tokenization support for throughput benchmark (#12712) · 0ca3b8e0
  Aleksandr Malyshev authored Mar 07, 2025
```
Signed-off-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
```
  0ca3b8e0
- [Misc] Set default value of seed to None (#14274) · cc102814
  மனோஜ்குமார் பழனிச்சாமி authored Mar 07, 2025
```
Signed-off-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
```
  cc102814
- [Bugfix] Clean up multi-modal processors (#14417) · 05fb6718
  Cyrus Leung authored Mar 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  05fb6718
- [Bugfix] Further clean up LoRA test (#14422) · 12c29a88
  Jee Jee Li authored Mar 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  12c29a88
- correct wrong markdown syntax (#14414) · 70da0c07
  Peng Li authored Mar 07, 2025
```
Signed-off-by: vincent-pli <justdoit.pli@gmail.com>
```
  70da0c07
- [GH] Auto-apply multi-modality label to relevant PRs (#14402) · c1588a2c
  Cyrus Leung authored Mar 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c1588a2c
- OpenVINO: added CPU-like conditions (#14338) · 8ca7a71d
  Ilya Lavrenov authored Mar 07, 2025
```
Signed-off-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
```
  8ca7a71d
- [Build] Add nightly wheel fallback when latest commit wheel unavailable (#14358) · 63137cd9
  Isotr0py authored Mar 07, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  63137cd9
- [Bugfix] Fix JambaForCausalLM LoRA (#14370) · ddd1ef66
  Jee Jee Li authored Mar 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ddd1ef66
- [BugFix] Illegal Memory Access in the blockwise cutlass fp8 GEMMs (#14396) · e5e03c2c
  Lucas Wilkinson authored Mar 07, 2025
  
  e5e03c2c
- [FP8] Refactor apply_fp8_linear and apply_fp8_linear_generic into an object (#14390) · e1744502
  Luka Govedič authored Mar 07, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  e1744502
- [Perf] Reduce MLA CPU overheads in V1 (#14384) · dae68969
  Lucas Wilkinson authored Mar 06, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  dae68969
- [Bugfix] Correctly call `cudaProfilerStop` in benchmarks script (#14183) · c34eeec5
  Brayden Zhong authored Mar 06, 2025
```
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  c34eeec5
- [Doc] Fix a typo (#14385) · ad60bbb2
  Daniel Li authored Mar 06, 2025
  
  ad60bbb2
06 Mar, 2025 11 commits
- [Hardware][TPU]Enable ragged paged attention kernel and resolve recompilation issue (#14310) · 0578e5a4
  Chengji Yao authored Mar 06, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  0578e5a4
- [Docs] Add nsight guide to profiling docs (#14298) · 04222984
  Michael Goin authored Mar 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  04222984
- [V1][Bugfix] Standardize quantized kv cache rejection for attention backends (#14221) · 6832707e
  Michael Goin authored Mar 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  6832707e
- [Bug] Fix Attention when ignored in by quant_method (#14313) · 6b2ef5cd
  Michael Goin authored Mar 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  6b2ef5cd
- [Bugfix] Fix use_direct_call condition in FusedMoE layer for (#14382) · 958adce4
  Tyler Michael Smith authored Mar 06, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  958adce4
- [Kernel] Add needs_fixed_stride_order tag to most GEMMs (#14306) · 99b0915d
  Tyler Michael Smith authored Mar 06, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  99b0915d
- [CI] Disable spawn when running V1 Test (#14345) · 8ca2b21c
  Thomas Parnell authored Mar 06, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  8ca2b21c
- [CI/Build] Use uv python for docker rather than ppa:deadsnakes/ppa (#13569) · d9292786
  Michael Goin authored Mar 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  d9292786
- [Distributed] Add enable_expert_parallel arg (#14305) · cc2f9b32
  Tyler Michael Smith authored Mar 06, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  cc2f9b32
- [V1] Do not detokenize if sampling param detokenize is False (#14224) · cd579352
  Himanshu Jaju authored Mar 06, 2025
```
Signed-off-by: Himanshu Jaju <hj@mistral.ai>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  cd579352
- Fix mla prefill context performance (#13897) · 9f1710f1
  Ying Zhong authored Mar 07, 2025
```
Signed-off-by: ZhongYingMatrix <zhongyingmatrix@gmail.com>
```
  9f1710f1