Commits · 93aee29fdb81fa51d5d44f34ea84df80d0764235 · OpenDAS / vllm_cscc · GitLab

17 Jun, 2025 1 commit
- Fixes IMA for TP w/ flex-attention (#19712) · ddfed314
  Driss Guessous authored Jun 16, 2025
```
Signed-off-by: drisspg <drisspguessous@gmail.com>
```
  ddfed314
16 Jun, 2025 4 commits
- [V1] Change return type on get_multimodal_embeddings() (#19446) · 90f9c2eb
  Russell Bryant authored Jun 16, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  90f9c2eb
- [Bugfix] Fix TP inference for Flex attention backend (#19657) · 1173804d
  Isotr0py authored Jun 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  1173804d
- [BugFix] Don't catch BaseException when dumping execute_model errors (#19626) · ee35e96a
  Nick Hill authored Jun 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  ee35e96a
- [TPU] support attention head dim smaller than 128 (#19620) · a77aea59
  Chengji Yao authored Jun 15, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  a77aea59
15 Jun, 2025 1 commit
- [Kernel] Raise verbose error and consolidate `num_heads/num_kv_heads` divisibility check (#19339) · 0b73736a
  22quinn authored Jun 14, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  0b73736a
14 Jun, 2025 2 commits
- [V1][Metrics] Deprecate metrics with gpu_ prefix for non GPU specific metrics. (#18354) · d1e34cc9
  Saheli Bhattacharjee authored Jun 14, 2025
```
Signed-off-by: Saheli Bhattacharjee <saheli@krai.ai>
```
  d1e34cc9
- [BugFix] Fix DP Coordinator incorrect debug log message (#19624) · bd517eb9
  Nick Hill authored Jun 13, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  bd517eb9
13 Jun, 2025 6 commits
- [CUDA] Enable full cudagraph for FlashMLA (#18581) · 3597b06a
  Luka Govedič authored Jun 13, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  3597b06a
- [BugFix] Honor `enable_caching` in connector-delayed kvcache load case (#19435) · 7e8d97dd
  Nick Hill authored Jun 13, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  7e8d97dd
- [torch.compile] reorganize the cache directory to support compiling multiple models (#19064) · d70bc7c0
  youkaichao authored Jun 13, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  d70bc7c0
- [Fix] The zip function in Python 3.9 does not have the strict argument (#19549) · cefdb996
  汪志鹏 authored Jun 13, 2025
```
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
```
  cefdb996
- [CPU] Refine default config for the CPU backend (#19539) · 64587211
  Li, Jiang authored Jun 13, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  64587211
- [Bugfix] Fix EAGLE vocab embedding for multimodal target model (#19570) · c68698b3
  qizixi authored Jun 12, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  c68698b3
12 Jun, 2025 4 commits
- [V1] Resolve failed concurrent structured output requests (#19565) · c57bb199
  Russell Bryant authored Jun 12, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  c57bb199
- [torch.compile][ROCm] Fuse quantization onto attention using a torch.compile pass (#16756) · f98548b9
  Luka Govedič authored Jun 12, 2025
```
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
```
  f98548b9
- [Bugfix] Respect num-gpu-blocks-override in v1 (#19503) · c9280e63
  jmswen authored Jun 12, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  c9280e63
- [BugFix] Work-around incremental detokenization edge case error (#19449) · d5bdf899
  Nick Hill authored Jun 11, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  d5bdf899
11 Jun, 2025 1 commit
- [UX] Add Feedback During CUDAGraph Capture (#19501) · 97a9465b
  Robert Shaw authored Jun 11, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  97a9465b
10 Jun, 2025 6 commits
- [BugFix][FlashInfer] Fix attention backend interface mismatch with unexpected... · 467bef18
  Rachel Guo authored Jun 10, 2025
```
[BugFix][FlashInfer] Fix attention backend interface mismatch with unexpected keyword `use_irope` (#19134)
Signed-off-by: Yunqiu Guo <guorachel@meta.com>
```
  467bef18
- Automatically bind CPU OMP Threads of a rank to CPU ids of a NUMA node. (#17930) · 9368cc90
  Louie Tsai authored Jun 09, 2025
```
Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Co-authored-by: Li, Jiang <bigpyj64@gmail.com>
```
  9368cc90
- [Core] Batch multi modal input using pinned memory (#19169) · 319cb1e3
  Lukas Geiger authored Jun 10, 2025
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
```
  319cb1e3
- [Core] Use tuple for kv cache group block ids (#19175) · 646d62f6
  Nick Hill authored Jun 09, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  646d62f6
- Use xla flag to improve the quantized model performance (#19303) · 9af6d22e
  XiongfeiWei authored Jun 09, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  9af6d22e
- [V1] Reuse V0's memory_profiling util for gpu worker memory profiling (#19312) · cc867be1
  Ye (Charlotte) Qi authored Jun 09, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  cc867be1
09 Jun, 2025 3 commits
- [HOT-FIX] Add `kv_sharing_target_layer_name` argument to cutlass_mla backend (#19374) · 8058c911
  Pavani Majety authored Jun 09, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  8058c911
- [Misc] Fixes and Optimizations for DeepEP + DeepGEMM combination. (#19298) · 5cf2daea
  Varun Sundar Rabindranath authored Jun 09, 2025
```
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
```
  5cf2daea
- [full_graph] Fix query_start_loc padding (#19321) · 770e5dcd
  Yinghai Lu authored Jun 09, 2025
```
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
```
  770e5dcd
08 Jun, 2025 2 commits
- [Multi Modal] Add an env var for message queue max chunk bytes (#19242) · cda10fa3
  jennyyyyzhen authored Jun 08, 2025
```
Signed-off-by: yZhen <yZhen@fb.com>
Co-authored-by: yZhen <yZhen@fb.com>
```
  cda10fa3
- [Bugfix]: Fix TypeError: 'float' object cannot be interpreted as an integer (#19283) · d77f7fb8
  Chauncey authored Jun 08, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  d77f7fb8
07 Jun, 2025 2 commits
- [BugFix][V1] Fix memory profiling bug (#18974) · 2d8476e4
  Luka Govedič authored Jun 07, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  2d8476e4
- Add FlexAttention to V1 (#16078) · cf02f9b2
  Driss Guessous authored Jun 07, 2025
```
Signed-off-by: drisspg <drisspguessous@gmail.com>
```
  cf02f9b2
06 Jun, 2025 8 commits
- [BugFix] Fix tpu_model_runner block_id concatenation (#19228) · 46ecc579
  Nick Hill authored Jun 06, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  46ecc579
- [Core] Fix abrupt request abort (#18485) · b6a3a9f7
  Nicolò Lucchesi authored Jun 07, 2025
```
Signed-off-by: nicklucche <nlucches@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  b6a3a9f7
- [Bugfix][Core] Update cancellation logic in `generate()` to handle Generator exits (#19225) · ca27f0f9
  Adolfo Victoria authored Jun 06, 2025
```
Co-authored-by: Adolfo Victoria <adovi@meta.com>
```
  ca27f0f9
- [BugFix] Fix MultiConnector test after HMA changes (#19291) · aad30bd3
  Nick Hill authored Jun 06, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  aad30bd3
- improve logits bias (#19041) · 8267f991
  Yu Guo authored Jun 06, 2025
  
  8267f991
- [Core] Raise when non-multi-instance DP clients target a DP rank (#19227) · 7353492a
  jmswen authored Jun 06, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  7353492a
- [Docs] Improve V1 KVConnector interface documentation (#19172) · 65c69444
  Nick Hill authored Jun 06, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  65c69444
- [TPU] fix kv cache dtype in model runner (#19244) · 0d49483e
  Chengji Yao authored Jun 06, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  0d49483e