Commits · 7bf6c98f2f1d52175b1da7dc32bef7aedf433e48 · OpenDAS / vllm_cscc · GitLab

12 Jul, 2025 1 commit
- [fix]1.优化mtp代码，解决prefix-caching不兼容问题; 2.修复v1 engine从tokenizer config中读取max_model_len导致长文本输入报错 · b16f183c
  王敏 authored Jul 12, 2025
  
  b16f183c
07 Jul, 2025 4 commits
- 还原误删代码 · f0acced0
  王敏 authored Jul 07, 2025
  
  f0acced0
- 删掉无效代码 · ee9ee960
  王敏 authored Jul 07, 2025
  
  ee9ee960
- [feat]支持v1 engine flashmla和mtp同时开启 · ac4dedb1
  王敏 authored Jul 07, 2025
  
  ac4dedb1
- update vllm_flash_attn_varlen_func · ef6c0877
  zhuwenwen authored Jul 07, 2025
  
  ef6c0877
04 Jul, 2025 1 commit
- update flash_attn.py · 4080ac85
  zhuwenwen authored Jul 04, 2025
  
  4080ac85
30 Jun, 2025 1 commit
- [fix]解决v1 engine mtp只能设为1的问题 · 504c262e
  王敏 authored Jun 30, 2025
  
  504c262e
23 Jun, 2025 1 commit
- update common.py · 2da6fd72
  zhuwenwen authored Jun 23, 2025
  
  2da6fd72
10 Jun, 2025 6 commits
- [BugFix][FlashInfer] Fix attention backend interface mismatch with unexpected... · 467bef18
  Rachel Guo authored Jun 10, 2025
```
[BugFix][FlashInfer] Fix attention backend interface mismatch with unexpected keyword `use_irope` (#19134)
Signed-off-by: Yunqiu Guo <guorachel@meta.com>
```
  467bef18
- Automatically bind CPU OMP Threads of a rank to CPU ids of a NUMA node. (#17930) · 9368cc90
  Louie Tsai authored Jun 09, 2025
```
Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Co-authored-by: Li, Jiang <bigpyj64@gmail.com>
```
  9368cc90
- [Core] Batch multi modal input using pinned memory (#19169) · 319cb1e3
  Lukas Geiger authored Jun 10, 2025
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
```
  319cb1e3
- [Core] Use tuple for kv cache group block ids (#19175) · 646d62f6
  Nick Hill authored Jun 09, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  646d62f6
- Use xla flag to improve the quantized model performance (#19303) · 9af6d22e
  XiongfeiWei authored Jun 09, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  9af6d22e
- [V1] Reuse V0's memory_profiling util for gpu worker memory profiling (#19312) · cc867be1
  Ye (Charlotte) Qi authored Jun 09, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  cc867be1
09 Jun, 2025 3 commits
- [HOT-FIX] Add `kv_sharing_target_layer_name` argument to cutlass_mla backend (#19374) · 8058c911
  Pavani Majety authored Jun 09, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  8058c911
- [Misc] Fixes and Optimizations for DeepEP + DeepGEMM combination. (#19298) · 5cf2daea
  Varun Sundar Rabindranath authored Jun 09, 2025
```
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
```
  5cf2daea
- [full_graph] Fix query_start_loc padding (#19321) · 770e5dcd
  Yinghai Lu authored Jun 09, 2025
```
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
```
  770e5dcd
08 Jun, 2025 2 commits
- [Multi Modal] Add an env var for message queue max chunk bytes (#19242) · cda10fa3
  jennyyyyzhen authored Jun 08, 2025
```
Signed-off-by: yZhen <yZhen@fb.com>
Co-authored-by: yZhen <yZhen@fb.com>
```
  cda10fa3
- [Bugfix]: Fix TypeError: 'float' object cannot be interpreted as an integer (#19283) · d77f7fb8
  Chauncey authored Jun 08, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  d77f7fb8
07 Jun, 2025 2 commits
- [BugFix][V1] Fix memory profiling bug (#18974) · 2d8476e4
  Luka Govedič authored Jun 07, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  2d8476e4
- Add FlexAttention to V1 (#16078) · cf02f9b2
  Driss Guessous authored Jun 07, 2025
```
Signed-off-by: drisspg <drisspguessous@gmail.com>
```
  cf02f9b2
06 Jun, 2025 13 commits
- [BugFix] Fix tpu_model_runner block_id concatenation (#19228) · 46ecc579
  Nick Hill authored Jun 06, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  46ecc579
- [Core] Fix abrupt request abort (#18485) · b6a3a9f7
  Nicolò Lucchesi authored Jun 07, 2025
```
Signed-off-by: nicklucche <nlucches@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  b6a3a9f7
- [Bugfix][Core] Update cancellation logic in `generate()` to handle Generator exits (#19225) · ca27f0f9
  Adolfo Victoria authored Jun 06, 2025
```
Co-authored-by: Adolfo Victoria <adovi@meta.com>
```
  ca27f0f9
- [BugFix] Fix MultiConnector test after HMA changes (#19291) · aad30bd3
  Nick Hill authored Jun 06, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  aad30bd3
- improve logits bias (#19041) · 8267f991
  Yu Guo authored Jun 06, 2025
  
  8267f991
- [Core] Raise when non-multi-instance DP clients target a DP rank (#19227) · 7353492a
  jmswen authored Jun 06, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  7353492a
- [Docs] Improve V1 KVConnector interface documentation (#19172) · 65c69444
  Nick Hill authored Jun 06, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  65c69444
- [TPU] fix kv cache dtype in model runner (#19244) · 0d49483e
  Chengji Yao authored Jun 06, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  0d49483e
- [Model] Add VLLM_USE_NN to use nn layout · 6880bf15
  zhuwenwen authored Jun 06, 2025
  
  6880bf15
- [v1][P/D] Fix a edge case in kv cache schedule (#19182) · 90b78ec5
  Jinghui Zhang authored Jun 05, 2025
```
Co-authored-by: jinghui <jinghui@fb.com>
```
  90b78ec5
- [TPU] update torch_xla pin (#19231) · b61dc5f9
  Chengji Yao authored Jun 05, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  b61dc5f9
- [v1] Hybrid Memory Allocator (#17996) · f8a1a2d1
  Chen Zhang authored Jun 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  f8a1a2d1
- [Bugfix] Fix EAGLE vocab embedding construction for Llama 70B (#19033) · 3465b87e
  Benjamin Chislett authored Jun 05, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
  3465b87e
05 Jun, 2025 3 commits
- [Misc] Remove unnecessary fallback to prefill-decode attention (#19138) · 18093084
  vllmellm authored Jun 05, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  18093084
- [BugFix][Minor] Fix full cuda graph bug when max_num_seqs < 512 (#19171) · af7fc84f
  Woosuk Kwon authored Jun 04, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  af7fc84f
- [Bugfix][Nixl] Fix full prefix cache hit bug (#18632) · c56ed8bb
  Robert Shaw authored Jun 04, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  c56ed8bb
04 Jun, 2025 3 commits
- [P/D] Heterogeneous TP (#18833) · b2fac671
  Nicolò Lucchesi authored Jun 05, 2025
```
Signed-off-by: nicklucche <nlucches@redhat.com>
```
  b2fac671
- [Misc] refactor: simplify EngineCoreClient.make_async_mp_client in AsyncLLM (#18817) · 23027e2d
  CYJiang authored Jun 05, 2025
```
Signed-off-by: googs1025 <googs1025@gmail.com>
```
  23027e2d
- Allow AsyncLLMEngine.generate to target a specific DP rank (#19102) · c8dcc159
  jmswen authored Jun 04, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  c8dcc159