Commits · 3be8d312a216727dd3624afa92984dab6e170274 · OpenDAS / vllm_cscc

01 Jul, 2025 1 commit
- [Optimization] Cache sampled token ids in model runner (#20291) · 7f280d69
  Woosuk Kwon authored Jul 01, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7f280d69
30 Jun, 2025 1 commit
- [Optimization] Use Shared `CachedRequestData` Instance Across All Requests (#20232) · 2863befc
  Woosuk Kwon authored Jun 30, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2863befc
26 Jun, 2025 3 commits
- [TPU] add kv cache update kernel (#19928) · 04e1642e
  Chengji Yao authored Jun 26, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  04e1642e
- [Bugfix] Allow `CUDA_VISIBLE_DEVICES=''` in `Platform.device_id_to_physical_device_id` (#18979) · 65397e40
  Seiji Eicher authored Jun 26, 2025
```
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
```
  65397e40
- [PD] Skip `tp_size` exchange with rank0 (#19413) · 25826835
  Nicolò Lucchesi authored Jun 26, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  25826835
25 Jun, 2025 2 commits
- [TPU] Add TPU specific var VLLM_TPU_MOST_MODEL_LEN (#19919) · 2d7620c3
  Chenyaaang authored Jun 25, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  2d7620c3
- [PD] let toy proxy handle /chat/completions (#19730) · 4734704b
  lkchen authored Jun 25, 2025
```
Signed-off-by: Linkun <github@lkchen.net>
```
  4734704b
24 Jun, 2025 2 commits

[P/D] Asynchronously do _nixl_handshake (#19836) · 91f7d9d0

lkchen authored Jun 24, 2025


Signed-off-by: Linkun Chen <github@lkchen.net>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

91f7d9d0

[Fix][V1] Remove --scheduling-policy oracle (#20010) · 981eeca4
amit authored Jun 24, 2025
```
Signed-off-by: amit <amit.man@gmail.com>
```
981eeca4

23 Jun, 2025 3 commits
- [TPU] Fix tpu model runner test (#19995) · 33d5e29b
  Chenyaaang authored Jun 23, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  33d5e29b
- [BugFix][P/D] Fix for cases where _recving_transfers can be cleaned up when... · 1bcd15ed
  lkchen authored Jun 22, 2025
```
[BugFix][P/D] Fix for cases where _recving_transfers can be cleaned up when *all* transfer done (#19874)
Signed-off-by: Linkun Chen <github@lkchen.net>
```
  1bcd15ed
- [Core] feat: Implement Priority Scheduling in V1 Engine (#19057) · 4a0f7888
  amit authored Jun 23, 2025
```
Signed-off-by: amit <amit.man@gmail.com>
Co-authored-by: Roger Wang <Rogerw0108@gmail.com>
```
  4a0f7888
20 Jun, 2025 2 commits
- Export NaNs in logits to scheduler_stats if output is corrupted (#18777) · 2e3e3c86
  Vlad Tiberiu Mihailescu authored Jun 20, 2025
```
Signed-off-by: Vlad Mihailescu <vtmihailescu@gmail.com>
```
  2e3e3c86
- [CI/Build][Bugfix] Fix deadlock on v1 engine test CI (#19872) · ee9a1531
  Isotr0py authored Jun 20, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  ee9a1531
19 Jun, 2025 2 commits

[Bugfix] Add check_health to v1 async client. (#19821) · e2148dc5
kourosh hakhamaneshi authored Jun 18, 2025
```
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
```
e2148dc5

Support embedding models in V1 (#16188) · 799397ee

Maximilien de Bayser authored Jun 19, 2025


Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>

799397ee

18 Jun, 2025 2 commits
- [v1] Support mamba2 (#19327) · a89209b7
  Chen Zhang authored Jun 19, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  a89209b7
- [Misc] Add __str__ for RequestStatus (#19780) · d4629dc4
  lkchen authored Jun 17, 2025
```
Signed-off-by: Linkun Chen <github@lkchen.net>
```
  d4629dc4
16 Jun, 2025 2 commits
- [Bugfix] Fix TP inference for Flex attention backend (#19657) · 1173804d
  Isotr0py authored Jun 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  1173804d
- [TPU] support attention head dim smaller than 128 (#19620) · a77aea59
  Chengji Yao authored Jun 15, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  a77aea59
14 Jun, 2025 1 commit
- [Bugfix] Fix auto dtype casting for BatchFeature (#19316) · 2db9044a
  Isotr0py authored Jun 14, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  2db9044a
12 Jun, 2025 3 commits
- [Bugfix] Respect num-gpu-blocks-override in v1 (#19503) · c9280e63
  jmswen authored Jun 12, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  c9280e63
- [BugFix] Work-around incremental detokenization edge case error (#19449) · d5bdf899
  Nick Hill authored Jun 11, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  d5bdf899
- [CI] change spell checker from codespell to typos (#18711) · 2f1c19b2
  Ning Xie authored Jun 12, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  2f1c19b2
11 Jun, 2025 1 commit
- Fix Typo in Documentation and Function Name (#19442) · 7c644ab6
  leopardracer authored Jun 11, 2025
  
  7c644ab6
10 Jun, 2025 2 commits
- Revert "[v1] Add fp32 support to v1 engine through flex attn" (#19404) · 5f1ac1e1
  Isotr0py authored Jun 10, 2025
  
  5f1ac1e1
- [Core] Use tuple for kv cache group block ids (#19175) · 646d62f6
  Nick Hill authored Jun 09, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  646d62f6
09 Jun, 2025 2 commits
- [TPU]Fix KV cache sharing tests (#19371) · 7d44c469
  Siyuan Liu authored Jun 09, 2025
  
  7d44c469
- [v1] Add fp32 support to v1 engine through flex attn (#19319) · b8089195
  Isotr0py authored Jun 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  b8089195
07 Jun, 2025 1 commit
- [BugFix][V1] Fix memory profiling bug (#18974) · 2d8476e4
  Luka Govedič authored Jun 07, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  2d8476e4
06 Jun, 2025 6 commits
- [BugFix] Fix tpu_model_runner block_id concatenation (#19228) · 46ecc579
  Nick Hill authored Jun 06, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  46ecc579
- [Bugfix][Core] Update cancellation logic in `generate()` to handle Generator exits (#19225) · ca27f0f9
  Adolfo Victoria authored Jun 06, 2025
```
Co-authored-by: Adolfo Victoria <adovi@meta.com>
```
  ca27f0f9
- [BugFix] Fix MultiConnector test after HMA changes (#19291) · aad30bd3
  Nick Hill authored Jun 06, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  aad30bd3
- [Core] Raise when non-multi-instance DP clients target a DP rank (#19227) · 7353492a
  jmswen authored Jun 06, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  7353492a
- [v1] Hybrid Memory Allocator (#17996) · f8a1a2d1
  Chen Zhang authored Jun 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  f8a1a2d1
- [Bugfix] Fix EAGLE vocab embedding construction for Llama 70B (#19033) · 3465b87e
  Benjamin Chislett authored Jun 05, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
  3465b87e
05 Jun, 2025 1 commit

[Bugfix][Nixl] Fix full prefix cache hit bug (#18632) · c56ed8bb

Robert Shaw authored Jun 04, 2025


Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

c56ed8bb

04 Jun, 2025 3 commits
- [P/D] Heterogeneous TP (#18833) · b2fac671
  Nicolò Lucchesi authored Jun 05, 2025
```
Signed-off-by: nicklucche <nlucches@redhat.com>
```
  b2fac671
- [TPU] Update dynamo dump file name in compilation test (#19108) · 7ee25904
  Siyuan Liu authored Jun 04, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  7ee25904
- Allow AsyncLLMEngine.generate to target a specific DP rank (#19102) · c8dcc159
  jmswen authored Jun 04, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  c8dcc159