Commits · e9fd658a736a4d30f7a367c317506c87ad7f5359 · OpenDAS / vllm_cscc

26 Jun, 2025 3 commits
- [Feature] Expert Parallelism Load Balancer (EPLB) (#18343) · e9fd658a
  Bowen Wang authored Jun 26, 2025
```
Signed-off-by: Bowen Wang <abmfy@icloud.com>
```
  e9fd658a
- [TPU] add kv cache update kernel (#19928) · 04e1642e
  Chengji Yao authored Jun 26, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  04e1642e
- [Hardware][Intel GPU] Add v1 Intel GPU support with Flash attention backend. (#19560) · b69781f1
  Kunshang Ji authored Jun 27, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  b69781f1
25 Jun, 2025 2 commits
- [TPU] Add TPU specific var VLLM_TPU_MOST_MODEL_LEN (#19919) · 2d7620c3
  Chenyaaang authored Jun 25, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  2d7620c3
- [TPU][Bugfix] fix kv cache padding (#20048) · 2cc20699
  Chengji Yao authored Jun 25, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  2cc20699
24 Jun, 2025 2 commits
- [Bugfix][CPU] Fix InputBatch for pooling models in the CPU v1 (#20014) · 53da4cd3
  Li, Jiang authored Jun 24, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  53da4cd3
- [PERF] Speedup of MRoPE prepare inputs (#19939) · 9a3b8832
  Vadim Gimpelson authored Jun 24, 2025
```
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@centml.ai>
```
  9a3b8832
23 Jun, 2025 1 commit
- [Bugfix][v1] Fix step pooler implementation and step pooling usage in v1 (#19956) · 61f4fc5d
  Isotr0py authored Jun 24, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  61f4fc5d
20 Jun, 2025 1 commit
- Export NaNs in logits to scheduler_stats if output is corrupted (#18777) · 2e3e3c86
  Vlad Tiberiu Mihailescu authored Jun 20, 2025
```
Signed-off-by: Vlad Mihailescu <vtmihailescu@gmail.com>
```
  2e3e3c86
19 Jun, 2025 2 commits

Support embedding models in V1 (#16188) · 799397ee

Maximilien de Bayser authored Jun 19, 2025


Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>

799397ee

[BugFix] Fix use_cudagraph=False (#19612) · ed333497
Richard Zou authored Jun 18, 2025
```
Signed-off-by: Richard Zou <zou3519@gmail.com>
```
ed333497

18 Jun, 2025 2 commits
- [v1] Support mamba2 (#19327) · a89209b7
  Chen Zhang authored Jun 19, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  a89209b7
- [V1] Decouple GPU and TPU `InputBatch` (#19778) · 19a53b27
  afeldman-nm authored Jun 18, 2025
```
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
```
  19a53b27
16 Jun, 2025 2 commits
- [V1] Change return type on get_multimodal_embeddings() (#19446) · 90f9c2eb
  Russell Bryant authored Jun 16, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  90f9c2eb
- [Bugfix] Fix TP inference for Flex attention backend (#19657) · 1173804d
  Isotr0py authored Jun 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  1173804d
13 Jun, 2025 3 commits
- [CUDA] Enable full cudagraph for FlashMLA (#18581) · 3597b06a
  Luka Govedič authored Jun 13, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  3597b06a
- [Fix] The zip function in Python 3.9 does not have the strict argument (#19549) · cefdb996
  汪志鹏 authored Jun 13, 2025
```
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
```
  cefdb996
- [CPU] Refine default config for the CPU backend (#19539) · 64587211
  Li, Jiang authored Jun 13, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  64587211
12 Jun, 2025 1 commit
- [V1] Resolve failed concurrent structured output requests (#19565) · c57bb199
  Russell Bryant authored Jun 12, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  c57bb199
11 Jun, 2025 1 commit
- [UX] Add Feedback During CUDAGraph Capture (#19501) · 97a9465b
  Robert Shaw authored Jun 11, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  97a9465b
10 Jun, 2025 5 commits
- Automatically bind CPU OMP Threads of a rank to CPU ids of a NUMA node. (#17930) · 9368cc90
  Louie Tsai authored Jun 09, 2025
```
Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Co-authored-by: Li, Jiang <bigpyj64@gmail.com>
```
  9368cc90
- [Core] Batch multi modal input using pinned memory (#19169) · 319cb1e3
  Lukas Geiger authored Jun 10, 2025
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
```
  319cb1e3
- [Core] Use tuple for kv cache group block ids (#19175) · 646d62f6
  Nick Hill authored Jun 09, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  646d62f6
- Use xla flag to improve the quantized model performance (#19303) · 9af6d22e
  XiongfeiWei authored Jun 09, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  9af6d22e
- [V1] Reuse V0's memory_profiling util for gpu worker memory profiling (#19312) · cc867be1
  Ye (Charlotte) Qi authored Jun 09, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  cc867be1
09 Jun, 2025 2 commits
- [Misc] Fixes and Optimizations for DeepEP + DeepGEMM combination. (#19298) · 5cf2daea
  Varun Sundar Rabindranath authored Jun 09, 2025
```
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
```
  5cf2daea
- [full_graph] Fix query_start_loc padding (#19321) · 770e5dcd
  Yinghai Lu authored Jun 09, 2025
```
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
```
  770e5dcd
07 Jun, 2025 1 commit
- [BugFix][V1] Fix memory profiling bug (#18974) · 2d8476e4
  Luka Govedič authored Jun 07, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  2d8476e4
06 Jun, 2025 4 commits
- [BugFix] Fix tpu_model_runner block_id concatenation (#19228) · 46ecc579
  Nick Hill authored Jun 06, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  46ecc579
- [TPU] fix kv cache dtype in model runner (#19244) · 0d49483e
  Chengji Yao authored Jun 06, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  0d49483e
- [TPU] update torch_xla pin (#19231) · b61dc5f9
  Chengji Yao authored Jun 05, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  b61dc5f9
- [v1] Hybrid Memory Allocator (#17996) · f8a1a2d1
  Chen Zhang authored Jun 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  f8a1a2d1
05 Jun, 2025 1 commit
- [BugFix][Minor] Fix full cuda graph bug when max_num_seqs < 512 (#19171) · af7fc84f
  Woosuk Kwon authored Jun 04, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  af7fc84f
04 Jun, 2025 3 commits
- [Bugfix] Fix FA3 full cuda graph correctness (#19106) · b124e108
  Woosuk Kwon authored Jun 03, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b124e108
- [Core] Cast multimodal input in hf processor (#18862) · 1409ef91
  Lukas Geiger authored Jun 04, 2025
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
```
  1409ef91
- [CPU] V1 support for the CPU backend (#16441) · 4555143e
  Li, Jiang authored Jun 04, 2025
  
  4555143e
03 Jun, 2025 4 commits
- [v1] Re-init input batch for multiple kv cache groups (#18654) · 6cac54f4
  Chen Zhang authored Jun 04, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  6cac54f4
- [V1] Support cross-layer KV sharing (#18212) · bdf13965
  Yong Hoon Shin authored Jun 03, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  bdf13965
- [Misc] Add SPDX-FileCopyrightText (#19100) · 02f0c7b2
  Simon Mo authored Jun 03, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  02f0c7b2
- [Hardware][TPU] Initial support of model parallelism with single worker using SPMD (#18011) · 9112b443
  Siyuan Liu authored Jun 02, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Co-authored-by: Hossein Sarshar <hossein.sarshar@gmail.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
```
  9112b443