Commits · cbb799e314178abf2fa99e7306087198c5f43580 · OpenDAS / vllm_cscc

08 Nov, 2025 1 commit
- [Feat] Drop-in Torch CUDA Profiler (#27841) · 975676d1
  Benjamin Chislett authored Nov 08, 2025
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
  975676d1
07 Nov, 2025 2 commits
- [PerfFix] Avoid separate thread for MP executor shm spin (take 2) (#28319) · 67a2da89
  Nick Hill authored Nov 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  67a2da89
- Revert "[PerfFix] Avoid separate thread for MP executor shm spin (#28012)" (#28289) · 68a72a5c
  Nicolò Lucchesi authored Nov 07, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  68a72a5c
06 Nov, 2025 1 commit
- [Debugging] Add annotation for easier trace analysis (#22496) · 17676585
  Dayeol Lee authored Nov 05, 2025
  
  17676585
05 Nov, 2025 1 commit

[BugFix] Support EP/DP + EPLB with MTP (#25311) · e50c4546

Ilya Markov authored Nov 05, 2025


Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>

e50c4546

04 Nov, 2025 2 commits
- [PerfFix] Avoid separate thread for MP executor shm spin (#28012) · c9f66da8
  Nick Hill authored Nov 04, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  c9f66da8
- [KV Connector] Make KVCacheConfig an explicit constructor argument (#27887) · 58279c60
  Mark McLoughlin authored Nov 04, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  58279c60
01 Nov, 2025 1 commit
- [Core] Async scheduling + structured outputs compatibility (#26866) · 0cdbe7b7
  Nick Hill authored Oct 31, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  0cdbe7b7
31 Oct, 2025 1 commit

[Bugfix] Missing NIXL metadata for handshake initialization if instance spans multi-node (#26338) · d6517be3

GuanLuo authored Nov 01, 2025


Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>

d6517be3

30 Oct, 2025 1 commit

[Misc] Replace CUDA_VISIBLE_DEVICES in DP with torch.cuda.set_device for... · 60f76baa

Ilya Markov authored Oct 30, 2025


[Misc] Replace CUDA_VISIBLE_DEVICES in DP with torch.cuda.set_device for device selection on cuda-like devices (#27564)
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

60f76baa

27 Oct, 2025 1 commit
- [Misc] Clean up more utils (#27567) · 6ebffafb
  Cyrus Leung authored Oct 27, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6ebffafb
25 Oct, 2025 2 commits
- [Core][Hybrid allocator + kv connector 1/n] Enable hybrid allocator + KV cache connector (#25712) · b8535403
  Kuntai Du authored Oct 24, 2025
```
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
```
  b8535403
- Revert "[Misc] Remove use of CUDA_VISIBLE_DEVICES for device selectio… (#27502) · 56ed7609
  Zhuohan Li authored Oct 24, 2025
  
  56ed7609
24 Oct, 2025 1 commit
- [Log] Optimize Startup Log (#26740) · 52efc34e
  Wentao Ye authored Oct 24, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  52efc34e
23 Oct, 2025 1 commit

[Misc] Remove use of CUDA_VISIBLE_DEVICES for device selection (fix DP slow... · 237cf6d3

Ilya Markov authored Oct 23, 2025


[Misc] Remove use of CUDA_VISIBLE_DEVICES for device selection (fix DP slow startup time &c) (#26709)
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

237cf6d3

18 Oct, 2025 1 commit

[Chore] Separate out `vllm.utils.mem_utils` (#27143) · 1d165d6d

iAmir97 authored Oct 18, 2025

Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Signed-off-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

1d165d6d

16 Oct, 2025 1 commit

Deepseek-v3 Batch Invariant on 8xH100 (#26609) · 7d8975de

Bram Wasti authored Oct 15, 2025


Signed-off-by: Bram Wasti <bwasti@meta.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

7d8975de

12 Oct, 2025 2 commits
- Update `Optional[x]` -> `x | None` and `Union[x, y]` to `x | y` (#26633) · 8fcaaf6a
  Harry Mellor authored Oct 12, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8fcaaf6a
- [MISC] Rename the torch profiler filename as instance_id+rank_id for merging... · 76852017
  wang.yuqi authored Oct 12, 2025
```
[MISC] Rename the torch profiler filename as instance_id+rank_id for merging the Profiler results of each Rank (#25867)
Signed-off-by: wang.yuqi <noooop@126.com>
```
  76852017
10 Oct, 2025 1 commit
- [Metrics] Log multi-modal cache stats and fix reset (#26285) · ad430a67
  Cyrus Leung authored Oct 10, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ad430a67
07 Oct, 2025 1 commit
- Optimize KV cache distribution for asymmetric pipeline parallelism (#25164) · d100d78e
  Grant Holmes (Ren) authored Oct 07, 2025
```
Signed-off-by: gholmes829 <g.holmes429@gmail.com>
```
  d100d78e
05 Oct, 2025 1 commit
- Convert formatting to use `ruff` instead of `yapf` + `isort` (#26247) · d6953beb
  Harry Mellor authored Oct 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6953beb
30 Sep, 2025 1 commit
- [V1] [P/D] Add Support for KV Load Failure Recovery (#19330) · 9a9f48df
  David Ben-David authored Oct 01, 2025
```
Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>
```
  9a9f48df
25 Sep, 2025 1 commit
- [Logging] Remove TORCH_NCCL_AVOID_RECORD_STREAMS to squash a warning (#25532) · 2e5df88c
  Tyler Michael Smith authored Sep 25, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  2e5df88c
23 Sep, 2025 4 commits
- [BugFix] Fix OOM in vLLM replicas by ensuring consistent NCCL memory accounting (#25359) · abad204b
  kourosh hakhamaneshi authored Sep 23, 2025
```
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
```
  abad204b
- [Log] Optimize kv cache memory log from Bytes to GiB (#25204) · 846197f5
  Wentao Ye authored Sep 23, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  846197f5
- [UX] Change kv-cache-memory log level to debug (#25479) · 2c58742d
  Michael Goin authored Sep 23, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  2c58742d
- [BugFix] [DP/EP] Fix slow execution when BS <= DP (#25407) · ac0048c0
  Matthew Bonanni authored Sep 22, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Chris Bamford <chrisbam4d@gmail.com>
```
  ac0048c0
20 Sep, 2025 1 commit

[BugFix] Ensure appropriate guards in destructors (#25284) · a25ade5d

Nick Hill authored Sep 19, 2025


Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

a25ade5d

16 Sep, 2025 1 commit
- [Bugfix] Fix sequence parallelism bug when enable pipeline parallelism (#24021) · 17871983
  cascade authored Sep 15, 2025
```
Signed-off-by: cascade812 <cascade812@outlook.com>
```
  17871983
11 Sep, 2025 1 commit

Allow users to specify kv cache memory size (#21489) · 94e6b2d5

Boyuan Feng authored Sep 11, 2025

Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

94e6b2d5

08 Sep, 2025 1 commit
- [P/D] Add a shutdown method to the Connector API (#22699) · 61aa4b29
  Chauncey authored Sep 08, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  61aa4b29
06 Sep, 2025 2 commits

[Feature] Support Decode Context Parallel (DCP) for MLA (#23734) · ac201a0e

yzds authored Sep 06, 2025


Signed-off-by: hongchao <hongchao@msh.team>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: hongchao <hongchao@msh.team>
Co-authored-by: youkaichao <youkaichao@gmail.com>

ac201a0e

[Perf][V1] Fully overlap model execution (#23569) · cee182b2
Benjamin Chislett authored Sep 05, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
cee182b2

03 Sep, 2025 1 commit
- [Kernels] Overlap shared experts with send/recv (#23273) · e9b92dcd
  bnellnm authored Sep 03, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  e9b92dcd
02 Sep, 2025 1 commit
- [Log] Only Print Profiler Results on Rank 0 (#23370) · 98aee612
  Wentao Ye authored Sep 02, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  98aee612
31 Aug, 2025 1 commit
- [Doc]: fix typos in Python comments (#24001) · 9701352e
  Didier Durand authored Aug 31, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
  9701352e
30 Aug, 2025 1 commit
- [LoRA] Much faster startup when LoRA is enabled (#23777) · 038e9be4
  Andy Lo authored Aug 30, 2025
```
Signed-off-by: Andy Lo <andy@mistral.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  038e9be4
29 Aug, 2025 1 commit
- [BugFix] Async scheduling and PP compatibility with DP (#23770) · d90d8eb6
  Nick Hill authored Aug 29, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  d90d8eb6
26 Aug, 2025 1 commit

Update Flashinfer to 0.2.14.post1 (#23537) · ae067888

weiliang authored Aug 26, 2025


Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
Signed-off-by: siyuanf <siyuanf@nvidia.com>
Signed-off-by: Weiliang Liu <weiliangl@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

ae067888