Commits · ca796e19528384abba8732bb9fd204273a728d11 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "2039c1c6510fe12cf9143d992c3c811f53855943"

19 Mar, 2025 1 commit
- [V1] Ensure using int64 for sampled token ids (#15065) · 05ccd0aa
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  05ccd0aa
18 Mar, 2025 6 commits
- fix long dtype in topk sampling (#15049) · 027827cc
  Chujie Zheng authored Mar 19, 2025
  
  027827cc
- [V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels (#14930) · 99abb8b6
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  99abb8b6
- [V1] Refactor Structured Output for multiple backends (#14694) · 3a1e6481
  Russell Bryant authored Mar 18, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  3a1e6481
- [Kernels] LoRA - Retire SGMV and BGMV Kernels (#14685) · 9e8f089d
  Varun Sundar Rabindranath authored Mar 18, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  9e8f089d
- [TPU][V1][Bugfix] Fix chunked prefill with padding (#15037) · af35d3a3
  Nicolò Lucchesi authored Mar 18, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  af35d3a3
- [Kernels] LoRA - Retire SGMV and BGMV Kernels (#14685) · 400d483e
  Varun Sundar Rabindranath authored Mar 18, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  400d483e
17 Mar, 2025 6 commits

[Fix][Structured Output] using vocab_size to construct matcher (#14868) · c0efdd65

Aaron Pham authored Mar 17, 2025

Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

c0efdd65

[V1][TPU] Apply the ragged paged attention kernel fix and remove the padding. (#14846) · b4ad56c1
iefgnoix authored Mar 17, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
b4ad56c1

[V1] Remove input cache client (#14864) · b539222d

Cyrus Leung authored Mar 17, 2025


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

b539222d

[V1] [Spec Decode] Support random sampling for spec decode (#13933) · 8d6cf895
Lily Liu authored Mar 16, 2025
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
8d6cf895
[V1][Minor] Add __repr__ to ConstantList (#14907) · 7f6c5ee0
Woosuk Kwon authored Mar 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
7f6c5ee0
[V1] Optimize the overhead of rewinding (#14905) · faa02757
Woosuk Kwon authored Mar 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
faa02757

16 Mar, 2025 2 commits
- [V1][BugFix] Detect interleaved sliding window attention (#14896) · 31060b27
  Woosuk Kwon authored Mar 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  31060b27
- [BugFix][V1] Fix overhead related to bad_words sampling when not in use (#14894) · fc1f6771
  Nick Hill authored Mar 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  fc1f6771
15 Mar, 2025 5 commits
- [Core] Expose API endpoint `/is_sleeping` (#14312) · 74bc397b
  Jun Duan authored Mar 15, 2025
```
Signed-off-by: Jun Duan <jun.duan.phd@outlook.com>
```
  74bc397b
- [VLM] Limit multimodal input cache by memory (#14805) · 3556a414
  Cyrus Leung authored Mar 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3556a414
- [V1][Structured Output] calculate vocab_size eagerly (#14851) · 4c7629ca
  Aaron Pham authored Mar 15, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  4c7629ca
- [V1] V1 Enablement Oracle (#13726) · d4d93db2
  Robert Shaw authored Mar 15, 2025
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  d4d93db2
- [Bugfix][V1] Fix flashinfer sampling (#14815) · acaea3bb
  DefTruth authored Mar 15, 2025
  
  acaea3bb
14 Mar, 2025 5 commits
- [V1] Fix vocab size calculation for structured output (#14826) · 1140991a
  Russell Bryant authored Mar 14, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  1140991a
- [V1][Minor] Minor code cleanup for scheduling metrics (#14800) · c77620d2
  Woosuk Kwon authored Mar 14, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  c77620d2
- [Attention] MLA get rid of materialization (#14770) · 9532c498
  Lucas Wilkinson authored Mar 14, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  9532c498
- [V1] Temporarily disable FlashInfer Rejection Sampler (#14788) · 32ef4983
  Woosuk Kwon authored Mar 13, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  32ef4983
- [V1] Move OOM check into sampler run (#14728) · ad19c8a0
  Roger Wang authored Mar 13, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  ad19c8a0
13 Mar, 2025 4 commits
- [V1] Detokenizer: Respect Stop Tokens + not include_stop_str_in_output (#14624) · 02fcaa3d
  afeldman-nm authored Mar 13, 2025
```
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
```
  02fcaa3d
- [V1][Core] using cached vocab_size for Structured Outputs (#14630) · 8a4a2efc
  Aaron Pham authored Mar 13, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  8a4a2efc
- [V1][Minor] Minor enhancements on scheduler (#14732) · 01b3fd0a
  Woosuk Kwon authored Mar 13, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  01b3fd0a
- [bugfix] fixup warning message for plugged schedulers for v1 (#14700) · 165290d3
  Joe Runde authored Mar 12, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  165290d3
12 Mar, 2025 4 commits
- [BugFix][V1] Fix parallel sampling finishing/aborts (#14512) · f5d3acd4
  Nick Hill authored Mar 12, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  f5d3acd4
- [V1][Bugfix][Spec Decode] Fix incorrect outputs in V1 speculative decoding due... · 5c538c37
  Benjamin Chislett authored Mar 12, 2025
```
[V1][Bugfix][Spec Decode] Fix incorrect outputs in V1 speculative decoding due to batch indexing (#14645)
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
  5c538c37
- [V1][Core] Support MistralTokenizer for Structured Output (#14625) · 77a318bd
  Aaron Pham authored Mar 11, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  77a318bd
- [core][V1] pluggable scheduler (#14466) · 47532cd9
  Joe Runde authored Mar 11, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  47532cd9
11 Mar, 2025 7 commits
- [Bugfix][V1][PP] Only warmup sampler at last PP rank (#14643) · b706d898
  Cody Yu authored Mar 11, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  b706d898
- [V1][TPU] Pad the block_table.shape[1] so the ragged paged attention can handle correctly (#14597) · 863d315c
  iefgnoix authored Mar 11, 2025
  
  863d315c
- [V1] Delay all xgrammar usage until needed (#14616) · 61a01b27
  Russell Bryant authored Mar 11, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  61a01b27
- [V1] Remove cache from StructuredOutputManager (#14622) · 4cbf2867
  Russell Bryant authored Mar 11, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  4cbf2867
- [V1] Add regex structured output support with xgrammar (#14590) · 4bf82d4b
  Russell Bryant authored Mar 11, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  4bf82d4b
- dynamic distpatch of fp8 kernels (#14245) · a1c8f379
  Jeff Daily authored Mar 11, 2025
```
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
```
  a1c8f379
- [V1][Core] Fix memory issue with logits & sampling (#14508) · 1fc973c0
  Roger Wang authored Mar 10, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Varun Sundar Rabindranath <3337719+varun-sundar-rabindranath@users.noreply.github.com>
```
  1fc973c0