Commits · d4d93db2c54ad989ea800004f491c8d116017a4c · OpenDAS / vllm_cscc

".github/vscode:/vscode.git/clone" did not exist on "c5041f899f586644326dc67b21d7de50c62eb410"

15 Mar, 2025 2 commits

[V1] V1 Enablement Oracle (#13726) · d4d93db2

Robert Shaw authored Mar 15, 2025


Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

d4d93db2

[Bugfix][V1] Fix flashinfer sampling (#14815) · acaea3bb
DefTruth authored Mar 15, 2025

acaea3bb

14 Mar, 2025 5 commits
- [V1] Fix vocab size calculation for structured output (#14826) · 1140991a
  Russell Bryant authored Mar 14, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  1140991a
- [V1][Minor] Minor code cleanup for scheduling metrics (#14800) · c77620d2
  Woosuk Kwon authored Mar 14, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  c77620d2
- [Attention] MLA get rid of materialization (#14770) · 9532c498
  Lucas Wilkinson authored Mar 14, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  9532c498
- [V1] Temporarily disable FlashInfer Rejection Sampler (#14788) · 32ef4983
  Woosuk Kwon authored Mar 13, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  32ef4983
- [V1] Move OOM check into sampler run (#14728) · ad19c8a0
  Roger Wang authored Mar 13, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  ad19c8a0
13 Mar, 2025 4 commits
- [V1] Detokenizer: Respect Stop Tokens + not include_stop_str_in_output (#14624) · 02fcaa3d
  afeldman-nm authored Mar 13, 2025
```
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
```
  02fcaa3d
- [V1][Core] using cached vocab_size for Structured Outputs (#14630) · 8a4a2efc
  Aaron Pham authored Mar 13, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  8a4a2efc
- [V1][Minor] Minor enhancements on scheduler (#14732) · 01b3fd0a
  Woosuk Kwon authored Mar 13, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  01b3fd0a
- [bugfix] fixup warning message for plugged schedulers for v1 (#14700) · 165290d3
  Joe Runde authored Mar 12, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  165290d3
12 Mar, 2025 4 commits
- [BugFix][V1] Fix parallel sampling finishing/aborts (#14512) · f5d3acd4
  Nick Hill authored Mar 12, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  f5d3acd4
- [V1][Bugfix][Spec Decode] Fix incorrect outputs in V1 speculative decoding due... · 5c538c37
  Benjamin Chislett authored Mar 12, 2025
```
[V1][Bugfix][Spec Decode] Fix incorrect outputs in V1 speculative decoding due to batch indexing (#14645)
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
  5c538c37
- [V1][Core] Support MistralTokenizer for Structured Output (#14625) · 77a318bd
  Aaron Pham authored Mar 11, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  77a318bd
- [core][V1] pluggable scheduler (#14466) · 47532cd9
  Joe Runde authored Mar 11, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  47532cd9
11 Mar, 2025 8 commits
- [Bugfix][V1][PP] Only warmup sampler at last PP rank (#14643) · b706d898
  Cody Yu authored Mar 11, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  b706d898
- [V1][TPU] Pad the block_table.shape[1] so the ragged paged attention can handle correctly (#14597) · 863d315c
  iefgnoix authored Mar 11, 2025
  
  863d315c
- [V1] Delay all xgrammar usage until needed (#14616) · 61a01b27
  Russell Bryant authored Mar 11, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  61a01b27
- [V1] Remove cache from StructuredOutputManager (#14622) · 4cbf2867
  Russell Bryant authored Mar 11, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  4cbf2867
- [V1] Add regex structured output support with xgrammar (#14590) · 4bf82d4b
  Russell Bryant authored Mar 11, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  4bf82d4b
- dynamic distpatch of fp8 kernels (#14245) · a1c8f379
  Jeff Daily authored Mar 11, 2025
```
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
```
  a1c8f379
- [V1][Core] Fix memory issue with logits & sampling (#14508) · 1fc973c0
  Roger Wang authored Mar 10, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Varun Sundar Rabindranath <3337719+varun-sundar-rabindranath@users.noreply.github.com>
```
  1fc973c0
- [V1][PP] Do not block engine core when no requests to schedule (#14585) · 4290b704
  Cody Yu authored Mar 10, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  4290b704
10 Mar, 2025 4 commits
- [V1] Prevent xgrammar from breaking TPU support (#14575) · 04421dff
  Russell Bryant authored Mar 10, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  04421dff
- [V1] LoRA - Add triton kernels for V1 (#13096) · 5ff0d325
  Varun Sundar Rabindranath authored Mar 10, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  5ff0d325
- [Perf] Improve MLA on V1 (#14540) · fb0acb6c
  Simon Mo authored Mar 10, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  fb0acb6c
- Correct capitalisation: `VLLM` -> `vLLM` (#14562) · 3b352a2f
  Harry Mellor authored Mar 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  3b352a2f
09 Mar, 2025 3 commits
- [Hardware][TPU] Fix the recompiling issue in logits processor after warmup (#14510) · 212007b1
  Chengji Yao authored Mar 09, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  212007b1
- [V1][TPU] Remove unnecessary padding for running on TPU. (#14467) · 10f75527
  iefgnoix authored Mar 08, 2025
  
  10f75527
- Revert "[V1][Core] Fix memory issue with logits & sampling" (#14504) · 5f0b53c6
  Robert Shaw authored Mar 08, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  5f0b53c6
08 Mar, 2025 8 commits
- [V1] Support bad_words in sampler (#13376) · eb8b5eb1
  22quinn authored Mar 08, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  eb8b5eb1
- [Bugfix] DeepSeek Accuracy (#14476) · db84f5eb
  Lucas Wilkinson authored Mar 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  db84f5eb
- [V1][Core] Fix memory issue with logits & sampling (#13776) · 8d5aa466
  Roger Wang authored Mar 08, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  8d5aa466
- [V1] TPU - Add tensor parallel support via Ray (#13618) · cb8bdfad
  Alexander Matveev authored Mar 08, 2025
```
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
```
  cb8bdfad
- [MISC][V1] Register process killing handler only in the main thread (#14380) · 9f3bc0f5
  Cody Yu authored Mar 07, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  9f3bc0f5
- Revert "[Perf] Reduce MLA CPU overheads in V1 (#14384)" (#14471) · ca7a2d5f
  Tyler Michael Smith authored Mar 08, 2025
  
  ca7a2d5f
- [Bugfix][V1] Handle MLA in kv_cache_interface (#14462) · 33368140
  Tyler Michael Smith authored Mar 08, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  33368140
- [V1] Prompt logprobs + APC compatibility; prompt logprobs reqs cannot fill APC (#13949) · ef640440
  afeldman-nm authored Mar 07, 2025
  
  ef640440
07 Mar, 2025 2 commits
- [Bugfix] Fix torch_xla which can't handle None seed introduced in #14274 (#14459) · 66e16a03
  yarongmu-google authored Mar 07, 2025
```
Signed-off-by: Yarong Mu <ymu@google.com>
```
  66e16a03
- [V1][Metrics] Fix traceback with preemptions+LoRA (#14220) · e1f0835a
  Mark McLoughlin authored Mar 07, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  e1f0835a