Commits · 04437e313dbbf4427734a3ed2d1d650efc57ef66 · OpenDAS / vllm_cscc

28 Mar, 2025 4 commits
- [V1] Support disable_any_whtespace for guidance backend (#15584) · 7329ff54
  Russell Bryant authored Mar 28, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  7329ff54
- [Bugfix][v1] xgrammar structured output supports Enum. (#15594) · 3b00ff91
  Chauncey authored Mar 28, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  3b00ff91
- [TPU][CI] Fix TPUModelRunner Test (#15667) · 2d9045fc
  Robert Shaw authored Mar 28, 2025
```
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
```
  2d9045fc
- [CI][TPU] Temporarily Disable Quant Test on TPU (#15649) · 8a49eea7
  Robert Shaw authored Mar 27, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  8a49eea7
27 Mar, 2025 3 commits
- [V1] AsyncLLM data parallel (#13923) · 15dac210
  Nick Hill authored Mar 27, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  15dac210
- [Bugfix][TPU][V1] Fix recompilation (#15553) · 4098b722
  Nicolò Lucchesi authored Mar 27, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  4098b722
- [V1] Refactor num_computed_tokens logic (#15307) · 54aa6194
  Cody Yu authored Mar 26, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  54aa6194
26 Mar, 2025 2 commits
- Support SHA256 as hash function in prefix caching (#15297) · 27df5199
  marko authored Mar 26, 2025
```
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
```
  27df5199
- [V1][Sampler] Faster top-k only implementation (#15478) · 35fad35a
  Nick Hill authored Mar 26, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  35fad35a
25 Mar, 2025 5 commits
- [core] add bucket padding to tpu_model_runner (#14995) · ac3cd6e8
  Chenyaaang authored Mar 25, 2025
```
Signed-off-by: Chenyaaang <llccyy1212@gmail.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  ac3cd6e8
- [V1] Support long_prefill_token_threshold in v1 scheduler (#15419) · 082ab86f
  Lu Fang authored Mar 25, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  082ab86f
- [CI/Build] Add tests for the V1 tpu_model_runner. (#14843) · 0a049c7d
  yarongmu-google authored Mar 25, 2025
```
Signed-off-by: Yarong Mu <ymu@google.com>
```
  0a049c7d
- [V1] guidance backend for structured output + `auto` fallback mode (#14779) · a09ad90a
  Russell Bryant authored Mar 25, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <jc1da.3011@gmail.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
```
  a09ad90a
- [V1][Spec Decode] Enable spec decode for top-p & top-k sampling (#15063) · ebcebeeb
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  ebcebeeb
24 Mar, 2025 1 commit

[V1][Perf] Simpler request output queues (#15156) · 9d72daf4

Nick Hill authored Mar 24, 2025


Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>

9d72daf4

23 Mar, 2025 2 commits
- [V1][Spec Decode] Respect prompt_lookup_max (#15348) · b9bd76ca
  Woosuk Kwon authored Mar 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b9bd76ca
- [V1][Usage] Refactor speculative decoding configuration and tests (#14434) · 50c9636d
  shangmingc authored Mar 23, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  50c9636d
22 Mar, 2025 1 commit
- [V1] Add `disable-any-whitespace` option support for xgrammar (#15316) · eb63ea1e
  Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  eb63ea1e
21 Mar, 2025 4 commits
- [TPU][V1] MHA Pallas backend (#15288) · cfbb8c93
  Nicolò Lucchesi authored Mar 21, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  cfbb8c93
- [v1] Refactor KVCacheConfig (#14079) · 93a00d7d
  Chen Zhang authored Mar 21, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  93a00d7d
- [V1][TPU] Speed up top-k on TPU by using torch.topk (#15242) · 47195057
  Hyesoo Yang authored Mar 20, 2025
```
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
```
  47195057
- [V1] Scheduler Refactoring [1/N] - Add Scheduler Interface (#15250) · 0c6f5023
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  0c6f5023
20 Mar, 2025 2 commits
- [Bugfix] fix V1 Engine crash while handling requests with duplicate request id (#15043) · d8e82bc0
  Jason authored Mar 21, 2025
```
Signed-off-by: Jiahui Sun <jhsun2020@gmail.com>
```
  d8e82bc0
- [V1][TPU] Support V1 Sampler for ragged attention (#14227) · d8c6d7d6
  Nicolò Lucchesi authored Mar 20, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  d8c6d7d6
19 Mar, 2025 2 commits
- [V1] Minor V1 async engine test refactor (#15075) · 61c7a1b8
  Murali Andoorveedu authored Mar 19, 2025
```
Signed-off-by: andoorve <murali.andoorveedu@mail.utoronto.ca>
Co-authored-by: andoorve <murali.andoorveedu@mail.utoronto.ca>
```
  61c7a1b8
- [Core] Update dtype detection and defaults (#14858) · f690372b
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f690372b
18 Mar, 2025 2 commits
- [V1] TPU - CI/CD use smaller model (#15054) · 72a8639b
  Alexander Matveev authored Mar 18, 2025
```
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
```
  72a8639b
- [V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels (#14930) · 99abb8b6
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  99abb8b6
17 Mar, 2025 5 commits

[Fix][Structured Output] using vocab_size to construct matcher (#14868) · c0efdd65

Aaron Pham authored Mar 17, 2025

Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

c0efdd65

[Bugfix][ROCm] running new process using spawn method for rocm in tests. (#14810) · 2bb0e1a7

vllmellm authored Mar 17, 2025


Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

2bb0e1a7

[V1] [Spec Decode] Support random sampling for spec decode (#13933) · 8d6cf895
Lily Liu authored Mar 16, 2025
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
8d6cf895

[Misc] Replace os environ to monkeypatch in test suite (#14516) · a73e183e

Sibi authored Mar 17, 2025


Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>

a73e183e

[V1] Enable Entrypoints Tests (#14903) · aecc780d
Robert Shaw authored Mar 16, 2025

aecc780d

16 Mar, 2025 2 commits
- [BugFix][V1] Fix overhead related to bad_words sampling when not in use (#14894) · fc1f6771
  Nick Hill authored Mar 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  fc1f6771
- [V1] [Spec Decode] Fix ngram tests (#14878) · d1ad2a57
  Lily Liu authored Mar 16, 2025
  
  d1ad2a57
15 Mar, 2025 1 commit

[V1] V1 Enablement Oracle (#13726) · d4d93db2

Robert Shaw authored Mar 15, 2025


Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

d4d93db2

14 Mar, 2025 1 commit
- [V1] Fix model parameterization for structured output tests (#14833) · 46f98893
  Russell Bryant authored Mar 14, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  46f98893
13 Mar, 2025 1 commit
- [V1] Detokenizer: Respect Stop Tokens + not include_stop_str_in_output (#14624) · 02fcaa3d
  afeldman-nm authored Mar 13, 2025
```
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
```
  02fcaa3d
12 Mar, 2025 2 commits
- [BugFix][V1] Fix parallel sampling finishing/aborts (#14512) · f5d3acd4
  Nick Hill authored Mar 12, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  f5d3acd4
- [V1][Bugfix][Spec Decode] Fix incorrect outputs in V1 speculative decoding due... · 5c538c37
  Benjamin Chislett authored Mar 12, 2025
```
[V1][Bugfix][Spec Decode] Fix incorrect outputs in V1 speculative decoding due to batch indexing (#14645)
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
  5c538c37