Commits · 70fedd0f7954079ebee36a7ca834cdf2f3e5d568 · OpenDAS / vllm_cscc

30 Mar, 2025 2 commits
- [V1] Override `mm_counts` for dummy data creation (#15703) · 803d5c35
  Cyrus Leung authored Mar 30, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  803d5c35
- [Bugfix] Fix Mistral guided generation using xgrammar (#15704) · 6909a762
  Julien Denize authored Mar 30, 2025
```
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
```
  6909a762
29 Mar, 2025 6 commits
- [V1][Minor] Simplify rejection sampler's parse_output (#15741) · 2bc4be4e
  Woosuk Kwon authored Mar 29, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2bc4be4e
- [V1] Support interleaved modality items (#15605) · c67abd61
  Roger Wang authored Mar 29, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  c67abd61
- [V1] [Feature] Collective RPC (#15444) · 94744ba4
  wwl2755 authored Mar 29, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  94744ba4
- [Kernel][TPU][ragged-paged-attn] vLLM code change for PR#8896 (#15659) · 7c1f7600
  yarongmu-google authored Mar 28, 2025
```
Signed-off-by: Yarong Mu <ymu@google.com>
```
  7c1f7600
- [TPU][V1][Bugfix] Fix w8a8 recompiilation with GSM8K (#15714) · da461f3c
  Nicolò Lucchesi authored Mar 29, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  da461f3c
- [Misc][V1] Misc code streamlining (#15723) · 6d531ad7
  Nick Hill authored Mar 28, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  6d531ad7
28 Mar, 2025 6 commits
- [V1] TPU - Fix the chunked prompt bug (#15713) · c3f687ac
  Alexander Matveev authored Mar 28, 2025
```
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
```
  c3f687ac
- [TPU] [Perf] Improve Memory Usage Estimation (#15671) · 038beded
  Robert Shaw authored Mar 28, 2025
```
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
```
  038beded
- [Misc] Remove unused utils and clean up imports (#15708) · c6bc0034
  Cyrus Leung authored Mar 29, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c6bc0034
- [V1] Support disable_any_whtespace for guidance backend (#15584) · 7329ff54
  Russell Bryant authored Mar 28, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  7329ff54
- [Bugfix][v1] xgrammar structured output supports Enum. (#15594) · 3b00ff91
  Chauncey authored Mar 28, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  3b00ff91
- [V1] Remove legacy input registry (#15673) · 355f6634
  Cyrus Leung authored Mar 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  355f6634
27 Mar, 2025 9 commits
- [V1] AsyncLLM data parallel (#13923) · 15dac210
  Nick Hill authored Mar 27, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  15dac210
- [Bugfix][TPU][V1] Fix recompilation (#15553) · 4098b722
  Nicolò Lucchesi authored Mar 27, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  4098b722
- [Misc] Avoid direct access of global `mm_registry` in `compute_encoder_budget` (#15621) · 13ac9cab
  Cyrus Leung authored Mar 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  13ac9cab
- [Misc] Replace `is_encoder_decoder_inputs` with `split_enc_dec_inputs` (#15620) · 24718153
  Cyrus Leung authored Mar 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  24718153
- [Bugfix] Fix use_cascade_attention handling for Alibi-based models on vllm/v1 (#15211) · 8958217a
  Hiroaki Sugiyama authored Mar 27, 2025
```
Signed-off-by: h-sugi <h.sugi@ieee.org>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8958217a
- [TPU] [V1] fix cases when max_num_reqs is set smaller than MIN_NUM_SEQS (#15583) · 619d3de8
  Chengji Yao authored Mar 26, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  619d3de8
- [V1] Refactor num_computed_tokens logic (#15307) · 54aa6194
  Cody Yu authored Mar 26, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  54aa6194
- add platform check back (#15578) · 69db16a4
  Chenyaaang authored Mar 26, 2025
```
Signed-off-by: Chenyaaang <llccyy1212@gmail.com>
```
  69db16a4
- [TPU] support disabling xla compilation cache (#15567) · e74ff409
  Chengji Yao authored Mar 26, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  e74ff409
26 Mar, 2025 4 commits
- [V1] TPU - Revert to exponential padding by default (#15565) · b2e85e26
  Alexander Matveev authored Mar 26, 2025
```
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
```
  b2e85e26
- Support SHA256 as hash function in prefix caching (#15297) · 27df5199
  marko authored Mar 26, 2025
```
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
```
  27df5199
- [V1][Sampler] Faster top-k only implementation (#15478) · 35fad35a
  Nick Hill authored Mar 26, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  35fad35a
- [Refactor] Remove unnecessary backend parameter in structured output interface (#15317) · 733e7c9e
  Aaron Pham authored Mar 26, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  733e7c9e
25 Mar, 2025 10 commits
- [Core] LoRA: V1 Scheduler optimization (#15422) · a5cfbab3
  Varun Sundar Rabindranath authored Mar 25, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  a5cfbab3
- [core] add bucket padding to tpu_model_runner (#14995) · ac3cd6e8
  Chenyaaang authored Mar 25, 2025
```
Signed-off-by: Chenyaaang <llccyy1212@gmail.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  ac3cd6e8
- [V1] Support long_prefill_token_threshold in v1 scheduler (#15419) · 082ab86f
  Lu Fang authored Mar 25, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  082ab86f
- [V1][Minor] Use `SchedulerInterface` type for engine scheduler field (#15499) · 6aa196c8
  Nick Hill authored Mar 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  6aa196c8
- [TPU][V1] Fix Sampler recompilation (#15309) · a0dd7dcd
  Nicolò Lucchesi authored Mar 25, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  a0dd7dcd
- [Hardware][TPU][Bugfix] Fix v1 mp profiler (#15409) · 4157f563
  Siyuan Liu authored Mar 25, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  4157f563
- [V1][Spec Decode] Update target_logits in place for rejection sampling (#15427) · 25f560a6
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  25f560a6
- [V1] guidance backend for structured output + `auto` fallback mode (#14779) · a09ad90a
  Russell Bryant authored Mar 25, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <jc1da.3011@gmail.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
```
  a09ad90a
- [Minor][Spec Decode] Remove compiled_softmax (#15416) · 911c8eb0
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  911c8eb0
- [V1][Spec Decode] Enable spec decode for top-p & top-k sampling (#15063) · ebcebeeb
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  ebcebeeb
24 Mar, 2025 3 commits

[V1][Perf] Simpler request output queues (#15156) · 9d72daf4

Nick Hill authored Mar 24, 2025


Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>

9d72daf4

[V1] Aggregate chunked prompt logprobs in model runner (#14875) · 3aee6573
Nick Hill authored Mar 24, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
3aee6573

[V1][Minor] fix comments (#15392) · 0893567d

Chen1022 authored Mar 24, 2025


Signed-off-by: chenjincong <chenjincong@baidu.com>
Signed-off-by: Chen-0210 <chenjincong11@gmail.com>
Co-authored-by: chenjincong <chenjincong@baidu.com>

0893567d