Commits · 35fad35a485eac9195c510731ba4a9d297dfd963 · OpenDAS / vllm_cscc

"vllm/vscode:/vscode.git/clone" did not exist on "98a011e9639c30d7de6e0c224009633e9a85f9d3"

26 Mar, 2025 2 commits
- [V1][Sampler] Faster top-k only implementation (#15478) · 35fad35a
  Nick Hill authored Mar 26, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  35fad35a
- [Refactor] Remove unnecessary backend parameter in structured output interface (#15317) · 733e7c9e
  Aaron Pham authored Mar 26, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  733e7c9e
25 Mar, 2025 10 commits
- [Core] LoRA: V1 Scheduler optimization (#15422) · a5cfbab3
  Varun Sundar Rabindranath authored Mar 25, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  a5cfbab3
- [core] add bucket padding to tpu_model_runner (#14995) · ac3cd6e8
  Chenyaaang authored Mar 25, 2025
```
Signed-off-by: Chenyaaang <llccyy1212@gmail.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  ac3cd6e8
- [V1] Support long_prefill_token_threshold in v1 scheduler (#15419) · 082ab86f
  Lu Fang authored Mar 25, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  082ab86f
- [V1][Minor] Use `SchedulerInterface` type for engine scheduler field (#15499) · 6aa196c8
  Nick Hill authored Mar 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  6aa196c8
- [TPU][V1] Fix Sampler recompilation (#15309) · a0dd7dcd
  Nicolò Lucchesi authored Mar 25, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  a0dd7dcd
- [Hardware][TPU][Bugfix] Fix v1 mp profiler (#15409) · 4157f563
  Siyuan Liu authored Mar 25, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  4157f563
- [V1][Spec Decode] Update target_logits in place for rejection sampling (#15427) · 25f560a6
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  25f560a6
- [V1] guidance backend for structured output + `auto` fallback mode (#14779) · a09ad90a
  Russell Bryant authored Mar 25, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <jc1da.3011@gmail.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
```
  a09ad90a
- [Minor][Spec Decode] Remove compiled_softmax (#15416) · 911c8eb0
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  911c8eb0
- [V1][Spec Decode] Enable spec decode for top-p & top-k sampling (#15063) · ebcebeeb
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  ebcebeeb
24 Mar, 2025 3 commits

[V1][Perf] Simpler request output queues (#15156) · 9d72daf4

Nick Hill authored Mar 24, 2025


Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>

9d72daf4

[V1] Aggregate chunked prompt logprobs in model runner (#14875) · 3aee6573
Nick Hill authored Mar 24, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
3aee6573

[V1][Minor] fix comments (#15392) · 0893567d

Chen1022 authored Mar 24, 2025


Signed-off-by: chenjincong <chenjincong@baidu.com>
Signed-off-by: Chen-0210 <chenjincong11@gmail.com>
Co-authored-by: chenjincong <chenjincong@baidu.com>

0893567d

23 Mar, 2025 3 commits
- [V1] Enable V1 Fp8 cache for FA3 in the oracle (#15191) · dccf535f
  Lucas Wilkinson authored Mar 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  dccf535f
- [V1][Spec Decode] Respect prompt_lookup_max (#15348) · b9bd76ca
  Woosuk Kwon authored Mar 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b9bd76ca
- [V1][Usage] Refactor speculative decoding configuration and tests (#14434) · 50c9636d
  shangmingc authored Mar 23, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  50c9636d
22 Mar, 2025 2 commits
- [BugFix][Typing] Fix Imprecise Type Annotations (#15208) · dd861b99
  Wang Ran (汪然) authored Mar 23, 2025
```
Signed-off-by: Wang Ran (汪然) <wrran@outlook.com>
```
  dd861b99
- [V1] Add `disable-any-whitespace` option support for xgrammar (#15316) · eb63ea1e
  Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  eb63ea1e
21 Mar, 2025 7 commits
- [v1] Refactor KVCacheConfig (#14079) · 93a00d7d
  Chen Zhang authored Mar 21, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  93a00d7d
- [V1] Fix wrong import path of get_flash_attn_version (#15280) · 91ca929d
  Lehua Ding authored Mar 21, 2025
```
Signed-off-by: Lehua Ding <lehuading@tencent.com>
```
  91ca929d
- [V1] Avoid redundant input processing in n>1 case (#14985) · da6ea29f
  Nick Hill authored Mar 20, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  da6ea29f
- [V1] Enable Triton(ROCm) Attention backend for Nvidia GPUs (#14071) · f8a08cb9
  Isotr0py authored Mar 21, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f8a08cb9
- [Hardware][TPU] Add check for no additional graph compilation during runtime (#14710) · b15fd2be
  Siyuan Liu authored Mar 20, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  b15fd2be
- [V1][TPU] Speed up top-k on TPU by using torch.topk (#15242) · 47195057
  Hyesoo Yang authored Mar 20, 2025
```
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
```
  47195057
- [V1] Scheduler Refactoring [1/N] - Add Scheduler Interface (#15250) · 0c6f5023
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  0c6f5023
20 Mar, 2025 5 commits
- [V1] Add flag to disable cascade attention (#15243) · 2b22290c
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2b22290c
- [Bugfix] fix V1 Engine crash while handling requests with duplicate request id (#15043) · d8e82bc0
  Jason authored Mar 21, 2025
```
Signed-off-by: Jiahui Sun <jhsun2020@gmail.com>
```
  d8e82bc0
- [Attention] Flash Attention 3 - fp8 (#14570) · a597a575
  Mickaël Seznec authored Mar 20, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
```
  a597a575
- [V1][TPU] Support V1 Sampler for ragged attention (#14227) · d8c6d7d6
  Nicolò Lucchesi authored Mar 20, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  d8c6d7d6
- [BugFix] Lazily import XgrammarBackend to avoid early cuda init (#15171) · c47aafa3
  Nick Hill authored Mar 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  c47aafa3
19 Mar, 2025 4 commits
- [V1][TPU] Change kv cache shape. (#15145) · b0e96aae
  iefgnoix authored Mar 19, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  b0e96aae
- simple bugfix: Update stats.py (#15139) · 8310e0b5
  Wang Ran (汪然) authored Mar 20, 2025
  
  8310e0b5
- [FEAT]Support reset prefix cache by specified device (#15003) · 26dd972a
  maobaolong authored Mar 20, 2025
  
  26dd972a
- [V1] Ensure using int64 for sampled token ids (#15065) · 05ccd0aa
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  05ccd0aa
18 Mar, 2025 4 commits
- fix long dtype in topk sampling (#15049) · 027827cc
  Chujie Zheng authored Mar 19, 2025
  
  027827cc
- [V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels (#14930) · 99abb8b6
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  99abb8b6
- [V1] Refactor Structured Output for multiple backends (#14694) · 3a1e6481
  Russell Bryant authored Mar 18, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  3a1e6481
- [TPU][V1][Bugfix] Fix chunked prefill with padding (#15037) · af35d3a3
  Nicolò Lucchesi authored Mar 18, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  af35d3a3