Commits · 711aa9d5b6784d50c2f9ab89642566d5cc3fe4ab · OpenDAS / vllm_cscc

23 Jul, 2025 1 commit
- [Sampler] Introduce logprobs mode for logging (#21398) · accac829
  Lu Fang authored Jul 23, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  accac829
22 Jul, 2025 1 commit
- [Core] Optimize update checks in LogitsProcessor (#21245) · a3223766
  Jialin Ouyang authored Jul 22, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  a3223766
21 Jul, 2025 1 commit
- [v1][sampler] Inplace logprobs comparison to get the token rank (#21283) · 8d0a01a5
  Lu Fang authored Jul 21, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  8d0a01a5
12 Jul, 2025 1 commit
- Remove extra tensor on CPU (#20693) · 5de8d9f1
  Maximilien de Bayser authored Jul 12, 2025
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  5de8d9f1
05 Jul, 2025 1 commit
- [v1] Re-add fp32 support to v1 engine through FlexAttention (#19754) · 32c9be22
  Isotr0py authored Jul 05, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  32c9be22
04 Jul, 2025 1 commit
- [Misc] Add SPDX-FileCopyrightText (#20428) · 1caca5a5
  Jee Jee Li authored Jul 04, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1caca5a5
02 Jul, 2025 1 commit

[V1] LogitsProcessor programming model (#16728) · 48fb076c

afeldman-nm authored Jul 02, 2025


Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

48fb076c

18 Jun, 2025 1 commit
- [V1] Decouple GPU and TPU `InputBatch` (#19778) · 19a53b27
  afeldman-nm authored Jun 18, 2025
```
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
```
  19a53b27
06 Jun, 2025 1 commit
- improve logits bias (#19041) · 8267f991
  Yu Guo authored Jun 06, 2025
  
  8267f991
03 Jun, 2025 1 commit
- [Misc] Add SPDX-FileCopyrightText (#19100) · 02f0c7b2
  Simon Mo authored Jun 03, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  02f0c7b2
26 May, 2025 1 commit
- [V1][Sampler] Improve performance of FlashInfer sampling by sampling logits... · e7523c2e
  Lukas Geiger authored May 26, 2025
```
[V1][Sampler] Improve performance of FlashInfer sampling by sampling logits instead of probs (#18608)
```
  e7523c2e
16 May, 2025 1 commit
- [Sampler] Adapt to FlashInfer 0.2.3 sampler API (#15777) · 7fdfa015
  Bowen Wang authored May 16, 2025
```
Signed-off-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  7fdfa015
06 May, 2025 1 commit
- [Bugfix] Fix triton import with local TritonPlaceholder (#17446) · f9bc5a06
  Mengqing Cao authored May 06, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  f9bc5a06
05 May, 2025 1 commit
- [TPU][V1] Add support for top-logprobs (#17072) · 5941e0b7
  Nicolò Lucchesi authored May 05, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  5941e0b7
04 May, 2025 1 commit
- Add full API docs and improve the UX of navigating them (#17485) · d6484ef3
  Harry Mellor authored May 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6484ef3
23 Apr, 2025 1 commit
- [V1][Spec Decode] Always use argmax for sampling draft tokens (#16899) · 41fb013d
  Woosuk Kwon authored Apr 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  41fb013d
22 Apr, 2025 1 commit

[TPU][V1] Enable Top-P (#16843) · fa3bba2a

Nicolò Lucchesi authored Apr 22, 2025


Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

fa3bba2a

17 Apr, 2025 1 commit

[V1][TPU] Enable Top K (#15489) · eb5819b2

Nicolò Lucchesi authored Apr 17, 2025


Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: Hyesoo Yang <hyeygit@gmail.com>

eb5819b2

12 Apr, 2025 1 commit
- [Bugfix] Validate logit biases to prevent out of vocab ids crashing engine (#16529) · 6c11ecf8
  Ryan McConville authored Apr 12, 2025
```
Signed-off-by: Ryan McConville <ryan@ryanmcconville.com>
```
  6c11ecf8
10 Apr, 2025 2 commits
- [TPU][V1] Disable per-request seed/Generator (#16172) · 3cc9af88
  Nicolò Lucchesi authored Apr 10, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  3cc9af88
- [TPU][V1] Refine tpu_model_runner to mitigate future recompilation issues (#16275) · a4547485
  Chengji Yao authored Apr 09, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  a4547485
03 Apr, 2025 1 commit

[V1][TPU] TPU-optimized top-p implementation (avoids scattering). (#15736) · 1b84eff0

Hyesoo Yang authored Apr 02, 2025


Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: root <root@t1v-n-822696b7-w-0.us-central2-b.c.tpu-prod-env-large-adhoc.internal>

1b84eff0

02 Apr, 2025 1 commit
- [V1] Fix: make sure `k_index` is int64 for `apply_top_k_only` (#15907) · 6efb195a
  Brayden Zhong authored Apr 01, 2025
```
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  6efb195a
29 Mar, 2025 1 commit
- [V1][Minor] Simplify rejection sampler's parse_output (#15741) · 2bc4be4e
  Woosuk Kwon authored Mar 29, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2bc4be4e
27 Mar, 2025 2 commits
- [Bugfix][TPU][V1] Fix recompilation (#15553) · 4098b722
  Nicolò Lucchesi authored Mar 27, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  4098b722
- [V1] Refactor num_computed_tokens logic (#15307) · 54aa6194
  Cody Yu authored Mar 26, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  54aa6194
26 Mar, 2025 1 commit
- [V1][Sampler] Faster top-k only implementation (#15478) · 35fad35a
  Nick Hill authored Mar 26, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  35fad35a
25 Mar, 2025 4 commits
- [TPU][V1] Fix Sampler recompilation (#15309) · a0dd7dcd
  Nicolò Lucchesi authored Mar 25, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  a0dd7dcd
- [V1][Spec Decode] Update target_logits in place for rejection sampling (#15427) · 25f560a6
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  25f560a6
- [Minor][Spec Decode] Remove compiled_softmax (#15416) · 911c8eb0
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  911c8eb0
- [V1][Spec Decode] Enable spec decode for top-p & top-k sampling (#15063) · ebcebeeb
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  ebcebeeb
24 Mar, 2025 1 commit

[V1][Minor] fix comments (#15392) · 0893567d

Chen1022 authored Mar 24, 2025


Signed-off-by: chenjincong <chenjincong@baidu.com>
Signed-off-by: Chen-0210 <chenjincong11@gmail.com>
Co-authored-by: chenjincong <chenjincong@baidu.com>

0893567d

21 Mar, 2025 1 commit
- [V1][TPU] Speed up top-k on TPU by using torch.topk (#15242) · 47195057
  Hyesoo Yang authored Mar 20, 2025
```
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
```
  47195057
20 Mar, 2025 1 commit
- [V1][TPU] Support V1 Sampler for ragged attention (#14227) · d8c6d7d6
  Nicolò Lucchesi authored Mar 20, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  d8c6d7d6
19 Mar, 2025 1 commit
- [V1] Ensure using int64 for sampled token ids (#15065) · 05ccd0aa
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  05ccd0aa
18 Mar, 2025 2 commits
- fix long dtype in topk sampling (#15049) · 027827cc
  Chujie Zheng authored Mar 19, 2025
  
  027827cc
- [V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels (#14930) · 99abb8b6
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  99abb8b6
17 Mar, 2025 1 commit
- [V1] [Spec Decode] Support random sampling for spec decode (#13933) · 8d6cf895
  Lily Liu authored Mar 16, 2025
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8d6cf895
15 Mar, 2025 1 commit
- [Bugfix][V1] Fix flashinfer sampling (#14815) · acaea3bb
  DefTruth authored Mar 15, 2025
  
  acaea3bb
14 Mar, 2025 1 commit
- [V1] Temporarily disable FlashInfer Rejection Sampler (#14788) · 32ef4983
  Woosuk Kwon authored Mar 13, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  32ef4983