Commits · 0d243f2a54fbd1c56da8a571f0899c30b6aba5d9 · OpenDAS / vllm_cscc · GitLab

20 Feb, 2025 1 commit
- [BugFix] Avoid error traceback in logs when V1 `LLM` terminates (#13565) · a4c402a7
  Nick Hill authored Feb 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  a4c402a7
19 Feb, 2025 1 commit
- [V1][Core] Generic mechanism for handling engine utility (#13060) · caf7ff44
  Nick Hill authored Feb 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  caf7ff44
18 Feb, 2025 4 commits
- [V1][Spec Decode] Optimize N-gram matching with Numba (#13365) · 4c822298
  Woosuk Kwon authored Feb 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4c822298
- [V1] Optimize handling of sampling metadata and req_ids list (#13244) · 30172b49
  Nick Hill authored Feb 18, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  30172b49
- [V1][PP] Enable true PP with Ray executor (#13472) · 4fb8142a
  Woosuk Kwon authored Feb 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4fb8142a
- [v1] fix parallel config rank (#13445) · 932b51ce
  youkaichao authored Feb 18, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  932b51ce
17 Feb, 2025 4 commits
- [V1][Spec decode] Move drafter to model runner (#13363) · cd4a72a2
  Woosuk Kwon authored Feb 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  cd4a72a2
- [V1][PP] Fix intermediate tensor values (#13417) · 6ac485a9
  Cody Yu authored Feb 17, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  6ac485a9
- [V1] Get input tokens from scheduler (#13339) · 4c21ce9e
  Woosuk Kwon authored Feb 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4c21ce9e
- [V1][Misc] Avoid unnecessary log output (#13289) · 2010f04c
  Jee Jee Li authored Feb 17, 2025
  
  2010f04c
16 Feb, 2025 4 commits
- [V1][BugFix] Clean up rejection sampler & Fix warning msg (#13362) · 69e1d23e
  Woosuk Kwon authored Feb 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  69e1d23e
- [V1][PP] Cache Intermediate Tensors (#13353) · e18227b0
  Woosuk Kwon authored Feb 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e18227b0
- [V1][BugFix] Add __init__.py to v1/spec_decode/ (#13359) · 7b893865
  Woosuk Kwon authored Feb 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7b893865
- [V1][Spec Decode] Ngram Spec Decode (#12193) · 80f63a39
  Lily Liu authored Feb 15, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  80f63a39
15 Feb, 2025 4 commits
- [V1][PP] Run engine busy loop with batch queue (#13064) · 9206b3d7
  Cody Yu authored Feb 15, 2025
  
  9206b3d7
- [V1][Metrics] Add iteration_tokens_total histogram from V0 (#13288) · 2ad1bc7a
  Mark McLoughlin authored Feb 15, 2025
  
  2ad1bc7a
- [V1][PP] Fix memory profiling in PP (#13315) · 0c730268
  Woosuk Kwon authored Feb 14, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  0c730268
- [V1][Sampler] Don't apply temp for greedy-only (#13311) · 6a854c7a
  Nick Hill authored Feb 14, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  6a854c7a
14 Feb, 2025 8 commits
- [V1][Core] min_p sampling support (#13191) · a12934d3
  Aoyu authored Feb 15, 2025
```
Signed-off-by: Aoyu <aoyuzhan@amazon.com>
Co-authored-by: Aoyu <aoyuzhan@amazon.com>
```
  a12934d3
- Support logit_bias in v1 Sampler (#13079) · 6224a9f6
  Lu Fang authored Feb 14, 2025
  
  6224a9f6
- [V1] Simplify GPUModelRunner._update_states check (#13265) · 085b7b2d
  Nick Hill authored Feb 14, 2025
  
  085b7b2d
- [WIP] TPU V1 Support Refactored (#13049) · 45f90bcb
  Alexander Matveev authored Feb 14, 2025
  
  45f90bcb
- [Bugfix][V1] GPUModelRunner._update_states should return True when there is a... · b0ccfc56
  Kero Liang authored Feb 14, 2025
```
[Bugfix][V1] GPUModelRunner._update_states should return True when there is a finished request in batch (#13126)
```
  b0ccfc56
- [ROCm][V1] Add intial ROCm support to V1 (#12790) · ba59b78a
  Sage Moore authored Feb 13, 2025
  
  ba59b78a
- [V1] LoRA - Enable Serving Usecase (#12883) · cbc40128
  Varun Sundar Rabindranath authored Feb 14, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  cbc40128
- [V1] Consolidate MM cache size to vllm.envs (#13239) · dd5ede44
  Roger Wang authored Feb 13, 2025
  
  dd5ede44
13 Feb, 2025 3 commits
- [V1][Core] Add worker_base for v1 worker (#12816) · 2092a6fa
  Aoyu authored Feb 13, 2025
```
Signed-off-by: Aoyu <aoyuzhan@amazon.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Aoyu <aoyuzhan@amazon.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  2092a6fa
- [V1] Clarify input processing and multimodal feature caching logic (#13211) · fdcf64d3
  Roger Wang authored Feb 13, 2025
  
  fdcf64d3
- [V1][core] Implement pipeline parallel on Ray (#12996) · 9605c125
  Rui Qiao authored Feb 13, 2025
  
  9605c125
12 Feb, 2025 2 commits
- [V1][Bugfix] Copy encoder input ids to fix set iteration issue during VLM abort (#13173) · 4c0d93f4
  Murali Andoorveedu authored Feb 12, 2025
```
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
```
  4c0d93f4
- [Bug] [V1] Try fetching stop_reason from EngineOutput before checking the request (#13108) · f4d97e4f
  bnellnm authored Feb 12, 2025
  
  f4d97e4f
11 Feb, 2025 3 commits
- [V1][Metrics] Add several request timing histograms (#12644) · 75e6e145
  Mark McLoughlin authored Feb 11, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  75e6e145
- [V1][Metrics] Add GPU prefix cache hit rate % gauge (#12592) · 41c5dd45
  Cody Yu authored Feb 11, 2025
  
  41c5dd45
- [V1][Minor] Move scheduler outputs to a separate file (#13062) · 2ff48576
  Woosuk Kwon authored Feb 10, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2ff48576
10 Feb, 2025 1 commit
- [V1] Use msgpack for core request serialization (#12918) · 67c4637c
  Nick Hill authored Feb 09, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  67c4637c
08 Feb, 2025 5 commits
- [V1] Cache `uses_mrope` in GPUModelRunner (#12969) · 24700c34
  Woosuk Kwon authored Feb 08, 2025
  
  24700c34
- [V1][Minor] Remove outdated comment (#12968) · 870c3748
  Woosuk Kwon authored Feb 08, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  870c3748
- [bugfix] fix early import of flash attention (#12959) · fe743b79
  youkaichao authored Feb 09, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  fe743b79
- [V1][Minor] Move cascade attn logic outside _prepare_inputs (#12943) · 4ea48fb3
  Woosuk Kwon authored Feb 08, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4ea48fb3
- [bugfix] respect distributed_executor_backend in world_size=1 (#12934) · 91dd8f7a
  youkaichao authored Feb 08, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  91dd8f7a