Commits · ab1091d5f2fc879ba9e62002f4d9eec013984d4d · OpenDAS / vllm_cscc

25 Feb, 2025 2 commits
- [Misc] set single whitespace between log sentences (#13771) · 51010a18
  cjackal authored Feb 25, 2025
```
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  51010a18
- Remove unused kwargs from model definitions (#13555) · cdc1fa12
  Harry Mellor authored Feb 25, 2025
  
  cdc1fa12
24 Feb, 2025 3 commits
- Revert "[V1][Core] Fix memory issue with logits & sampling" (#13775) · 22757848
  Roger Wang authored Feb 24, 2025
  
  22757848
- [V1] V1 engine implements parallel sampling (AsyncLLM and LLMEngine) (#10980) · befc402d
  afeldman-nm authored Feb 24, 2025
```
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  befc402d
- [V1][Core] Fix memory issue with logits & sampling (#13721) · 437b76ff
  Roger Wang authored Feb 24, 2025
  
  437b76ff
23 Feb, 2025 2 commits

[V1][BugFix] Fix engine core client shutdown hangs (#13298) · cbae7af5

Nick Hill authored Feb 23, 2025

Even though ZMQ context.destroy() is meant to close open sockets before terminating the context, it appears to be necessary to do this explicitly or else it can hang in the context.term() method.

Close zmq sockets explicitly before terminating context, make shutdown of client resource more robust, shut down engine core process prior to terminating zmq context.
Signed-off-by: Nick Hill <nhill@redhat.com>

cbae7af5

[v1] torchrun compatibility (#13642) · eb24dc4a
youkaichao authored Feb 23, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
eb24dc4a

22 Feb, 2025 9 commits
- [V1][Kernel] Refactor the prefix_prefill kernel so that the caller no longer... · 558db808
  Sage Moore authored Feb 22, 2025
```
[V1][Kernel] Refactor the prefix_prefill kernel so that the caller no longer has to pass in the context lengths (#13095)
```
  558db808
- [ci] fix linter (#13701) · 2382ad29
  youkaichao authored Feb 22, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  2382ad29
- [core] set up data parallel communication (#13591) · 3e472d88
  youkaichao authored Feb 22, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3e472d88
- [CI/Build] Fix pre-commit errors (#13696) · 7f6bae56
  Cyrus Leung authored Feb 22, 2025
  
  7f6bae56
- [Metrics] Add `--show-hidden-metrics-for-version` CLI arg (#13295) · 2cb8c154
  Mark McLoughlin authored Feb 22, 2025
  
  2cb8c154
- [V1][Metrics] Support `vllm:cache_config_info` (#13299) · 1cd981da
  Mark McLoughlin authored Feb 22, 2025
  
  1cd981da
- [Bugfix] V1 Memory Profiling: V0 Sampler Integration without Rejection Sampler (#13594) · da31b533
  Jennifer Zhao authored Feb 22, 2025
```
Signed-off-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  da31b533
- [v1] Support allowed_token_ids in v1 Sampler (#13210) · bb78fb31
  Lu Fang authored Feb 21, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  bb78fb31
- [Misc] Capture and log the time of loading weights (#13666) · 68d535ef
  Jun Duan authored Feb 22, 2025
  
  68d535ef
21 Feb, 2025 2 commits

[Attention] MLA with chunked prefill (#12639) · 288cc6c2

Lucas Wilkinson authored Feb 21, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patrick Horn <patrick.horn@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

288cc6c2

[V1][Sampler] Avoid an operation during temperature application (#13587) · 31aa045c
Nick Hill authored Feb 20, 2025

31aa045c

20 Feb, 2025 3 commits
- [V1][Minor] Print KV cache size in token counts (#13596) · d3ea5011
  Woosuk Kwon authored Feb 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d3ea5011
- [core] add sleep and wake up endpoint and v1 support (#12987) · ba811639
  youkaichao authored Feb 20, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: cennn <2523403608@qq.com>
Co-authored-by: cennn <2523403608@qq.com>
```
  ba811639
- [BugFix] Avoid error traceback in logs when V1 `LLM` terminates (#13565) · a4c402a7
  Nick Hill authored Feb 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  a4c402a7
19 Feb, 2025 1 commit
- [V1][Core] Generic mechanism for handling engine utility (#13060) · caf7ff44
  Nick Hill authored Feb 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  caf7ff44
18 Feb, 2025 4 commits
- [V1][Spec Decode] Optimize N-gram matching with Numba (#13365) · 4c822298
  Woosuk Kwon authored Feb 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4c822298
- [V1] Optimize handling of sampling metadata and req_ids list (#13244) · 30172b49
  Nick Hill authored Feb 18, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  30172b49
- [V1][PP] Enable true PP with Ray executor (#13472) · 4fb8142a
  Woosuk Kwon authored Feb 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4fb8142a
- [v1] fix parallel config rank (#13445) · 932b51ce
  youkaichao authored Feb 18, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  932b51ce
17 Feb, 2025 4 commits
- [V1][Spec decode] Move drafter to model runner (#13363) · cd4a72a2
  Woosuk Kwon authored Feb 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  cd4a72a2
- [V1][PP] Fix intermediate tensor values (#13417) · 6ac485a9
  Cody Yu authored Feb 17, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  6ac485a9
- [V1] Get input tokens from scheduler (#13339) · 4c21ce9e
  Woosuk Kwon authored Feb 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4c21ce9e
- [V1][Misc] Avoid unnecessary log output (#13289) · 2010f04c
  Jee Jee Li authored Feb 17, 2025
  
  2010f04c
16 Feb, 2025 4 commits
- [V1][BugFix] Clean up rejection sampler & Fix warning msg (#13362) · 69e1d23e
  Woosuk Kwon authored Feb 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  69e1d23e
- [V1][PP] Cache Intermediate Tensors (#13353) · e18227b0
  Woosuk Kwon authored Feb 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e18227b0
- [V1][BugFix] Add __init__.py to v1/spec_decode/ (#13359) · 7b893865
  Woosuk Kwon authored Feb 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7b893865
- [V1][Spec Decode] Ngram Spec Decode (#12193) · 80f63a39
  Lily Liu authored Feb 15, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  80f63a39
15 Feb, 2025 4 commits
- [V1][PP] Run engine busy loop with batch queue (#13064) · 9206b3d7
  Cody Yu authored Feb 15, 2025
  
  9206b3d7
- [V1][Metrics] Add iteration_tokens_total histogram from V0 (#13288) · 2ad1bc7a
  Mark McLoughlin authored Feb 15, 2025
  
  2ad1bc7a
- [V1][PP] Fix memory profiling in PP (#13315) · 0c730268
  Woosuk Kwon authored Feb 14, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  0c730268
- [V1][Sampler] Don't apply temp for greedy-only (#13311) · 6a854c7a
  Nick Hill authored Feb 14, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  6a854c7a
14 Feb, 2025 2 commits
- [V1][Core] min_p sampling support (#13191) · a12934d3
  Aoyu authored Feb 15, 2025
```
Signed-off-by: Aoyu <aoyuzhan@amazon.com>
Co-authored-by: Aoyu <aoyuzhan@amazon.com>
```
  a12934d3
- Support logit_bias in v1 Sampler (#13079) · 6224a9f6
  Lu Fang authored Feb 14, 2025
  
  6224a9f6