Commits · 774c5fde307dcadb8e0d2a58bf2d429eb6cb648d · OpenDAS / vllm_cscc

28 May, 2025 1 commit
- [V1] fix torch profiling for V1 offline scenarios (#18445) · 774c5fde
  Divakar Verma authored May 27, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  774c5fde
27 May, 2025 2 commits
- [Core] Automatically cast multi-modal input dtype (#18756) · 696259ca
  Cyrus Leung authored May 27, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  696259ca
- [Doc] Convert Sphinx directives ( `{class}`, `{meth}`, `{attr}`, ...) to... · a68e293c
  Hyogeun Oh (오효근) authored May 27, 2025
```
[Doc]  Convert Sphinx directives ( `{class}`, `{meth}`, `{attr}`, ...) to MkDocs format for better documentation linking (#18663)
Signed-off-by: Zerohertz <ohg3417@gmail.com>
```
  a68e293c
24 May, 2025 2 commits
- [V1][Spec Decode] Support multi-layer eagle draft model (#18030) · c1e4a405
  qizixi authored May 24, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  c1e4a405
- [V1][Spec Decode] Small refactors to improve eagle bookkeeping performance (#18424) · d55e446d
  qizixi authored May 23, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  d55e446d
23 May, 2025 6 commits
- [V1] Support Deepseek MTP (#18435) · 2628a69e
  Jiayi Yao authored May 23, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
```
  2628a69e
- [v1] Redo "Support multiple KV cache groups in GPU model runner (#17945)" (#18593) · 6550114c
  Chen Zhang authored May 24, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  6550114c
- Refactor pplx init logic to make it modular (prepare for deepep) (#18200) · 6a7988c5
  youkaichao authored May 23, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6a7988c5
- Replace `{func}` with mkdocs style links (#18610) · 2edb533a
  Harry Mellor authored May 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  2edb533a
- Migrate docs from Sphinx to MkDocs (#18145) · a1fe24d9
  Harry Mellor authored May 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a1fe24d9
- [Frontend] [Core] Add Tensorizer support for V1, LoRA adapter serialization... · c32e249a
  Sanger Steel authored May 22, 2025
```
[Frontend] [Core] Add Tensorizer support for V1, LoRA adapter serialization and deserialization (#17926)
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
```
  c32e249a
22 May, 2025 1 commit
- [Bugfix] Use random hidden states in dummy sampler run (#18543) · 4e04eceb
  Bowen Wang authored May 22, 2025
```
Signed-off-by: Bowen Wang <abmfy@icloud.com>
```
  4e04eceb
21 May, 2025 1 commit
- Revert "[v1] Support multiple KV cache groups in GPU model runner (#17945) (#18459) · bb0a3112
  Mark McLoughlin authored May 21, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  bb0a3112
17 May, 2025 2 commits

Support sequence parallelism combined with pipeline parallelism (#18243) · 9ab2c02f
cascade authored May 17, 2025
```
Signed-off-by: cascade812 <cascade812@outlook.com>
```
9ab2c02f

[Hardware][TPU] Optionally import for TPU backend (#18269) · 48ac2bed

Siyuan Liu authored May 17, 2025


Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: Carol Zheng <cazheng@google.com>
Co-authored-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: Hongmin Fan <fanhongmin@google.com>

48ac2bed

16 May, 2025 2 commits
- [Feature] Support Pipeline Parallism in torchrun SPMD offline inference for V1 (#17827) · 3d2779c2
  Lucia Fang authored May 15, 2025
```
Signed-off-by: Lucia Fang <fanglu@fb.com>
```
  3d2779c2
- [Model] vLLM v1 supports Medusa (#17956) · f4937a51
  Sky Lee authored May 16, 2025
```
Signed-off-by: lisiqi23 <lisiqi23@xiaomi.com>
Signed-off-by: skylee-01 <497627264@qq.com>
Co-authored-by: lisiqi23 <lisiqi23@xiaomi.com>
```
  f4937a51
15 May, 2025 1 commit
- [v1] Support multiple KV cache groups in GPU model runner (#17945) · e60f550b
  Chen Zhang authored May 15, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  e60f550b
14 May, 2025 3 commits
- Modularize fused experts and integrate PPLX kernels (#15956) · f9c069c8
  bnellnm authored May 14, 2025
  
  f9c069c8
- [core][distributed] add ep group and all2all interface (#18077) · 6266c57b
  youkaichao authored May 14, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6266c57b
- [Misc] Remove unused numpy tensor (#18084) · 6e27c6d8
  Roger Wang authored May 13, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.me>
```
  6e27c6d8
13 May, 2025 1 commit
- [Bugfix][V1] Only get input embeddings w/ multi-modal models if first PP (#17916) · 8dd0671b
  Jin Huang authored May 13, 2025
```
Signed-off-by: Jin Huang <jinhun@amazon.com>
Co-authored-by: Jin Huang <jinhun@amazon.com>
```
  8dd0671b
12 May, 2025 2 commits

[P/D] NIXL Integration (#17751) · d1911020

Robert Shaw authored May 12, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Brent Salisbury <bsalisbu@redhat.com>

d1911020

[Bugfix][TPU] Use np array when updating cache slot_mapping (#17971) · 43078301
Siyuan Liu authored May 11, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
43078301

10 May, 2025 1 commit
- [v1] Pass BlockTable and KVCacheSpec to AttentionMetadataBuilders (#17483) · 950751a9
  Chen Zhang authored May 11, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  950751a9
08 May, 2025 1 commit

[Core] Support full cuda graph in v1 (#16072) · 7ea2adb8

Chanh Nguyen authored May 07, 2025


Signed-off-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>

7ea2adb8

07 May, 2025 2 commits

[Hardware][TPU][V1] Multi-LoRA implementation for the V1 TPU backend (#14238) · c20ef40f

Akshat Tripathi authored May 07, 2025


Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>

c20ef40f

[Misc] Split model loader (#17712) · 822de7fb
Jee Jee Li authored May 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
822de7fb

06 May, 2025 2 commits
- [v1] AttentionMetadata for each layer (#17394) · cba31c47
  Chen Zhang authored May 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cba31c47
- [V1][PP] Support PP for MultiprocExecutor (#14219) · a6fed020
  Li, Jiang authored May 06, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: jiang.li <jiang1.li@intel.com>
```
  a6fed020
05 May, 2025 1 commit
- [TPU][V1] Add support for top-logprobs (#17072) · 5941e0b7
  Nicolò Lucchesi authored May 05, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  5941e0b7
04 May, 2025 1 commit
- Add full API docs and improve the UX of navigating them (#17485) · d6484ef3
  Harry Mellor authored May 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6484ef3
30 Apr, 2025 1 commit
- [V1][Feature] Enable Speculative Decoding with Structured Outputs (#14702) · 34120f5a
  Benjamin Chislett authored Apr 29, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
```
  34120f5a
29 Apr, 2025 2 commits
- [V1][Spec Decode] Apply torch.compile & cudagraph to EAGLE (#17211) · 70788bdb
  Bryan Lu authored Apr 29, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
```
  70788bdb
- [V1] Remove num_input_tokens from attn_metadata (#17193) · 24e6ad3f
  Chen Zhang authored Apr 30, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  24e6ad3f
27 Apr, 2025 2 commits

[Feature] support sequence parallelism using compilation pass (#16155) · 690fe019

cascade authored Apr 27, 2025


Signed-off-by: cascade812 <cascade812@outlook.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

690fe019

[Bugfix] Get a specific type of layer from forward context (#17222) · 838cedad
Chen Zhang authored Apr 27, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
838cedad

26 Apr, 2025 2 commits
- [Core] Remove prompt string from engine core data structures (#17214) · df6f3ce8
  Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  df6f3ce8
- [Minor][Spec Decode] Add use_eagle to SpeculativeConfig (#17213) · 1cf0719e
  Woosuk Kwon authored Apr 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1cf0719e
25 Apr, 2025 1 commit

[V1][Spec Decode] EAGLE-3 Support (#16937) · a0e619e6

Benjamin Chislett authored Apr 25, 2025


Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Bryan Lu <yuzhelu@amazon.com>

a0e619e6