Commits · cb84e45ac75b42ba6795145923e8eb323bb825ad · OpenDAS / vllm_cscc

09 Apr, 2025 1 commit
- [Core] Upgrade to xgrammar 0.1.18, add cache size limit (#16283) · cb84e45a
  Russell Bryant authored Apr 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  cb84e45a
06 Apr, 2025 1 commit
- [Misc] Improve model redirect to accept json dictionary (#16119) · c2a96715
  Isotr0py authored Apr 06, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c2a96715
01 Apr, 2025 2 commits
- Add option to use DeepGemm contiguous grouped gemm kernel for fused MoE operations. (#13932) · e59ca942
  bnellnm authored Apr 01, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  e59ca942
- [Misc] Use envs.VLLM_USE_RAY_COMPILED_DAG_CHANNEL_TYPE (#15831) · 8dd41d6b
  Rui Qiao authored Apr 01, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8dd41d6b
29 Mar, 2025 1 commit
- [FEAT] [ROCm] Add AITER int8 scaled gemm kernel (#15433) · 4965ec42
  TJian authored Mar 29, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  4965ec42
27 Mar, 2025 4 commits
- [V1] AsyncLLM data parallel (#13923) · 15dac210
  Nick Hill authored Mar 27, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  15dac210
- [Feature] Add middleware to log API Server responses (#15593) · 66aa4c0b
  Yuan Tang authored Mar 27, 2025
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  66aa4c0b
- [Misc] Use model_redirect to redirect the model name to a local folder. (#14116) · 3f532cb6
  wang.yuqi authored Mar 27, 2025
  
  3f532cb6
- [ROCm] Env variable to trigger custom PA (#15557) · ecff8309
  Gregory Shtrasberg authored Mar 27, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  ecff8309
26 Mar, 2025 2 commits

[V1] TPU - Revert to exponential padding by default (#15565) · b2e85e26
Alexander Matveev authored Mar 26, 2025
```
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
```
b2e85e26

[FEAT][ROCm] Integrate Fused MoE Kernels from AITER (#14967) · 5ebf6674

vllmellm authored Mar 26, 2025


Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>

5ebf6674

25 Mar, 2025 1 commit

[core] add bucket padding to tpu_model_runner (#14995) · ac3cd6e8

Chenyaaang authored Mar 25, 2025


Signed-off-by: Chenyaaang <llccyy1212@gmail.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>

ac3cd6e8

24 Mar, 2025 3 commits

[ROCm][Kernel] MoE weights padding (#14454) · f533b583

Gregory Shtrasberg authored Mar 24, 2025


Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: charlifu <charlifu@amd.com>

f533b583

[Doc] Update docs on handling OOM (#15357) · 6dd55af6

Cyrus Leung authored Mar 25, 2025


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

6dd55af6

[Core] Don't force uppercase for VLLM_LOGGING_LEVEL (#15306) · 8abe69b4
Russell Bryant authored Mar 24, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
8abe69b4

23 Mar, 2025 1 commit
- [Bugfix] consider related env vars for torch.compiled cache hash (#14953) · 6ebaf9ac
  DefTruth authored Mar 23, 2025
```
Signed-off-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
```
  6ebaf9ac
22 Mar, 2025 2 commits
- Remove openvino support in favor of external plugin (#15339) · b877031d
  Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  b877031d
- [FEAT] [ROCm]: Add AITER RMS Norm (Layer Norm) Feature (#14959) · ec870fba
  TJian authored Mar 22, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  ec870fba
21 Mar, 2025 2 commits
- [Hardware][TPU] Add check for no additional graph compilation during runtime (#14710) · b15fd2be
  Siyuan Liu authored Mar 20, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  b15fd2be
- [V1][TPU] Speed up top-k on TPU by using torch.topk (#15242) · 47195057
  Hyesoo Yang authored Mar 20, 2025
```
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
```
  47195057
20 Mar, 2025 1 commit
- [Attention] Flash Attention 3 - fp8 (#14570) · a597a575
  Mickaël Seznec authored Mar 20, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
```
  a597a575
18 Mar, 2025 1 commit
- [V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels (#14930) · 99abb8b6
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  99abb8b6
15 Mar, 2025 4 commits

[VLM] Limit multimodal input cache by memory (#14805) · 3556a414
Cyrus Leung authored Mar 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
3556a414
[Attention] Get rid of mla cache alignment (#14842) · 5952d8ab
Lucas Wilkinson authored Mar 15, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
5952d8ab

[V1] V1 Enablement Oracle (#13726) · d4d93db2

Robert Shaw authored Mar 15, 2025


Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

d4d93db2

Disable outlines cache by default (#14837) · 776dcec8
Russell Bryant authored Mar 14, 2025

776dcec8

14 Mar, 2025 2 commits
- [Attention] MLA get rid of materialization (#14770) · 9532c498
  Lucas Wilkinson authored Mar 14, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  9532c498
- [Bugfix][IPEX] Add `VLLM_CPU_MOE_PREPACK` to allow disabling MoE prepack when... · 95d680b8
  Thien Tran authored Mar 14, 2025
```
[Bugfix][IPEX] Add `VLLM_CPU_MOE_PREPACK` to allow disabling MoE prepack when CPU does not support it (#14681)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
  95d680b8
10 Mar, 2025 1 commit
- Correct capitalisation: `VLLM` -> `vLLM` (#14562) · 3b352a2f
  Harry Mellor authored Mar 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  3b352a2f
07 Mar, 2025 1 commit
- [Kernel] optimize performance of gptq marlin kernel when n is small (#14138) · d0feea31
  Jinzhen Lin authored Mar 08, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  d0feea31
06 Mar, 2025 1 commit
- [Distributed] Add enable_expert_parallel arg (#14305) · cc2f9b32
  Tyler Michael Smith authored Mar 06, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  cc2f9b32
05 Mar, 2025 1 commit
- [misc] Add FlashMLA as a new option of VLLM_ATTENTION_BACKEND env (#14267) · 1b7624bf
  Serena authored Mar 06, 2025
  
  1b7624bf
03 Mar, 2025 2 commits
- [Build] Make sure local main branch is synced when VLLM_USE_PRECOMPILED=1 (#13921) · f35f8e22
  Cody Yu authored Mar 03, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  f35f8e22
- Update deprecated Python 3.8 typing (#13971) · cf069aa8
  Harry Mellor authored Mar 03, 2025
  
  cf069aa8
27 Feb, 2025 1 commit
- [misc] Rename Ray ADAG to Compiled Graph (#13928) · c9944acb
  Rui Qiao authored Feb 26, 2025
  
  c9944acb
24 Feb, 2025 1 commit
- Expert Parallelism (EP) Support for DeepSeek V2 (#12583) · 781096e3
  Jongseok Park authored Feb 24, 2025
  
  781096e3
23 Feb, 2025 1 commit
- [ci] Use env var to control whether to use S3 bucket in CI (#13634) · 2c5e637b
  Kevin H. Luu authored Feb 22, 2025
  
  2c5e637b
22 Feb, 2025 3 commits
- [Bugfix] Fix boolean conversion for OpenVINO env variable (#13615) · 382f66fb
  Helena Kloosterman authored Feb 22, 2025
  
  382f66fb
- [ROCm] Apply FP8 weights padding to values not divisible by 512 bytes on ROCm (#13231) · c904fddd
  Gregory Shtrasberg authored Feb 22, 2025
  
  c904fddd
- [core] set up data parallel communication (#13591) · 3e472d88
  youkaichao authored Feb 22, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3e472d88