Commits · aa906d9876df404fdc5a524ecf8ed87ad9a6dd87 · OpenDAS / vllm_cscc

12 Jun, 2025 1 commit
- add VLLM_TBO_DECODE_BS to support and setting the min bs on tbo decode · aa906d98
  lizhigong authored Jun 12, 2025
  
  aa906d98
29 May, 2025 1 commit
- add delay setting in server, sleep and waiting more requests to merge in one batch · 0c5b1695
  lizhigong authored May 29, 2025
  
  0c5b1695
28 May, 2025 2 commits
- [feat]量化模型添加支持 moe_fused_gate kernel，并使用VLLM_ENABLE_MOE_FUSED_GATE环境变量控制开关，默认打开 · 1016e339
  王敏 authored May 28, 2025
  
  1016e339
- set VLLM_FLASH_ATTN_BACKEND to use FlashAttention Backend for attention computation on rocm · 82f60bef
  zhuwenwen authored May 28, 2025
  
  82f60bef
23 May, 2025 2 commits
- tbo增加ds int8支持，修改tb zerooverhead环境变量到envs.py中统一管理 · dbbb148b
  lizhigong authored May 23, 2025
  
  dbbb148b
- update setup.py · cb563bb5
  zhuwenwen authored May 23, 2025
  
  cb563bb5
22 May, 2025 1 commit
- 修改MLA prefill阶段出现的Device2Host拷贝同步现象 · 0c1fa562
  zhuwenwen authored May 22, 2025
  
  0c1fa562
13 May, 2025 1 commit
- support cutlass prefix-cache · bdc70899
  zhuwenwen authored May 13, 2025
  
  bdc70899
08 May, 2025 2 commits
- set VLLM_USE_V1=0 · 3d83fa41
  zhuwenwen authored May 08, 2025
  
  3d83fa41
- set VLLM_ROCM_MOE_PADDING=0 · ee405280
  zhuwenwen authored May 08, 2025
  
  ee405280
24 Apr, 2025 1 commit
- Addendum Fix to support FIPS enabled machines with MD5 hashing (#17043) · ed2e4646
  Areeb Syed authored Apr 24, 2025
```
Signed-off-by: sydarb <areebsyed237@gmail.com>
```
  ed2e4646
23 Apr, 2025 1 commit

[INTEL-HPU][v0] Port delayed sampling to upstream (#16949) · 56a73526

Chendi.Xue authored Apr 22, 2025


Signed-off-by: Michal Adamczyk <michal.adamczyk@intel.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>

56a73526

22 Apr, 2025 4 commits

[FEAT][ROCm]: Support AITER MLA (#15893) · 30bc3e0f

vllmellm authored Apr 23, 2025


Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>

30bc3e0f

[FEAT][ROCm] Integrate Paged Attention Kernel from AITER (#15001) · 0e237f00

vllmellm authored Apr 22, 2025


Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>

0e237f00

[Performance][ROCm] Add skinny gemms for unquantized linear on ROCm (#15830) · 188b7f9b
Charlie Fu authored Apr 21, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
```
188b7f9b

[ROCm] Add aiter tkw1 kernel for Llama4 fp8 (#16727) · 5b794cae

kliuae authored Apr 22, 2025


Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>

5b794cae

19 Apr, 2025 1 commit

[Model][VLM] Add Qwen2.5-Omni model support (thinker only) (#15130) · 2c1bd848

Yang Fan authored Apr 19, 2025


Signed-off-by: fyabc <suyang.fy@alibaba-inc.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Xiong Wang <wangxiongts@163.com>

2c1bd848

18 Apr, 2025 1 commit
- add K100AI custom allreduce · b0eacb5b
  zhuwenwen authored Apr 18, 2025
  
  b0eacb5b
17 Apr, 2025 2 commits

[V1][TPU] Enable Top K (#15489) · eb5819b2

Nicolò Lucchesi authored Apr 17, 2025


Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: Hyesoo Yang <hyeygit@gmail.com>

eb5819b2

[V1][Performance] Implement custom serializaton for MultiModalKwargs [Rebased] (#16432) · 3092375e

Staszek Paśko authored Apr 17, 2025


Signed-off-by: Staszek Pasko <staszek@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

3092375e

11 Apr, 2025 1 commit
- add VLLM_USE_FLASH_MLA to use flashmla backend · a0e22db9
  zhuwenwen authored Apr 11, 2025
  
  a0e22db9
09 Apr, 2025 1 commit
- [Core] Upgrade to xgrammar 0.1.18, add cache size limit (#16283) · cb84e45a
  Russell Bryant authored Apr 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  cb84e45a
06 Apr, 2025 1 commit
- [Misc] Improve model redirect to accept json dictionary (#16119) · c2a96715
  Isotr0py authored Apr 06, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c2a96715
01 Apr, 2025 2 commits
- Add option to use DeepGemm contiguous grouped gemm kernel for fused MoE operations. (#13932) · e59ca942
  bnellnm authored Apr 01, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  e59ca942
- [Misc] Use envs.VLLM_USE_RAY_COMPILED_DAG_CHANNEL_TYPE (#15831) · 8dd41d6b
  Rui Qiao authored Apr 01, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8dd41d6b
31 Mar, 2025 1 commit
- skip fp8 kernels and paged_attention_rocm · 52675626
  zhuwenwen authored Mar 31, 2025
  
  52675626
29 Mar, 2025 1 commit
- [FEAT] [ROCm] Add AITER int8 scaled gemm kernel (#15433) · 4965ec42
  TJian authored Mar 29, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  4965ec42
28 Mar, 2025 1 commit
- [feat]添加VLLM_SPEC_DECODE_EAGER环境变量，用于选择draft model是否强制使用eager模式，在hygon cpu上ds3 mtp提升较大 · 87223113
  zhuwenwen authored Mar 28, 2025
  
  87223113
27 Mar, 2025 4 commits
- [V1] AsyncLLM data parallel (#13923) · 15dac210
  Nick Hill authored Mar 27, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  15dac210
- [Feature] Add middleware to log API Server responses (#15593) · 66aa4c0b
  Yuan Tang authored Mar 27, 2025
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  66aa4c0b
- [Misc] Use model_redirect to redirect the model name to a local folder. (#14116) · 3f532cb6
  wang.yuqi authored Mar 27, 2025
  
  3f532cb6
- [ROCm] Env variable to trigger custom PA (#15557) · ecff8309
  Gregory Shtrasberg authored Mar 27, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  ecff8309
26 Mar, 2025 2 commits

[V1] TPU - Revert to exponential padding by default (#15565) · b2e85e26
Alexander Matveev authored Mar 26, 2025
```
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
```
b2e85e26

[FEAT][ROCm] Integrate Fused MoE Kernels from AITER (#14967) · 5ebf6674

vllmellm authored Mar 26, 2025


Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>

5ebf6674

25 Mar, 2025 1 commit

[core] add bucket padding to tpu_model_runner (#14995) · ac3cd6e8

Chenyaaang authored Mar 25, 2025


Signed-off-by: Chenyaaang <llccyy1212@gmail.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>

ac3cd6e8

24 Mar, 2025 3 commits

[ROCm][Kernel] MoE weights padding (#14454) · f533b583

Gregory Shtrasberg authored Mar 24, 2025


Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: charlifu <charlifu@amd.com>

f533b583

[Doc] Update docs on handling OOM (#15357) · 6dd55af6

Cyrus Leung authored Mar 25, 2025


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

6dd55af6

[Core] Don't force uppercase for VLLM_LOGGING_LEVEL (#15306) · 8abe69b4
Russell Bryant authored Mar 24, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
8abe69b4

23 Mar, 2025 1 commit
- [Bugfix] consider related env vars for torch.compiled cache hash (#14953) · 6ebaf9ac
  DefTruth authored Mar 23, 2025
```
Signed-off-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
```
  6ebaf9ac
22 Mar, 2025 1 commit
- Remove openvino support in favor of external plugin (#15339) · b877031d
  Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  b877031d