Commits · 7a23da927a8436302362c7f4ad01a5f85432caad · OpenDAS / vllm_cscc

17 Apr, 2025 1 commit

[fix]修复开启并行解码后，在极端测试情况下，由于设置了speculative-disable-by-batch-size导致不跑并行解码导致previo... · 7a23da92

王敏 authored Apr 17, 2025

[fix]修复开启并行解码后，在极端测试情况下，由于设置了speculative-disable-by-batch-size导致不跑并行解码导致previous_hidden_states不断增加，最终导致显存用尽服务无响应问题

7a23da92

15 Apr, 2025 5 commits
- Merge branch 'v0.7.2-dev-wm' into 'v0.7.2-dev' · d0de006f
  zhuwenwen authored Apr 15, 2025
```
[feat]添加VLLM_ENFORCE_EAGER_BS_THRESHOLD环境变量，支持cudagraph模式下，当bs大于阈值时，强制切换为eager模式，对大bs有效果

See merge request dcutoolkit/deeplearing/vllm!95
```
  d0de006f
- [feat]添加VLLM_ENFORCE_EAGER_BS_THRESHOLD环境变量，支持cudagraph模式下，当bs大于阈值时，强制切换为eager模式，对大bs有效果 · 3ce3e3d9
  王敏 authored Apr 15, 2025
  
  3ce3e3d9
- back to mla v2 · 6f49c1ed
  zhuwenwen authored Apr 15, 2025
  
  6f49c1ed
- vdim pad 32 · cf28e5a4
  zhuwenwen authored Apr 15, 2025
  
  cf28e5a4
- set vdim=128 · 9c3190d0
  zhuwenwen authored Apr 15, 2025
  
  9c3190d0
14 Apr, 2025 3 commits
- support numa bind · ec2e17d8
  zhuwenwen authored Apr 14, 2025
  
  ec2e17d8
- update vdim pad · 1d57ec3d
  zhuwenwen authored Apr 14, 2025
  
  1d57ec3d
- update prefix cache interface and vdim padding strategy · 4564b2f5
  zhuwenwen authored Apr 14, 2025
  
  4564b2f5
11 Apr, 2025 1 commit
- fix glm register error · b99b5676
  zhuwenwen authored Apr 11, 2025
  
  b99b5676
10 Apr, 2025 1 commit
- support vdim 128 · b62fe3c1
  zhuwenwen authored Apr 10, 2025
  
  b62fe3c1
09 Apr, 2025 1 commit
- update triton_decode_attention.py · 085c3454
  zhuwenwen authored Apr 09, 2025
  
  085c3454
08 Apr, 2025 1 commit
- update triton_decode_attention.py · 8b5a09f6
  zhuwenwen authored Apr 08, 2025
  
  8b5a09f6
07 Apr, 2025 1 commit
- update layout · f6044f1a
  zhuwenwen authored Apr 07, 2025
  
  f6044f1a
03 Apr, 2025 2 commits
- update triton_decode_attention.py · a30b3ce2
  zhuwenwen authored Apr 03, 2025
  
  a30b3ce2
- [update] MLA stage=2 并启动pipelinev2，修复pipeline subop 操作类型不一致 · 343a10fa
  zhuwenwen authored Apr 03, 2025
  
  343a10fa
02 Apr, 2025 1 commit
- Fix reasoning_content for chat_template include <think> tag as input · 2c35b6cd
  zhuwenwen authored Apr 02, 2025
  
  2c35b6cd
28 Mar, 2025 3 commits
- update beta version · 21833462
  zhuwenwen authored Mar 28, 2025
  
  21833462
- Merge branch 'v0.7.2-dev-wm' into 'v0.7.2-dev' · feeb058b
  zhuwenwen authored Mar 28, 2025
```
[feat]添加VLLM_SPEC_DECODE_EAGER环境变量，用于选择draft model是否强制使用eager模式，在hygon cpu上ds3 mtp提升较大

See merge request dcutoolkit/deeplearing/vllm!91
```
  feeb058b
- [feat]添加VLLM_SPEC_DECODE_EAGER环境变量，用于选择draft model是否强制使用eager模式，在hygon cpu上ds3 mtp提升较大 · 7488257b
  王敏 authored Mar 28, 2025
  
  7488257b
27 Mar, 2025 1 commit
- 更新bw以及k500sm_ai的支持 · 18ec9eaa
  yangql authored Mar 27, 2025
  
  18ec9eaa
26 Mar, 2025 8 commits
- Merge remote-tracking branch 'origin/v0.7.2-dev-quant' into v0.7.2-dev · d01a8fa8
  zhuwenwen authored Mar 26, 2025
  
  d01a8fa8
- update mla kernel and configs · e1600abd
  zhuwenwen authored Mar 26, 2025
  
  e1600abd
- 增加blockint8支持优化 · 5f2801b1
  gaoqiong authored Mar 26, 2025
  
  5f2801b1
- 增加blockint8支持优化 · 96bfa3ce
  gaoqiong authored Mar 26, 2025
  
  96bfa3ce
- 增加blockint8支持优化 · 5f15bdb5
  gaoqiong authored Mar 26, 2025
  
  5f15bdb5
- 增加blockint8支持优化 · f3deca99
  gaoqiong authored Mar 26, 2025
  
  f3deca99
- update custom_all_reduce · 5c241fa9
  zhuwenwen authored Mar 26, 2025
  
  5c241fa9
- [FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash... · 3b81dd6c
  zhuwenwen authored Mar 26, 2025
```
[FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash Attention to enable embedding models.
```
  3b81dd6c
25 Mar, 2025 2 commits
- Merge branch 'v0.7.2-dev-custom' into 'v0.7.2-dev' · b5a9a18d
  zhuwenwen authored Mar 25, 2025
```
V0.7.2 dev custom

See merge request dcutoolkit/deeplearing/vllm!90
```
  b5a9a18d
- add custom allreduce check · c8a63b38
  xiabo authored Mar 25, 2025
  
  c8a63b38
24 Mar, 2025 5 commits
- Merge branch 'v0.7.2-dev-wm' into 'v0.7.2-dev' · ae0ed592
  zhuwenwen authored Mar 24, 2025
```
[fix]修复fused_moe.py中fused_moe接口未初始化moe_ep_size导致的deekseek等模型报错

See merge request dcutoolkit/deeplearing/vllm!89
```
  ae0ed592
- [fix]修复fused_moe.py中fused_moe接口未初始化moe_ep_size导致的deekseek等模型报错 · 4b7f382f
  王敏 authored Mar 24, 2025
  
  4b7f382f
- update triton_mla.py · a54eca71
  zhuwenwen authored Mar 24, 2025
  
  a54eca71
- add mla tuning configs of k100-ai · 30e0b082
  zhuwenwen authored Mar 24, 2025
  
  30e0b082
- add mla tuning script and configs · 13b1dcfe
  zhuwenwen authored Mar 24, 2025
  
  13b1dcfe
22 Mar, 2025 1 commit
- add custom allreduce cudagraph · a8c92908
  xiabo authored Mar 22, 2025
  
  a8c92908
18 Mar, 2025 1 commit
- update benchmarks · b95d1275
  zhuwenwen authored Mar 18, 2025
  
  b95d1275
17 Mar, 2025 2 commits
- add custom allreduce · 2d6bccd9
  xiabo authored Mar 17, 2025
  
  2d6bccd9
- update version · d9e67e78
  zhuwenwen authored Mar 17, 2025
  
  d9e67e78