Commits · e0c80c12532fdf7f084972c26229b6fc9de62c40 · OpenDAS / vllm_cscc

07 May, 2025 7 commits
- [fix]修复cudagraph和eager分段模式下，开启mla后报错问题 · e0c80c12
  王敏 authored May 07, 2025
  
  e0c80c12
- Merge branch 'v0.7.2-dev' into v0.7.2-fusion · f98420b4
  zhuwenwen authored May 07, 2025
  
  f98420b4
- update requirements-common.txt · 5e88530d
  zhuwenwen authored May 07, 2025
  
  5e88530d
- update numpy version · 6f40d21d
  zhuwenwen authored May 07, 2025
  
  6f40d21d
- Merge branch 'v0.7.2-dev' into v0.7.2-fusion · ffa31925
  zhuwenwen authored May 07, 2025
  
  ffa31925
- release opt1 version · a9267f52
  zhuwenwen authored May 07, 2025
  
  a9267f52
- Merge branch 'v0.7.2-dev-wm' into 'v0.7.2-dev' · c7cd5a9c
  zhuwenwen authored May 07, 2025
```
[fix]修复并行解码多机推理报错

See merge request dcutoolkit/deeplearing/vllm!107
```
  c7cd5a9c
06 May, 2025 1 commit
- [fix]修复并行解码多机推理报错 · 0858880a
  王敏 authored May 06, 2025
  
  0858880a
29 Apr, 2025 6 commits
- 需改mtp 16卡性能差问题 · 5e77e9b1
  xiabo authored Apr 29, 2025
  
  5e77e9b1
- Merge remote-tracking branch 'origin/v0.7.2-dev-zero-overhead' into v0.7.2-dev · 3812059e
  zhuwenwen authored Apr 29, 2025
  
  3812059e
- debug and fix error · 333e3374
  lizhigong authored Apr 29, 2025
  
  333e3374
- fix，去掉对base类的耦合 · 3bbb6e9d
  lizhigong authored Apr 29, 2025
  
  3bbb6e9d
- remove triton_key of benchmarks · 07bcd2d4
  zhuwenwen authored Apr 29, 2025
  
  07bcd2d4
- add triton_key · 62920e37
  zhuwenwen authored Apr 29, 2025
  
  62920e37
27 Apr, 2025 3 commits
- Merge branch 'v0.7.2-dev' into v0.7.2-fusion · 9e813a0e
  zhuwenwen authored Apr 27, 2025
  
  9e813a0e
- resolve the issue of requiring HIP_VISIBLEDEVICES to be set in order to run multiple cards · abf008ef
  zhuwenwen authored Apr 27, 2025
  
  abf008ef
- update version · 469a3a88
  zhuwenwen authored Apr 27, 2025
  
  469a3a88
24 Apr, 2025 2 commits
- update utils.py · f51086de
  zhuwenwen authored Apr 24, 2025
  
  f51086de
- fix llama and qwen layout · db94f061
  zhuwenwen authored Apr 24, 2025
  
  db94f061
23 Apr, 2025 1 commit
- fix nccl 报超时的问题，原因是双线程调度导致提前关闭远端worker，warmup后远端worker执行异常，引发rank0超时。 · 14f46a65
  lizhg1 authored Apr 23, 2025
```
http://hpczentao.sugon.com/bug-view-93029.html
```
  14f46a65
22 Apr, 2025 2 commits
- Merge branch 'v0.7.2-dev-wm' into 'v0.7.2-dev' · ba2ca2db
  zhuwenwen authored Apr 22, 2025
```
[fix]修复开并行解码后，speculative-disable-by-batch-size设的比测试的batch小的话可能出现的数组越界问题

See merge request dcutoolkit/deeplearing/vllm!99
```
  ba2ca2db
- [fix]修复开并行解码后，speculative-disable-by-batch-size设的比测试的batch小的话可能出现的数组越界问题 · 8776c63c
  王敏 authored Apr 22, 2025
  
  8776c63c
18 Apr, 2025 3 commits
- Merge branch 'v0.7.2-dev' into v0.7.2-fusion · f44e9f9e
  zhuwenwen authored Apr 18, 2025
  
  f44e9f9e
- deepseek_v3/r1 int8 量化首字调优 · 8fc15e04
  gaoqiong authored Apr 18, 2025
  
  8fc15e04
- Merge branch 'v0.7.2-dev-wm' into 'v0.7.2-dev' · f5f9f42f
  zhuwenwen authored Apr 18, 2025
```
[fix]修复开启并行解码后，在极端测试情况下服务无响应问题

See merge request dcutoolkit/deeplearing/vllm!97
```
  f5f9f42f
17 Apr, 2025 4 commits
- [fix]修复开启并行解码后，在极端测试情况下，由于设置了speculative-disable-by-batch-size导致不跑并行解码导致previo... · 7a23da92
  王敏 authored Apr 17, 2025
```
[fix]修复开启并行解码后，在极端测试情况下，由于设置了speculative-disable-by-batch-size导致不跑并行解码导致previous_hidden_states不断增加，最终导致显存用尽服务无响应问题
```
  7a23da92
- Merge branch 'v0.7.2-custom-eco' into 'v0.7.2-dev' · f5f65d24
  zhuwenwen authored Apr 17, 2025
```
add K100AI custom allreduce

See merge request dcutoolkit/deeplearing/vllm!96
```
  f5f65d24
- add K100AI custom allreduce · ec078ef1
  xiabo authored Apr 17, 2025
  
  ec078ef1
- Merge remote-tracking branch 'origin/v0.7.2_zero_overhead' into v0.7.2-dev · 645e9ec4
  zhuwenwen authored Apr 17, 2025
  
  645e9ec4
15 Apr, 2025 5 commits
- Merge branch 'v0.7.2-dev-wm' into 'v0.7.2-dev' · d0de006f
  zhuwenwen authored Apr 15, 2025
```
[feat]添加VLLM_ENFORCE_EAGER_BS_THRESHOLD环境变量，支持cudagraph模式下，当bs大于阈值时，强制切换为eager模式，对大bs有效果

See merge request dcutoolkit/deeplearing/vllm!95
```
  d0de006f
- [feat]添加VLLM_ENFORCE_EAGER_BS_THRESHOLD环境变量，支持cudagraph模式下，当bs大于阈值时，强制切换为eager模式，对大bs有效果 · 3ce3e3d9
  王敏 authored Apr 15, 2025
  
  3ce3e3d9
- back to mla v2 · 6f49c1ed
  zhuwenwen authored Apr 15, 2025
  
  6f49c1ed
- vdim pad 32 · cf28e5a4
  zhuwenwen authored Apr 15, 2025
  
  cf28e5a4
- set vdim=128 · 9c3190d0
  zhuwenwen authored Apr 15, 2025
  
  9c3190d0
14 Apr, 2025 3 commits
- support numa bind · ec2e17d8
  zhuwenwen authored Apr 14, 2025
  
  ec2e17d8
- update vdim pad · 1d57ec3d
  zhuwenwen authored Apr 14, 2025
  
  1d57ec3d
- update prefix cache interface and vdim padding strategy · 4564b2f5
  zhuwenwen authored Apr 14, 2025
  
  4564b2f5
11 Apr, 2025 1 commit
- fix glm register error · b99b5676
  zhuwenwen authored Apr 11, 2025
  
  b99b5676
10 Apr, 2025 2 commits
- add auto finish thread when use LLM object · c78f6594
  lizhigong authored Apr 10, 2025
  
  c78f6594
- support vdim 128 · b62fe3c1
  zhuwenwen authored Apr 10, 2025
  
  b62fe3c1