Commits · 61898ea3c5283fa488649160369ea00b77b9bb97 · OpenDAS / vllm_cscc · GitLab

24 Apr, 2025 2 commits
- fix llama and qwen layout · 61898ea3
  zhuwenwen authored Apr 24, 2025
  
  61898ea3
- [fix]修复多卡eager模式精度问题 · 9f29bc2d
  王敏 authored Apr 24, 2025
  
  9f29bc2d
23 Apr, 2025 1 commit
- 更新curr_topk_ids · f9a784a7
  yangql authored Apr 23, 2025
  
  f9a784a7
22 Apr, 2025 4 commits
- 修复ds3 int8调用的功能 · cd87548a
  gaoqiong authored Apr 22, 2025
  
  cd87548a
- [fix]解决无法启动并行报错 · 3e1f35bb
  王敏 authored Apr 22, 2025
  
  3e1f35bb
- [Bugfix] Fix GLM rotary_dim issue and support v1 · 7f3dec12
  zhuwenwen authored Apr 22, 2025
```
Fix glm4.py residual bug
```
  7f3dec12
- [fix]修复开并行解码后，speculative-disable-by-batch-size设的比测试的batch小的话可能出现的数组越界问题 · b956dfd6
  zhuwenwen authored Apr 22, 2025
  
  b956dfd6
19 Apr, 2025 1 commit
- [Bugfix] Fix GLM4 model · 9e14887f
  zhuwenwen authored Apr 19, 2025
  
  9e14887f
18 Apr, 2025 6 commits
- update utils.py · d1ea1295
  zhuwenwen authored Apr 18, 2025
  
  d1ea1295
- Revert "update fused_moe.py" · 606e3c92
  zhuwenwen authored Apr 18, 2025
```
This reverts commit 95b18847.
```
  606e3c92
- update fused_moe.py · 95b18847
  zhuwenwen authored Apr 18, 2025
  
  95b18847
- deepseek_v3/r1 int8 量化首字调优 · 777d5f95
  zhuwenwen authored Apr 18, 2025
  
  777d5f95
- add K100AI custom allreduce · b0eacb5b
  zhuwenwen authored Apr 18, 2025
  
  b0eacb5b
- [fix]修复开启并行解码后，在极端测试情况下，由于设置了speculative-disable-by-batch-size导致不跑并行解码导致previo... · 58fc3e31
  zhuwenwen authored Apr 18, 2025
```
[fix]修复开启并行解码后，在极端测试情况下，由于设置了speculative-disable-by-batch-size导致不跑并行解码导致previous_hidden_states不断增加，最终导致显存用尽服务无响应问题
```
  58fc3e31
17 Apr, 2025 1 commit
- update arg_utils.py · fdc44c0a
  zhuwenwen authored Apr 17, 2025
  
  fdc44c0a
16 Apr, 2025 1 commit
- update moe models of LM_NN · 2db90e2c
  zhuwenwen authored Apr 16, 2025
  
  2db90e2c
14 Apr, 2025 5 commits
- support numa bind · e8933c34
  zhuwenwen authored Apr 14, 2025
  
  e8933c34
- remove unused code · 641fc5b7
  zhuwenwen authored Apr 14, 2025
  
  641fc5b7
- remove unused mla utils.py · b01efa0b
  zhuwenwen authored Apr 14, 2025
  
  b01efa0b
- update vdim padding strategy · 4a19cdf5
  zhuwenwen authored Apr 14, 2025
  
  4a19cdf5
- [Core][V0] Enable regex support with xgrammar (#13228) · dc1b4a6f
  Russell Bryant authored Apr 13, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  dc1b4a6f
13 Apr, 2025 2 commits
- Enable PTPC FP8 for CompressedTensorsW8A8Fp8MoEMethod (triton fused_moe) (#16537) · d085a440
  Michael Goin authored Apr 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  d085a440
- [V1][Spec Decode] KV cache slots for eagle heads (#16370) · f49e5aff
  Lily Liu authored Apr 12, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  f49e5aff
12 Apr, 2025 10 commits
- [Bugfix] Validate logit biases to prevent out of vocab ids crashing engine (#16529) · 6c11ecf8
  Ryan McConville authored Apr 12, 2025
```
Signed-off-by: Ryan McConville <ryan@ryanmcconville.com>
```
  6c11ecf8
- [Perf] Optimize Preparing Inputs for GPU Model Runner (#16484) · 93e5f3c5
  SnowCharm authored Apr 12, 2025
```
Signed-off-by: snowcharm <snowcharmqq@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  93e5f3c5
- Fix syntaxWarning: invalid escape sequence '\s' (#16532) · 70363bcc
  Jie Fu (傅杰) authored Apr 12, 2025
```
Signed-off-by: Jie Fu <jiefu@tencent.com>
```
  70363bcc
- [MISC] Make GroupCoordinator compatible with out-of-tree devices (#16464) · 68bb122e
  Huazhong Ji authored Apr 12, 2025
```
Signed-off-by: hzji210@gmail.com <hzji210@gmail.com>
```
  68bb122e
- [V1] Enable multi-input by default (#15799) · d9fc8cd9
  Cyrus Leung authored Apr 12, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d9fc8cd9
- [Frontend] support matryoshka representation / support embedding API dimensions (#16331) · fbf722c6
  wang.yuqi authored Apr 12, 2025
  
  fbf722c6
- [Feature][V1] Add xgrammar to support minLength, maxLength with test (#16516) · e92d7085
  leon-seidel authored Apr 12, 2025
```
Signed-off-by: Leon Seidel <leon.seidel@fau.de>
```
  e92d7085
- Optimized topk for topk=1 (Llama-4) (#16512) · bd6028d6
  Michael Goin authored Apr 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  bd6028d6
- set MOE_NN to default value of 0 · 155d41d1
  zhuwenwen authored Apr 12, 2025
  
  155d41d1
- [BugFix] Handle non-contiguous tensors properly when serializing (#16492) · 41cc883c
  Nick Hill authored Apr 11, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  41cc883c
11 Apr, 2025 7 commits
- Bugfix for PixtralHF models without spatial_merge_size (#16513) · 87b836ba
  Michael Goin authored Apr 11, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  87b836ba
- [Bugfix] clean up duplicated code (#16485) · 56c76c2e
  rongfu.leng authored Apr 12, 2025
```
Signed-off-by: Gogs <gogs@fake.local>
Co-authored-by: Gogs <gogs@fake.local>
```
  56c76c2e
- [Kernel] Add tuned FusedMoE kernel config for Llama4 Scout, TP=8 on H100 (#16488) · a3bf8d4a
  Yong Hoon Shin authored Apr 11, 2025
  
  a3bf8d4a
- [Frontend] Added chat templates for LLaMa4 pythonic tool calling (#16463) · 16eda8c4
  Ye (Charlotte) Qi authored Apr 11, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Co-authored-by: Kai Wu <kaiwu@meta.com>
```
  16eda8c4
- Improve configs - `LoadConfig` (#16422) · cd77382a
  Harry Mellor authored Apr 11, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  cd77382a
- [Bugfix] handle alignment of encoder_seq_lens in mllama.py (#14784) · 71b9cde0
  Travis Johnson authored Apr 11, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  71b9cde0
- [Kernel] Support W8A8 channel-wise weights and per-token activations in triton... · f41647ee
  Michael Goin authored Apr 11, 2025
```
[Kernel] Support W8A8 channel-wise weights and per-token activations in triton fused_moe_kernel (#16366)
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f41647ee