Commits · 3357c074e01e7da7be034827aefa6f2662673c74 · OpenDAS / vllm_cscc · GitLab

19 Apr, 2025 1 commit
- [Bugfix] Fix GLM4 model · 9e14887f
  zhuwenwen authored Apr 19, 2025
  
  9e14887f
18 Apr, 2025 6 commits
- update utils.py · d1ea1295
  zhuwenwen authored Apr 18, 2025
  
  d1ea1295
- Revert "update fused_moe.py" · 606e3c92
  zhuwenwen authored Apr 18, 2025
```
This reverts commit 95b18847.
```
  606e3c92
- update fused_moe.py · 95b18847
  zhuwenwen authored Apr 18, 2025
  
  95b18847
- deepseek_v3/r1 int8 量化首字调优 · 777d5f95
  zhuwenwen authored Apr 18, 2025
  
  777d5f95
- add K100AI custom allreduce · b0eacb5b
  zhuwenwen authored Apr 18, 2025
  
  b0eacb5b
- [fix]修复开启并行解码后，在极端测试情况下，由于设置了speculative-disable-by-batch-size导致不跑并行解码导致previo... · 58fc3e31
  zhuwenwen authored Apr 18, 2025
```
[fix]修复开启并行解码后，在极端测试情况下，由于设置了speculative-disable-by-batch-size导致不跑并行解码导致previous_hidden_states不断增加，最终导致显存用尽服务无响应问题
```
  58fc3e31
17 Apr, 2025 1 commit
- update arg_utils.py · fdc44c0a
  zhuwenwen authored Apr 17, 2025
  
  fdc44c0a
16 Apr, 2025 1 commit
- update moe models of LM_NN · 2db90e2c
  zhuwenwen authored Apr 16, 2025
  
  2db90e2c
14 Apr, 2025 5 commits
- support numa bind · e8933c34
  zhuwenwen authored Apr 14, 2025
  
  e8933c34
- remove unused code · 641fc5b7
  zhuwenwen authored Apr 14, 2025
  
  641fc5b7
- remove unused mla utils.py · b01efa0b
  zhuwenwen authored Apr 14, 2025
  
  b01efa0b
- update vdim padding strategy · 4a19cdf5
  zhuwenwen authored Apr 14, 2025
  
  4a19cdf5
- [Core][V0] Enable regex support with xgrammar (#13228) · dc1b4a6f
  Russell Bryant authored Apr 13, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  dc1b4a6f
13 Apr, 2025 2 commits
- Enable PTPC FP8 for CompressedTensorsW8A8Fp8MoEMethod (triton fused_moe) (#16537) · d085a440
  Michael Goin authored Apr 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  d085a440
- [V1][Spec Decode] KV cache slots for eagle heads (#16370) · f49e5aff
  Lily Liu authored Apr 12, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  f49e5aff
12 Apr, 2025 10 commits
- [Bugfix] Validate logit biases to prevent out of vocab ids crashing engine (#16529) · 6c11ecf8
  Ryan McConville authored Apr 12, 2025
```
Signed-off-by: Ryan McConville <ryan@ryanmcconville.com>
```
  6c11ecf8
- [Perf] Optimize Preparing Inputs for GPU Model Runner (#16484) · 93e5f3c5
  SnowCharm authored Apr 12, 2025
```
Signed-off-by: snowcharm <snowcharmqq@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  93e5f3c5
- Fix syntaxWarning: invalid escape sequence '\s' (#16532) · 70363bcc
  Jie Fu (傅杰) authored Apr 12, 2025
```
Signed-off-by: Jie Fu <jiefu@tencent.com>
```
  70363bcc
- [MISC] Make GroupCoordinator compatible with out-of-tree devices (#16464) · 68bb122e
  Huazhong Ji authored Apr 12, 2025
```
Signed-off-by: hzji210@gmail.com <hzji210@gmail.com>
```
  68bb122e
- [V1] Enable multi-input by default (#15799) · d9fc8cd9
  Cyrus Leung authored Apr 12, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d9fc8cd9
- [Frontend] support matryoshka representation / support embedding API dimensions (#16331) · fbf722c6
  wang.yuqi authored Apr 12, 2025
  
  fbf722c6
- [Feature][V1] Add xgrammar to support minLength, maxLength with test (#16516) · e92d7085
  leon-seidel authored Apr 12, 2025
```
Signed-off-by: Leon Seidel <leon.seidel@fau.de>
```
  e92d7085
- Optimized topk for topk=1 (Llama-4) (#16512) · bd6028d6
  Michael Goin authored Apr 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  bd6028d6
- set MOE_NN to default value of 0 · 155d41d1
  zhuwenwen authored Apr 12, 2025
  
  155d41d1
- [BugFix] Handle non-contiguous tensors properly when serializing (#16492) · 41cc883c
  Nick Hill authored Apr 11, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  41cc883c
11 Apr, 2025 14 commits
- Bugfix for PixtralHF models without spatial_merge_size (#16513) · 87b836ba
  Michael Goin authored Apr 11, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  87b836ba
- [Bugfix] clean up duplicated code (#16485) · 56c76c2e
  rongfu.leng authored Apr 12, 2025
```
Signed-off-by: Gogs <gogs@fake.local>
Co-authored-by: Gogs <gogs@fake.local>
```
  56c76c2e
- [Kernel] Add tuned FusedMoE kernel config for Llama4 Scout, TP=8 on H100 (#16488) · a3bf8d4a
  Yong Hoon Shin authored Apr 11, 2025
  
  a3bf8d4a
- [Frontend] Added chat templates for LLaMa4 pythonic tool calling (#16463) · 16eda8c4
  Ye (Charlotte) Qi authored Apr 11, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Co-authored-by: Kai Wu <kaiwu@meta.com>
```
  16eda8c4
- Improve configs - `LoadConfig` (#16422) · cd77382a
  Harry Mellor authored Apr 11, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  cd77382a
- [Bugfix] handle alignment of encoder_seq_lens in mllama.py (#14784) · 71b9cde0
  Travis Johnson authored Apr 11, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  71b9cde0
- [Kernel] Support W8A8 channel-wise weights and per-token activations in triton... · f41647ee
  Michael Goin authored Apr 11, 2025
```
[Kernel] Support W8A8 channel-wise weights and per-token activations in triton fused_moe_kernel (#16366)
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f41647ee
- [TPU][V1] Make `--disable_chunked_mm_input` mandatory for serving MM models (#16483) · 4d022cbc
  Nicolò Lucchesi authored Apr 11, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  4d022cbc
- Fix erroneous "model doesn't support compile" warning (#16486) · 70de35a8
  Richard Zou authored Apr 11, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  70de35a8
- [Hardware][Intel-Gaudi] Multi-step scheduling implementation for HPU (#12779) · 34b2cf3b
  Tomasz Zielinski authored Apr 11, 2025
```
Signed-off-by: Tomasz Zielinski <tomasz.zielinski@intel.com>
```
  34b2cf3b
- [Bugfix] Fix bugs of running Quark quantized models (#16236) · 9e90c9f7
  chaow-amd authored Apr 11, 2025
```
Signed-off-by: chaow <chaow@amd.com>
```
  9e90c9f7
- [Kernel] support merge_attn_states CUDA kernel, 3x speedup (#16173) · e9528f6d
  DefTruth authored Apr 11, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  e9528f6d
- [Misc] Raise error for V1 not supporting Long LoRA. (#16415) · a26f59cc
  Jee Jee Li authored Apr 11, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a26f59cc
- support v1 engine · e19ddc39
  zhuwenwen authored Apr 11, 2025
  
  e19ddc39