Commits · b0eacb5bb69b31913847296483af1e30a79d1a68 · OpenDAS / vllm_cscc · GitLab

18 Apr, 2025 2 commits
- add K100AI custom allreduce · b0eacb5b
  zhuwenwen authored Apr 18, 2025
  
  b0eacb5b
- [fix]修复开启并行解码后，在极端测试情况下，由于设置了speculative-disable-by-batch-size导致不跑并行解码导致previo... · 58fc3e31
  zhuwenwen authored Apr 18, 2025
```
[fix]修复开启并行解码后，在极端测试情况下，由于设置了speculative-disable-by-batch-size导致不跑并行解码导致previous_hidden_states不断增加，最终导致显存用尽服务无响应问题
```
  58fc3e31
17 Apr, 2025 1 commit
- update arg_utils.py · fdc44c0a
  zhuwenwen authored Apr 17, 2025
  
  fdc44c0a
16 Apr, 2025 2 commits
- update moe models of LM_NN · 2db90e2c
  zhuwenwen authored Apr 16, 2025
  
  2db90e2c
- Merge tag 'v0.8.4' into v0.8.4-dev · 31330101
  zhuwenwen authored Apr 16, 2025
  
  31330101
14 Apr, 2025 6 commits
- support numa bind · e8933c34
  zhuwenwen authored Apr 14, 2025
  
  e8933c34
- remove unused code · 641fc5b7
  zhuwenwen authored Apr 14, 2025
  
  641fc5b7
- remove unused mla utils.py · b01efa0b
  zhuwenwen authored Apr 14, 2025
  
  b01efa0b
- update vdim padding strategy · 4a19cdf5
  zhuwenwen authored Apr 14, 2025
  
  4a19cdf5
- [Core][V0] Enable regex support with xgrammar (#13228) · dc1b4a6f
  Russell Bryant authored Apr 13, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  dc1b4a6f
- [Benchmark][Bugfix] Fix SonnetDataset default values in benchmark_throughput.py (#16556) · 63d2705e
  Jennifer Zhao authored Apr 13, 2025
  
  63d2705e
13 Apr, 2025 2 commits
- Enable PTPC FP8 for CompressedTensorsW8A8Fp8MoEMethod (triton fused_moe) (#16537) · d085a440
  Michael Goin authored Apr 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  d085a440
- [V1][Spec Decode] KV cache slots for eagle heads (#16370) · f49e5aff
  Lily Liu authored Apr 12, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  f49e5aff
12 Apr, 2025 17 commits
- [Bugfix] Validate logit biases to prevent out of vocab ids crashing engine (#16529) · 6c11ecf8
  Ryan McConville authored Apr 12, 2025
```
Signed-off-by: Ryan McConville <ryan@ryanmcconville.com>
```
  6c11ecf8
- [Perf] Optimize Preparing Inputs for GPU Model Runner (#16484) · 93e5f3c5
  SnowCharm authored Apr 12, 2025
```
Signed-off-by: snowcharm <snowcharmqq@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  93e5f3c5
- Fix syntaxWarning: invalid escape sequence '\s' (#16532) · 70363bcc
  Jie Fu (傅杰) authored Apr 12, 2025
```
Signed-off-by: Jie Fu <jiefu@tencent.com>
```
  70363bcc
- [Misc] Delete redundant code (#16530) · 3cdc5766
  Jee Jee Li authored Apr 12, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  3cdc5766
- [MISC] Make GroupCoordinator compatible with out-of-tree devices (#16464) · 68bb122e
  Huazhong Ji authored Apr 12, 2025
```
Signed-off-by: hzji210@gmail.com <hzji210@gmail.com>
```
  68bb122e
- [V1] Enable multi-input by default (#15799) · d9fc8cd9
  Cyrus Leung authored Apr 12, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d9fc8cd9
- [Misc] Openai transcription client example use same Whisper model (#16487) · f069f3ea
  Nicolò Lucchesi authored Apr 12, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  f069f3ea
- [Misc] Update chat utils tests (#16520) · c5bc0e7f
  Cyrus Leung authored Apr 12, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c5bc0e7f
- fix: spelling (#16466) · 4a3a5187
  Tianer Zhou authored Apr 12, 2025
```
Signed-off-by: Tianer Zhou <ezhoureal@gmail.com>
```
  4a3a5187
- [Frontend] support matryoshka representation / support embedding API dimensions (#16331) · fbf722c6
  wang.yuqi authored Apr 12, 2025
  
  fbf722c6
- [Feature][V1] Add xgrammar to support minLength, maxLength with test (#16516) · e92d7085
  leon-seidel authored Apr 12, 2025
```
Signed-off-by: Leon Seidel <leon.seidel@fau.de>
```
  e92d7085
- Optimized topk for topk=1 (Llama-4) (#16512) · bd6028d6
  Michael Goin authored Apr 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  bd6028d6
- set MOE_NN to default value of 0 · 155d41d1
  zhuwenwen authored Apr 12, 2025
  
  155d41d1
- [Doc] Update Llama4 Model Names in Supported Models (#16509) · 802329de
  Ye (Charlotte) Qi authored Apr 11, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  802329de
- [BugFix] Handle non-contiguous tensors properly when serializing (#16492) · 41cc883c
  Nick Hill authored Apr 11, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  41cc883c
- [CI][Bugfix] Add mistral_tool_use to Ci (#16517) · 57504a4b
  Michael Goin authored Apr 11, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  57504a4b
- [Doc] Fix link to vLLM blog (#16519) · ed4792c9
  Yuan Tang authored Apr 11, 2025
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  ed4792c9
11 Apr, 2025 10 commits
- Bugfix for PixtralHF models without spatial_merge_size (#16513) · 87b836ba
  Michael Goin authored Apr 11, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  87b836ba
- [Bugfix] clean up duplicated code (#16485) · 56c76c2e
  rongfu.leng authored Apr 12, 2025
```
Signed-off-by: Gogs <gogs@fake.local>
Co-authored-by: Gogs <gogs@fake.local>
```
  56c76c2e
- Update openai_compatible_server.md (#16507) · c09632a6
  Christian Sears authored Apr 11, 2025
```
Signed-off-by: Christian Sears <csears@redhat.com>
```
  c09632a6
- [Kernel] Add tuned FusedMoE kernel config for Llama4 Scout, TP=8 on H100 (#16488) · a3bf8d4a
  Yong Hoon Shin authored Apr 11, 2025
  
  a3bf8d4a
- [Frontend] Added chat templates for LLaMa4 pythonic tool calling (#16463) · 16eda8c4
  Ye (Charlotte) Qi authored Apr 11, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Co-authored-by: Kai Wu <kaiwu@meta.com>
```
  16eda8c4
- Improve configs - `LoadConfig` (#16422) · cd77382a
  Harry Mellor authored Apr 11, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  cd77382a
- [Bugfix] handle alignment of encoder_seq_lens in mllama.py (#14784) · 71b9cde0
  Travis Johnson authored Apr 11, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  71b9cde0
- [Doc] Document InternVL3 support (#16495) · 5285589f
  Isotr0py authored Apr 12, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  5285589f
- [Kernel] Support W8A8 channel-wise weights and per-token activations in triton... · f41647ee
  Michael Goin authored Apr 11, 2025
```
[Kernel] Support W8A8 channel-wise weights and per-token activations in triton fused_moe_kernel (#16366)
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f41647ee
- [TPU][V1] Make `--disable_chunked_mm_input` mandatory for serving MM models (#16483) · 4d022cbc
  Nicolò Lucchesi authored Apr 11, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  4d022cbc