Commits · a41351f363f3e7a212582e51b2c1c35c18aaa9df · OpenDAS / vllm_cscc · GitLab

25 Apr, 2025 13 commits
- [Quantization][FP8] Add support for FP8 models with input_scale for output... · a41351f3
  rasmith authored Apr 25, 2025
```
[Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization (#15734)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
```
  a41351f3
- [Bugfix] remove fallback in guided_json (int range, patterns) (#16725) · 6aae216b
  Sangyeon Cho authored Apr 25, 2025
```
Signed-off-by: csy1204 <josang1204@gmail.com>
Co-authored-by: 조상연[플레이스 AI] <sang-yeon.cho@navercorp.com>
```
  6aae216b
- [Perf]Optimize rotary_emb implementation to use Triton operator for improved... · b22980a1
  yexin(叶鑫) authored Apr 25, 2025
```
[Perf]Optimize rotary_emb implementation to use Triton operator for improved inference performance (#16457)
Signed-off-by: cynthieye <yexin93@qq.com>
Co-authored-by: MagnetoWang <magnetowang@outlook.com>
```
  b22980a1
- [Misc] Benchmark Serving Script Support Appending Results (#17028) · 881f7358
  Lucas Wilkinson authored Apr 25, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  881f7358
- [Bugfix][Misc] Use TritonPlaceholderModule to defensively import triton (#15099) · 2f540455
  Mengqing Cao authored Apr 25, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  2f540455
- [Misc] Clean up redundant code in uniproc_executor.py (#16762) · 5aa6efb9
  Lifu Huang authored Apr 24, 2025
```
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
```
  5aa6efb9
- Move missed `SchedulerConfig` args into scheduler config group in `EngineArgs` (#17131) · 6ca02344
  Harry Mellor authored Apr 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6ca02344
- [Docs] Fix True->true in supported_models.md (#17141) · 64981899
  Michael Goin authored Apr 24, 2025
  
  64981899
- [Doc] V1 : Update LoRA status (#17133) · 7a0a9da7
  Varun Sundar Rabindranath authored Apr 24, 2025
```
Signed-off-by: varun sundar rabindranath <vsundarr@redhat.com>
Co-authored-by: varun sundar rabindranath <vsundarr@redhat.com>
```
  7a0a9da7
- fix float16 support for kimi-vl (#17156) · 69bff9bc
  Zaida Zhou authored Apr 25, 2025
```
Co-authored-by: zhouzaida <zhouzaida@msh.team>
```
  69bff9bc
- [Attention] FA3 decode perf improvement - single mma warp group support for head dim 128 (#16864) · 41ca7eb4
  Lucas Wilkinson authored Apr 24, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  41ca7eb4
- [FEAT] [ROCm]: AITER Fused MOE V1 Support (#16752) · eef36472
  vllmellm authored Apr 25, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  eef36472
- Use custom address for listening socket (#15988) · 0d6e187e
  jglaser authored Apr 24, 2025
```
Signed-off-by: Jens Glaser <glaserj@ornl.gov>
```
  0d6e187e
24 Apr, 2025 27 commits
- Better error message for missing mistral params.json (#17132) · 9420a1fc
  Michael Goin authored Apr 24, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  9420a1fc
- [Misc] Add example to run DeepSeek with Ray Serve LLM (#17134) · 583e9009
  Rui Qiao authored Apr 24, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  583e9009
- Add chat template for Llama 4 models (#16428) · 05e1fbfc
  Maximilien de Bayser authored Apr 24, 2025
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  05e1fbfc
- Add collective_rpc to llm engine (#16999) · fe921763
  Yinghai Lu authored Apr 24, 2025
```
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
```
  fe921763
- [Docs] Generate correct github links for decorated functions (#17125) · 6d0df0eb
  Russell Bryant authored Apr 24, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  6d0df0eb
- Improve configs - `LoRAConfig` + `PromptAdapterConfig` (#16980) · 0fa939e2
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0fa939e2
- Add `:markdownhelp:` to `EngineArgs` docs so markdown docstrings render properly (#17124) · 0422ce10
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0422ce10
- Molmo Requirements (#17026) · 47bdee40
  Eyshika Agarwal authored Apr 24, 2025
```
Signed-off-by: Eyshika Agarwal <eyshikaengineer@gmail.com>
Signed-off-by: eyshika <eyshikaengineer@gmail.com>
```
  47bdee40
- existing torch installation pip command fix for docs (#17059) · 49f18943
  Atilla authored Apr 24, 2025
  
  49f18943
- Updating builkite job for IBM Power (#17111) · 5adf6f6b
  Aaruni Aggarwal authored Apr 24, 2025
```
Signed-off-by: Aaruni Aggarwal <aaruniagg@gmail.com>
```
  5adf6f6b
- [CI] Add automation for the `tool-calling` github label (#17118) · 4115f199
  Russell Bryant authored Apr 24, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  4115f199
- [V1][Spec Decoding] Add num_drafts and num_accepted_tokens_per_position metrics (#16665) · 340d7b1b
  Mark McLoughlin authored Apr 24, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  340d7b1b
- [Misc] refactor example series - structured outputs (#17040) · 1bcbcbf5
  Reid authored Apr 24, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  1bcbcbf5
- Add missing rocm_skinny_gemms kernel test to CI (#17060) · 82e43b2d
  Michael Goin authored Apr 24, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  82e43b2d
- [Frontend] Using matryoshka_dimensions control the allowed output dimensions. (#16970) · 67309a1c
  wang.yuqi authored Apr 24, 2025
  
  67309a1c
- [V1][Structured Output] Clear xgrammar compiler object when engine core shut... · b724afe3
  Shanshan Shen authored Apr 24, 2025
```
[V1][Structured Output] Clear xgrammar compiler object when engine core shut down to avoid nanobind leaked warning (#16954)
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  b724afe3
- Improve static type checking in `LoRAModelRunnerMixin` (#17104) · 21f4f1c9
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  21f4f1c9
- [Misc] Remove OLMo2 config copy (#17066) · b0c1f620
  Isotr0py authored Apr 24, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b0c1f620
- [V1][PP] Optimization: continue scheduling prefill chunks (#17080) · c0dfd975
  Rui Qiao authored Apr 24, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  c0dfd975
- Fix OOT registration test (#17099) · a9138e85
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a9138e85
- Simplify `TokenizerGroup` (#16790) · 0a05ed57
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0a05ed57
- Disable enforce_eager for V1 TPU sampler and structured output tests (#17016) · 14288d13
  Michael Goin authored Apr 24, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  14288d13
- [Chore] Remove Sampler from Model Code (#17084) · b411418f
  Woosuk Kwon authored Apr 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b411418f
- Add docs for runai_streamer_sharded (#17093) · 2bc0f72a
  omer-dayan authored Apr 24, 2025
```
Signed-off-by: Omer Dayan (SW-GPU) <omer@run.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  2bc0f72a
- [doc] update to hyperlink (#17096) · 9c1244de
  Reid authored Apr 24, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  9c1244de
- [V1] Update structured output (#16812) · db2f8d91
  Reid authored Apr 24, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  db2f8d91
- [Bugfix][Core] add seq_id_to_seq_group clearing to avoid memory leak when s… (#16472) · 6167c0e5
  张宇 authored Apr 24, 2025
```
Signed-off-by: 开哲 <kaizhe.zy@alibaba-inc.com>
Co-authored-by: 开哲 <kaizhe.zy@alibaba-inc.com>
```
  6167c0e5