Commits · 9883a1885970f53f88fd962c85dd406ade0966cd · OpenDAS / vllm_cscc

"docs/serving/offline_inference.md" did not exist on "3eb08ed9b1114fc16eaa2386b2e7ad14eae87073"

14 Apr, 2025 10 commits
- Fix triton install condition on CPU (#16600) · 9883a188
  Harry Mellor authored Apr 14, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  9883a188
- [TPU][V1] Fix exponential padding when `max-num-batched-tokens` is not a power of 2 (#16596) · b3f2fddd
  Nicolò Lucchesi authored Apr 14, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  b3f2fddd
- [Bugfix] Multi-modal caches not acting like LRU caches (#16593) · aa29841e
  Cyrus Leung authored Apr 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  aa29841e
- [fix]: Dockerfile.ppc64le fixes for opencv-python and hf-xet (#16048) · 6bf27aff
  Md. Shafi Hussain authored Apr 14, 2025
```
Signed-off-by: Md. Shafi Hussain <Md.Shafi.Hussain@ibm.com>
```
  6bf27aff
- [Misc] Update usage with mooncake lib for kv transfer (#16523) · 1dd23386
  shangmingc authored Apr 14, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  1dd23386
- [Misc] refactor examples (#16563) · 7cbfc109
  Reid authored Apr 14, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  7cbfc109
- [Misc] remove warning if triton>=3.2.0 (#16553) · ce4ddd2d
  DefTruth authored Apr 14, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  ce4ddd2d
- Improve configs - `SchedulerConfig` (#16533) · e51929eb
  Harry Mellor authored Apr 14, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e51929eb
- [Core][V0] Enable regex support with xgrammar (#13228) · dc1b4a6f
  Russell Bryant authored Apr 13, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  dc1b4a6f
- [Benchmark][Bugfix] Fix SonnetDataset default values in benchmark_throughput.py (#16556) · 63d2705e
  Jennifer Zhao authored Apr 13, 2025
  
  63d2705e
13 Apr, 2025 2 commits
- Enable PTPC FP8 for CompressedTensorsW8A8Fp8MoEMethod (triton fused_moe) (#16537) · d085a440
  Michael Goin authored Apr 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  d085a440
- [V1][Spec Decode] KV cache slots for eagle heads (#16370) · f49e5aff
  Lily Liu authored Apr 12, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  f49e5aff
12 Apr, 2025 16 commits
- [Bugfix] Validate logit biases to prevent out of vocab ids crashing engine (#16529) · 6c11ecf8
  Ryan McConville authored Apr 12, 2025
```
Signed-off-by: Ryan McConville <ryan@ryanmcconville.com>
```
  6c11ecf8
- [Perf] Optimize Preparing Inputs for GPU Model Runner (#16484) · 93e5f3c5
  SnowCharm authored Apr 12, 2025
```
Signed-off-by: snowcharm <snowcharmqq@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  93e5f3c5
- Fix syntaxWarning: invalid escape sequence '\s' (#16532) · 70363bcc
  Jie Fu (傅杰) authored Apr 12, 2025
```
Signed-off-by: Jie Fu <jiefu@tencent.com>
```
  70363bcc
- [Misc] Delete redundant code (#16530) · 3cdc5766
  Jee Jee Li authored Apr 12, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  3cdc5766
- [MISC] Make GroupCoordinator compatible with out-of-tree devices (#16464) · 68bb122e
  Huazhong Ji authored Apr 12, 2025
```
Signed-off-by: hzji210@gmail.com <hzji210@gmail.com>
```
  68bb122e
- [V1] Enable multi-input by default (#15799) · d9fc8cd9
  Cyrus Leung authored Apr 12, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d9fc8cd9
- [Misc] Openai transcription client example use same Whisper model (#16487) · f069f3ea
  Nicolò Lucchesi authored Apr 12, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  f069f3ea
- [Misc] Update chat utils tests (#16520) · c5bc0e7f
  Cyrus Leung authored Apr 12, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c5bc0e7f
- fix: spelling (#16466) · 4a3a5187
  Tianer Zhou authored Apr 12, 2025
```
Signed-off-by: Tianer Zhou <ezhoureal@gmail.com>
```
  4a3a5187
- [Frontend] support matryoshka representation / support embedding API dimensions (#16331) · fbf722c6
  wang.yuqi authored Apr 12, 2025
  
  fbf722c6
- [Feature][V1] Add xgrammar to support minLength, maxLength with test (#16516) · e92d7085
  leon-seidel authored Apr 12, 2025
```
Signed-off-by: Leon Seidel <leon.seidel@fau.de>
```
  e92d7085
- Optimized topk for topk=1 (Llama-4) (#16512) · bd6028d6
  Michael Goin authored Apr 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  bd6028d6
- [Doc] Update Llama4 Model Names in Supported Models (#16509) · 802329de
  Ye (Charlotte) Qi authored Apr 11, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  802329de
- [BugFix] Handle non-contiguous tensors properly when serializing (#16492) · 41cc883c
  Nick Hill authored Apr 11, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  41cc883c
- [CI][Bugfix] Add mistral_tool_use to Ci (#16517) · 57504a4b
  Michael Goin authored Apr 11, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  57504a4b
- [Doc] Fix link to vLLM blog (#16519) · ed4792c9
  Yuan Tang authored Apr 11, 2025
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  ed4792c9
11 Apr, 2025 12 commits
- Bugfix for PixtralHF models without spatial_merge_size (#16513) · 87b836ba
  Michael Goin authored Apr 11, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  87b836ba
- [Bugfix] clean up duplicated code (#16485) · 56c76c2e
  rongfu.leng authored Apr 12, 2025
```
Signed-off-by: Gogs <gogs@fake.local>
Co-authored-by: Gogs <gogs@fake.local>
```
  56c76c2e
- Update openai_compatible_server.md (#16507) · c09632a6
  Christian Sears authored Apr 11, 2025
```
Signed-off-by: Christian Sears <csears@redhat.com>
```
  c09632a6
- [Kernel] Add tuned FusedMoE kernel config for Llama4 Scout, TP=8 on H100 (#16488) · a3bf8d4a
  Yong Hoon Shin authored Apr 11, 2025
  
  a3bf8d4a
- [Frontend] Added chat templates for LLaMa4 pythonic tool calling (#16463) · 16eda8c4
  Ye (Charlotte) Qi authored Apr 11, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Co-authored-by: Kai Wu <kaiwu@meta.com>
```
  16eda8c4
- Improve configs - `LoadConfig` (#16422) · cd77382a
  Harry Mellor authored Apr 11, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  cd77382a
- [Bugfix] handle alignment of encoder_seq_lens in mllama.py (#14784) · 71b9cde0
  Travis Johnson authored Apr 11, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  71b9cde0
- [Doc] Document InternVL3 support (#16495) · 5285589f
  Isotr0py authored Apr 12, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  5285589f
- [Kernel] Support W8A8 channel-wise weights and per-token activations in triton... · f41647ee
  Michael Goin authored Apr 11, 2025
```
[Kernel] Support W8A8 channel-wise weights and per-token activations in triton fused_moe_kernel (#16366)
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f41647ee
- [TPU][V1] Make `--disable_chunked_mm_input` mandatory for serving MM models (#16483) · 4d022cbc
  Nicolò Lucchesi authored Apr 11, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  4d022cbc
- Fix erroneous "model doesn't support compile" warning (#16486) · 70de35a8
  Richard Zou authored Apr 11, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  70de35a8
- [Hardware][Intel-Gaudi] Multi-step scheduling implementation for HPU (#12779) · 34b2cf3b
  Tomasz Zielinski authored Apr 11, 2025
```
Signed-off-by: Tomasz Zielinski <tomasz.zielinski@intel.com>
```
  34b2cf3b