Commits · 3ec7170ff191a89eb494677a04013a0872fe84fe · OpenDAS / vllm_cscc

22 Jul, 2025 9 commits
- [Bugfix] Decode Tokenized IDs to Strings for `hf_processor` in `llm.chat()`... · 2226d5bd
  Aritra Roy Gosthipaty authored Jul 22, 2025
```
[Bugfix] Decode Tokenized IDs to Strings for `hf_processor` in `llm.chat()` with `model_impl=transformers` (#21353)
Signed-off-by: ariG23498 <aritra.born2fly@gmail.com>
```
  2226d5bd
- [feat] Enable mm caching for transformers backend (#21358) · f38ee34a
  Raushan Turganbay authored Jul 22, 2025
```
Signed-off-by: raushan <raushan@huggingface.co>
```
  f38ee34a
- Adds parallel model weight loading for runai_streamer (#21330) · b194557a
  Benjamin Bartels authored Jul 22, 2025
```
Signed-off-by: bbartels <benjamin@bartels.dev>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  b194557a
- [Perf] Cuda Kernel for Per Token Group Quant (#21083) · 774d0c01
  Wentao Ye authored Jul 22, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  774d0c01
- [feat]: add SM100 support for cutlass FP8 groupGEMM (#20447) · 2c8db17c
  Duncan Moss authored Jul 22, 2025
```
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  2c8db17c
- [perf] Add fused MLA QKV + strided layernorm (#21116) · 4fb56914
  Mickaël Seznec authored Jul 22, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  4fb56914
- Add arcee model (#21296) · 82b8027b
  Raghav Ravishankar authored Jul 22, 2025
```
Signed-off-by: alyosha-swamy <raghav@arcee.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  82b8027b
- Update fp4 quantize API (#21327) · 9e23ad96
  Shu Wang authored Jul 22, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
```
  9e23ad96
- Revert "[Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) (#21334) · e7b20426
  Ming Yang authored Jul 21, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
```
  e7b20426
21 Jul, 2025 3 commits
- [perf] Speed up align sum kernels (#21079) · 0ec82edd
  Himanshu Jaju authored Jul 21, 2025
```
Signed-off-by: Himanshu Jaju <hj@mistral.ai>
```
  0ec82edd
- Add Nvidia ModelOpt config adaptation (#19815) · 6b46c4b6
  Zhiyu authored Jul 21, 2025
```
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
```
  6b46c4b6
- [Model][1/N] Support multiple poolers at model level (#21227) · 042af0c8
  Cyrus Leung authored Jul 21, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  042af0c8
20 Jul, 2025 2 commits

[Model] Support VLMs with transformers backend (#20543) · 9499e26e

Raushan Turganbay authored Jul 20, 2025


Signed-off-by: raushan <raushan@huggingface.co>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

9499e26e

[Model] use AutoWeightsLoader for bart (#18299) · 51ba8395
Calvin Chen authored Jul 20, 2025
```
Signed-off-by: calvin chen <120380290@qq.com>
```
51ba8395

19 Jul, 2025 9 commits
- GLM-4 Update (#20736) · 10eb24cc
  Yuxuan Zhang authored Jul 20, 2025
```
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Lu Fang <fanglu@fb.com>
```
  10eb24cc
- [V0 Deprecation] Deprecate BlockSparse Attention & Phi3-Small (#21217) · 752c6ade
  Woosuk Kwon authored Jul 19, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  752c6ade
- [V1] [Hybrid] Enable piecewise CUDA Graph for mamba layers (#21194) · 881e3cbe
  Thomas Parnell authored Jul 19, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  881e3cbe
- [NVIDIA] Add SM100 Flashinfer MoE blockscale fp8 backend for low latency (#20645) · 6d0734c5
  Kaixi Hou authored Jul 19, 2025
```
Signed-off-by: kaixih <kaixih@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  6d0734c5
- [Model] EXAONE 4.0 model support (#21060) · 3e04107d
  김종곤 authored Jul 19, 2025
```
Signed-off-by: Deepfocused <rlawhdrhs27@gmail.com>
Signed-off-by: woongsik <rlawhdrhs27@gmail.com>
```
  3e04107d
- [Kernel][Performance] Tweak MoE Batched silu_mul_fp8_quant_deep_gemm kernel (#21193) · dcc6cfb9
  Varun Sundar Rabindranath authored Jul 19, 2025
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  dcc6cfb9
- [V0 Deprecation] Remove V0 Spec Decode workers (#21152) · dd572c0a
  Woosuk Kwon authored Jul 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  dd572c0a
- [Quantization] Enable BNB support for more MoE models (#21100) · 466e878f
  Jee Jee Li authored Jul 19, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  466e878f
- Elastic Expert Parallel Initial Support (#20775) · 21793722
  Rui Qiao authored Jul 18, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  21793722
18 Jul, 2025 7 commits
- [Kernel] Apply torch.Tag.needs_fixed_stride_order only for torch==2.6.0 (#19346) · b2eb2b5a
  Richard Zou authored Jul 18, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  b2eb2b5a
- Let GraniteMoeAttention use YaRN (#21174) · ed8cbfed
  Thomas Parnell authored Jul 18, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  ed8cbfed
- [Core] Set pooling params based on task and model (#21128) · 45badd05
  Cyrus Leung authored Jul 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  45badd05
- [Bugfix] Allocate less memory in non-batched CUTLASS MoE (#21121) · 4adc66f6
  ElizaWszola authored Jul 18, 2025
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
```
  4adc66f6
- [Model] Re-add the implicit conversion feature for as_seq_cls_model (#21103) · ca4eb82b
  wang.yuqi authored Jul 18, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
  ca4eb82b
- [Core] FlashInfer CUTLASS fused MoE backend (NVFP4) (#20037) · c7d8724e
  Shu Wang authored Jul 17, 2025
```
Signed-off-by: shuw <shuw@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  c7d8724e
- [Log] Debugging Log with more Information (#20770) · 8a8fc946
  Wentao Ye authored Jul 17, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  8a8fc946
17 Jul, 2025 10 commits
- [V0 deprecation] Remove V0 HPU backend (#21131) · 4de71463
  Woosuk Kwon authored Jul 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4de71463
- [Misc] Qwen MoE model supports LoRA (#20932) · a3a6c695
  Jee Jee Li authored Jul 18, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a3a6c695
- [Model] Update pooling model interface (#21058) · 90bd2ab6
  Cyrus Leung authored Jul 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  90bd2ab6
- [Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) · 9fb2d220
  ElizaWszola authored Jul 17, 2025
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
```
  9fb2d220
- [Docs] Improve docstring formatting for `FusedMoEParallelConfig.make` (#21117) · fe8a2c54
  Harry Mellor authored Jul 17, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  fe8a2c54
- [VLM] Add Nemotron-Nano-VL-8B-V1 support (#20349) · 4ef00b5c
  kYLe authored Jul 17, 2025
```
Signed-off-by: Kyle Huang <kylhuang@nvidia.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  4ef00b5c
- [Model] Add ToolParser and MoE Config for Hunyuan A13B (#20820) · 5a7fb3ab
  Asher authored Jul 17, 2025
```
Signed-off-by: Asher Zhang <asherszhang@tencent.com>
```
  5a7fb3ab
- [Kernel] DeepGemm MoE : Integrate triton permute / unpermute kernels (#20903) · 11dfdf21
  Varun Sundar Rabindranath authored Jul 17, 2025
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  11dfdf21
- [Misc] Fix PhiMoE expert mapping (#21085) · c5b8b595
  Jee Jee Li authored Jul 17, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c5b8b595
- [Bugfix] Fix Machete zero point issue for GPTQ models on SM90 (#21066) · 28a6d542
  Michael Goin authored Jul 16, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  28a6d542