Commits · 45badd05d04254eb75c48cee7b1d454a80de2165 · OpenDAS / vllm_cscc

18 Jul, 2025 16 commits
- [Core] Set pooling params based on task and model (#21128) · 45badd05
  Cyrus Leung authored Jul 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  45badd05
- [Bugfix] Allocate less memory in non-batched CUTLASS MoE (#21121) · 4adc66f6
  ElizaWszola authored Jul 18, 2025
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
```
  4adc66f6
- [Doc] Fix typo in model name (#21178) · 55ad6487
  Cyrus Leung authored Jul 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  55ad6487
- [Bugfix] The special_tokens in tokenizer should also be controlled by... · 5895afd7
  wang.yuqi authored Jul 18, 2025
```
[Bugfix] The special_tokens in tokenizer should also be controlled by do_lower_case in encoder_config. (#20750)
Signed-off-by: wang.yuqi <noooop@126.com>
```
  5895afd7
- [Model] Re-add the implicit conversion feature for as_seq_cls_model (#21103) · ca4eb82b
  wang.yuqi authored Jul 18, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
  ca4eb82b
- [Misc] Make MM embedding merge interface explicit in model runner (#21147) · ba2dfbb0
  Roger Wang authored Jul 18, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.me>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  ba2dfbb0
- [benchmark] Sending request strictly follows the random intervals (#21108) · 1bf65138
  Jialin Ouyang authored Jul 17, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  1bf65138
- [Misc] Do not print async output warning for v1 (#21151) · 54cf1cae
  Woosuk Kwon authored Jul 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  54cf1cae
- [Perf] Add swap_ab to SM90 FP8 non-block CUTLASS moe grouped gemm (#20911) · 5780121c
  shixianc authored Jul 17, 2025
```
Signed-off-by: Shixian Cui <shixian@amazon.com>
Co-authored-by: Shixian Cui <shixian@amazon.com>
```
  5780121c
- [Core] FlashInfer CUTLASS fused MoE backend (NVFP4) (#20037) · c7d8724e
  Shu Wang authored Jul 17, 2025
```
Signed-off-by: shuw <shuw@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  c7d8724e
- [Doc] Add inplace weights loading example (#19640) · b38baabc
  22quinn authored Jul 17, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  b38baabc
- [Attention] Make local attention backend agnostic (#21093) · 89cab4d0
  Lucas Wilkinson authored Jul 18, 2025
  
  89cab4d0
- [Docs] Update supported models documentation with missing models (#20844) · b9a21e91
  Lucia Fang authored Jul 18, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  b9a21e91
- [Docs] Add minimal demo of Ray Data API usage (#21080) · c4e3b125
  Ricardo Decal authored Jul 17, 2025
```
Signed-off-by: Ricardo Decal <rdecal@anyscale.com>
```
  c4e3b125
- [Bugfix] Fix the tensor non-contiguous issue for Flashinfer TRT-LLM backend... · 8dfb45ca
  elvischenv authored Jul 18, 2025
```
[Bugfix] Fix the tensor non-contiguous issue for Flashinfer TRT-LLM backend attention kernel (#21133)
```
  8dfb45ca
- [Log] Debugging Log with more Information (#20770) · 8a8fc946
  Wentao Ye authored Jul 17, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  8a8fc946
17 Jul, 2025 22 commits
- [V0 deprecation] Remove V0 HPU backend (#21131) · 4de71463
  Woosuk Kwon authored Jul 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4de71463
- On environments where numa cannot be detected we get 0 (#21115) · ac9fb732
  Eric Curtin authored Jul 17, 2025
```
Signed-off-by: Eric Curtin <ecurtin@redhat.com>
```
  ac9fb732
- [Misc] Qwen MoE model supports LoRA (#20932) · a3a6c695
  Jee Jee Li authored Jul 18, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a3a6c695
- [Model] Update pooling model interface (#21058) · 90bd2ab6
  Cyrus Leung authored Jul 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  90bd2ab6
- [Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) · 9fb2d220
  ElizaWszola authored Jul 17, 2025
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
```
  9fb2d220
- [Docs] Move code block out of admonition now that it's short (#21118) · 2d6a3820
  Harry Mellor authored Jul 17, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  2d6a3820
- [Misc] Avoid unnecessary import (#21106) · 89e3c4e9
  wangxiyuan authored Jul 17, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  89e3c4e9
- [Docs] Improve docstring formatting for `FusedMoEParallelConfig.make` (#21117) · fe8a2c54
  Harry Mellor authored Jul 17, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  fe8a2c54
- [VLM] Add Nemotron-Nano-VL-8B-V1 support (#20349) · 4ef00b5c
  kYLe authored Jul 17, 2025
```
Signed-off-by: Kyle Huang <kylhuang@nvidia.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  4ef00b5c
- [Model] Add ToolParser and MoE Config for Hunyuan A13B (#20820) · 5a7fb3ab
  Asher authored Jul 17, 2025
```
Signed-off-by: Asher Zhang <asherszhang@tencent.com>
```
  5a7fb3ab
- [Kernel] DeepGemm MoE : Integrate triton permute / unpermute kernels (#20903) · 11dfdf21
  Varun Sundar Rabindranath authored Jul 17, 2025
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  11dfdf21
- [Bugfix]: Fix final_res_batch list index out of range error (#21055) · fdc5b43d
  Chauncey authored Jul 17, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  fdc5b43d
- [Misc] Fix PhiMoE expert mapping (#21085) · c5b8b595
  Jee Jee Li authored Jul 17, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c5b8b595
- [V1] [KVConnector] Fix MultiprocExecutor worker output aggregation (#21048) · 4fcef49e
  David Ben-David authored Jul 17, 2025
```
Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>
```
  4fcef49e
- [V1][P/D]Enhance Performance and code readability for P2pNcclConnector (#20906) · 8a4e5c5f
  Zhonghua Deng authored Jul 17, 2025
```
Signed-off-by: Abatom <abzhonghua@gmail.com>
```
  8a4e5c5f
- [Attention] Refactor attention metadata builder interface (#20466) · 76b49444
  Lucas Wilkinson authored Jul 17, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  76b49444
- [Bugfix] Fix Machete zero point issue for GPTQ models on SM90 (#21066) · 28a6d542
  Michael Goin authored Jul 16, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  28a6d542
- [TPU] Start using python 3.12 (#21000) · 58760e12
  XiongfeiWei authored Jul 16, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  58760e12
- [Docker] Allow FlashInfer to be built in the ARM CUDA Dockerfile (#21013) · a50d9182
  Michael Goin authored Jul 16, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  a50d9182
- [Bugfix] weight loading use correct tp_group with patch_tensor_parallel_group (#21024) · c9ba8104
  Kevin_Xiong authored Jul 17, 2025
```
Signed-off-by: KevinXiong-C <kevin_xiong1997@outlook.com>
```
  c9ba8104
- Update PyTorch to `torch==2.7.1` for CUDA (#21011) · 4e7dfbe7
  Michael Goin authored Jul 16, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  4e7dfbe7
- Remove torch_xla.tpu.version() from pallas.py. (#21065) · 72ad2735
  QiliangCui authored Jul 16, 2025
```
Signed-off-by: Qiliang Cui <derrhein@gmail.com>
```
  72ad2735
16 Jul, 2025 2 commits

Support FP8 Quantization and Inference Run on Intel Gaudi (HPU) using INC... · 01513a33

Nir David authored Jul 16, 2025


Support FP8 Quantization and Inference Run on Intel Gaudi (HPU) using INC (Intel Neural Compressor) (#12010)
Signed-off-by: Nir David <ndavid@habana.ai>
Signed-off-by: Uri Livne <ulivne@habana.ai>
Co-authored-by: Uri Livne <ulivne@habana.ai>

01513a33

[Model] Remove model sampler (#21059) · ac2bf41e
Cyrus Leung authored Jul 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
ac2bf41e