Commits · 6a971ed692974b3d6309d556b15c8cc726b091f9 · OpenDAS / vllm_cscc

"examples/deployments/GKE/vllm/disagg.yaml" did not exist on "a01b79929b1e8ea625743ac9f8205604f2bfa59f"

19 Jul, 2025 16 commits
- [CI/CD][bugfix]fix: error argument to loads has incompatible type (#21223) · da6579bf
  Sungjae Lee authored Jul 19, 2025
```
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Signed-off-by: Sungjae Lee <sung-jae.lee@navercorp.com>
```
  da6579bf
- [bugfix] Fix auto thread-binding when world_size > 1 in CPU backend and refactor code (#21032) · e3a0e43d
  Li, Jiang authored Jul 19, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  e3a0e43d
- [Bugfix][Frontend] Fix openai CLI arg `middleware` (#21220) · b3d82108
  22quinn authored Jul 19, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  b3d82108
- [NVIDIA] Add SM100 Flashinfer MoE blockscale fp8 backend for low latency (#20645) · 6d0734c5
  Kaixi Hou authored Jul 19, 2025
```
Signed-off-by: kaixih <kaixih@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  6d0734c5
- [BugFix] Fix potential cuda-graph IMA (#21196) · 59f93530
  Lucas Wilkinson authored Jul 19, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  59f93530
- [Bugfix] Fix ndarray video color from VideoAsset (#21064) · 18e519ec
  Isotr0py authored Jul 19, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  18e519ec
- [V0 deprecation] Remove long context LoRA (#21169) · 1eaff278
  Jee Jee Li authored Jul 19, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1eaff278
- [Model] EXAONE 4.0 model support (#21060) · 3e04107d
  김종곤 authored Jul 19, 2025
```
Signed-off-by: Deepfocused <rlawhdrhs27@gmail.com>
Signed-off-by: woongsik <rlawhdrhs27@gmail.com>
```
  3e04107d
- [Bug] DeepGemm: Fix TypeError: per_block_cast_to_fp8() missing 1 required... · 37bd8d6e
  Wentao Ye authored Jul 19, 2025
```
[Bug] DeepGemm: Fix TypeError: per_block_cast_to_fp8() missing 1 required positional argument: 'use_ue8m0' for SM100 (#21187)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  37bd8d6e
- [BugFix][CPU] Fix `TorchSDPABackendImpl` doesn't have `use_irope` (#21200) · 468e2400
  Lucas Wilkinson authored Jul 19, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  468e2400
- [Kernel][Performance] Tweak MoE Batched silu_mul_fp8_quant_deep_gemm kernel (#21193) · dcc6cfb9
  Varun Sundar Rabindranath authored Jul 19, 2025
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  dcc6cfb9
- [V0 Deprecation] Remove V0 Spec Decode workers (#21152) · dd572c0a
  Woosuk Kwon authored Jul 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  dd572c0a
- [Bugfix][Model] Fix LoRA for Mistral-Small-3.1-24B-Instruct-2503 (#21183) · 9ffe905a
  Varun Sundar Rabindranath authored Jul 19, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  9ffe905a
- [Core] Support Local Chunked Attention for Hybrid KV Cache (#19351) · 9a9fda14
  Lucia Fang authored Jul 19, 2025
```
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Lu Fang <fanglu@meta.com>
```
  9a9fda14
- [Quantization] Enable BNB support for more MoE models (#21100) · 466e878f
  Jee Jee Li authored Jul 19, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  466e878f
- Elastic Expert Parallel Initial Support (#20775) · 21793722
  Rui Qiao authored Jul 18, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  21793722
18 Jul, 2025 15 commits
- [Bugfix] Voxtral on Blackwell GPUs (RTX 50 series) (#21077) · 5782581a
  hax0r31337 authored Jul 19, 2025
```
Signed-off-by: hax0r31337 <liulihaocaiqwq@gmail.com>
```
  5782581a
- [Core] Avoid KVCacheBlock.__eq__ invocations in FreeKVCacheBlockQueue (#21005) · 0f199f19
  JialinOuyang-Meta authored Jul 18, 2025
```
Signed-off-by: Jialin Ouyang <jialino@meta.com>
```
  0f199f19
- [Kernel] Apply torch.Tag.needs_fixed_stride_order only for torch==2.6.0 (#19346) · b2eb2b5a
  Richard Zou authored Jul 18, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  b2eb2b5a
- Let GraniteMoeAttention use YaRN (#21174) · ed8cbfed
  Thomas Parnell authored Jul 18, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  ed8cbfed
- [Core] Set pooling params based on task and model (#21128) · 45badd05
  Cyrus Leung authored Jul 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  45badd05
- [Bugfix] Allocate less memory in non-batched CUTLASS MoE (#21121) · 4adc66f6
  ElizaWszola authored Jul 18, 2025
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
```
  4adc66f6
- [Bugfix] The special_tokens in tokenizer should also be controlled by... · 5895afd7
  wang.yuqi authored Jul 18, 2025
```
[Bugfix] The special_tokens in tokenizer should also be controlled by do_lower_case in encoder_config. (#20750)
Signed-off-by: wang.yuqi <noooop@126.com>
```
  5895afd7
- [Model] Re-add the implicit conversion feature for as_seq_cls_model (#21103) · ca4eb82b
  wang.yuqi authored Jul 18, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
  ca4eb82b
- [Misc] Make MM embedding merge interface explicit in model runner (#21147) · ba2dfbb0
  Roger Wang authored Jul 18, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.me>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  ba2dfbb0
- [benchmark] Sending request strictly follows the random intervals (#21108) · 1bf65138
  Jialin Ouyang authored Jul 17, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  1bf65138
- [Misc] Do not print async output warning for v1 (#21151) · 54cf1cae
  Woosuk Kwon authored Jul 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  54cf1cae
- [Core] FlashInfer CUTLASS fused MoE backend (NVFP4) (#20037) · c7d8724e
  Shu Wang authored Jul 17, 2025
```
Signed-off-by: shuw <shuw@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  c7d8724e
- [Attention] Make local attention backend agnostic (#21093) · 89cab4d0
  Lucas Wilkinson authored Jul 18, 2025
  
  89cab4d0
- [Bugfix] Fix the tensor non-contiguous issue for Flashinfer TRT-LLM backend... · 8dfb45ca
  elvischenv authored Jul 18, 2025
```
[Bugfix] Fix the tensor non-contiguous issue for Flashinfer TRT-LLM backend attention kernel (#21133)
```
  8dfb45ca
- [Log] Debugging Log with more Information (#20770) · 8a8fc946
  Wentao Ye authored Jul 17, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  8a8fc946
17 Jul, 2025 9 commits
- [V0 deprecation] Remove V0 HPU backend (#21131) · 4de71463
  Woosuk Kwon authored Jul 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4de71463
- On environments where numa cannot be detected we get 0 (#21115) · ac9fb732
  Eric Curtin authored Jul 17, 2025
```
Signed-off-by: Eric Curtin <ecurtin@redhat.com>
```
  ac9fb732
- [Misc] Qwen MoE model supports LoRA (#20932) · a3a6c695
  Jee Jee Li authored Jul 18, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a3a6c695
- [Model] Update pooling model interface (#21058) · 90bd2ab6
  Cyrus Leung authored Jul 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  90bd2ab6
- [Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) · 9fb2d220
  ElizaWszola authored Jul 17, 2025
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
```
  9fb2d220
- [Misc] Avoid unnecessary import (#21106) · 89e3c4e9
  wangxiyuan authored Jul 17, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  89e3c4e9
- [Docs] Improve docstring formatting for `FusedMoEParallelConfig.make` (#21117) · fe8a2c54
  Harry Mellor authored Jul 17, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  fe8a2c54
- [VLM] Add Nemotron-Nano-VL-8B-V1 support (#20349) · 4ef00b5c
  kYLe authored Jul 17, 2025
```
Signed-off-by: Kyle Huang <kylhuang@nvidia.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  4ef00b5c
- [Model] Add ToolParser and MoE Config for Hunyuan A13B (#20820) · 5a7fb3ab
  Asher authored Jul 17, 2025
```
Signed-off-by: Asher Zhang <asherszhang@tencent.com>
```
  5a7fb3ab