Commits · 4fb56914c5f27ef062e10d44a0f79c6ceab382f9 · OpenDAS / vllm_cscc

"tests/vscode:/vscode.git/clone" did not exist on "8c38ee7007c50ac5aef9ed43ae91c6f031799c40"

22 Jul, 2025 16 commits
- [perf] Add fused MLA QKV + strided layernorm (#21116) · 4fb56914
  Mickaël Seznec authored Jul 22, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  4fb56914
- [Core] Introduce popleft_n and append_n in FreeKVCacheBlockQueue to further... · ed250545
  Jialin Ouyang authored Jul 22, 2025
```
[Core] Introduce popleft_n and append_n in FreeKVCacheBlockQueue to further optimize block_pool (#21222)
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  ed250545
- [benchmark] Port benchmark request sent optimization to benchmark_serving (#21209) · 10904e6d
  Jialin Ouyang authored Jul 22, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  10904e6d
- [Core] Optimize update checks in LogitsProcessor (#21245) · a3223766
  Jialin Ouyang authored Jul 22, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  a3223766
- [Misc] Remove deprecated args in v0.10 (#21349) · bc8a8ce5
  Kebe authored Jul 22, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
  bc8a8ce5
- [Bugfix] Fix eviction cached blocked logic (#21357) · 32142b3c
  Simon Mo authored Jul 22, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  32142b3c
- Add arcee model (#21296) · 82b8027b
  Raghav Ravishankar authored Jul 22, 2025
```
Signed-off-by: alyosha-swamy <raghav@arcee.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  82b8027b
- [Feature][eplb] add verify ep or tp or dp (#21102) · 3779eb8c
  rongfu.leng authored Jul 22, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  3779eb8c
- Update fp4 quantize API (#21327) · 9e23ad96
  Shu Wang authored Jul 22, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
```
  9e23ad96
- [Bug] DeepGemm: Fix Cuda Init Error (#21312) · e69a92a1
  Wentao Ye authored Jul 22, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  e69a92a1
- [Misc] DeepEPHighThroughtput - Enable Inductor pass (#21311) · 8425f785
  Varun Sundar Rabindranath authored Jul 22, 2025
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  8425f785
- Fix kv_cache_dtype handling for out-of-tree HPU plugin (#21302) · c17231e8
  Konrad Zawora authored Jul 22, 2025
```
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
```
  c17231e8
- [Core] Minimize number of dict lookup in _maybe_evict_cached_block (#21281) · af376ca1
  Jialin Ouyang authored Jul 21, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  af376ca1
- Revert "[Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) (#21334) · e7b20426
  Ming Yang authored Jul 21, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
```
  e7b20426
- [Intel GPU] Ray Compiled Graph avoid NCCL for Intel GPU (#21338) · 90f1e554
  Ratnam Parikh authored Jul 21, 2025
```
Signed-off-by: ratnampa <ratnam.parikh@intel.com>
```
  90f1e554
- [XPU] Enable external_launcher to serve as an executor via torchrun (#21021) · 25d585ab
  Chaojun Zhang authored Jul 22, 2025
```
Signed-off-by: chzhang <chaojun.zhang@intel.com>
```
  25d585ab
21 Jul, 2025 9 commits
- [v1][sampler] Inplace logprobs comparison to get the token rank (#21283) · 8d0a01a5
  Lu Fang authored Jul 21, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  8d0a01a5
- [perf] Speed up align sum kernels (#21079) · 0ec82edd
  Himanshu Jaju authored Jul 21, 2025
```
Signed-off-by: Himanshu Jaju <hj@mistral.ai>
```
  0ec82edd
- [DP] Fix Prometheus Logging (#21257) · 29d1ffc5
  Robert Shaw authored Jul 21, 2025
```
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
```
  29d1ffc5
- [Attention] Clean up iRoPE in V1 (#21188) · 304dce7e
  Lucas Wilkinson authored Jul 21, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  304dce7e
- [BugFix] make utils.current_stream thread-safety (#21252) (#21253) · a0e827e0
  simpx authored Jul 22, 2025
```
Signed-off-by: simpx <simpxx@gmail.com>
```
  a0e827e0
- [CPU] Enable shared-memory based pipeline parallel for CPU backend (#21289) · a15a50fc
  Li, Jiang authored Jul 22, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a15a50fc
- Add Nvidia ModelOpt config adaptation (#19815) · 6b46c4b6
  Zhiyu authored Jul 21, 2025
```
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
```
  6b46c4b6
- [Model][1/N] Support multiple poolers at model level (#21227) · 042af0c8
  Cyrus Leung authored Jul 21, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  042af0c8
- [Bugfix] Fix missing placeholder in logger debug (#21280) · 378d33c3
  Cyrus Leung authored Jul 21, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  378d33c3
20 Jul, 2025 4 commits

[Model] Support VLMs with transformers backend (#20543) · 9499e26e

Raushan Turganbay authored Jul 20, 2025


Signed-off-by: raushan <raushan@huggingface.co>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

9499e26e

[Model] use AutoWeightsLoader for bart (#18299) · 51ba8395
Calvin Chen authored Jul 20, 2025
```
Signed-off-by: calvin chen <120380290@qq.com>
```
51ba8395
Enable v1 metrics tests (#20953) · d1fb65bd
Seiji Eicher authored Jul 19, 2025
```
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
```
d1fb65bd
[TPU] support fp8 kv cache quantization (#19292) · 3a1d8940
Chengji Yao authored Jul 19, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
3a1d8940

19 Jul, 2025 11 commits
- GLM-4 Update (#20736) · 10eb24cc
  Yuxuan Zhang authored Jul 20, 2025
```
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Lu Fang <fanglu@fb.com>
```
  10eb24cc
- [BugFix] Fix full cuda graph slot_mapping (#21228) · 2e8cbb58
  fhl2000 authored Jul 20, 2025
```
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
```
  2e8cbb58
- [V0 Deprecation] Deprecate BlockSparse Attention & Phi3-Small (#21217) · 752c6ade
  Woosuk Kwon authored Jul 19, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  752c6ade
- [V1] [Hybrid] Enable piecewise CUDA Graph for mamba layers (#21194) · 881e3cbe
  Thomas Parnell authored Jul 19, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  881e3cbe
- [BugFix] Make PD work with Ray (#21072) · 9f414a12
  kourosh hakhamaneshi authored Jul 19, 2025
```
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
```
  9f414a12
- [CI/CD][bugfix]fix: error argument to loads has incompatible type (#21223) · da6579bf
  Sungjae Lee authored Jul 19, 2025
```
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Signed-off-by: Sungjae Lee <sung-jae.lee@navercorp.com>
```
  da6579bf
- [bugfix] Fix auto thread-binding when world_size > 1 in CPU backend and refactor code (#21032) · e3a0e43d
  Li, Jiang authored Jul 19, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  e3a0e43d
- [Bugfix][Frontend] Fix openai CLI arg `middleware` (#21220) · b3d82108
  22quinn authored Jul 19, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  b3d82108
- [NVIDIA] Add SM100 Flashinfer MoE blockscale fp8 backend for low latency (#20645) · 6d0734c5
  Kaixi Hou authored Jul 19, 2025
```
Signed-off-by: kaixih <kaixih@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  6d0734c5
- [BugFix] Fix potential cuda-graph IMA (#21196) · 59f93530
  Lucas Wilkinson authored Jul 19, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  59f93530
- [Bugfix] Fix ndarray video color from VideoAsset (#21064) · 18e519ec
  Isotr0py authored Jul 19, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  18e519ec