Commits · e7c4f9ee866296f49ac392fc32a662da0204e70c · OpenDAS / vllm_cscc · GitLab

26 Jul, 2025 25 commits
- [CI/Build][Doc] Move existing benchmark scripts in CI/document/example to vllm bench CLI (#21355) · e7c4f9ee
  Ye (Charlotte) Qi authored Jul 26, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  e7c4f9ee
- [Bugfix][Apple Silicon] fix missing symbols when build from source on Mac with... · 9094d11c
  Yeju Zhou authored Jul 26, 2025
```
[Bugfix][Apple Silicon] fix missing symbols when build from source on Mac with Apple Silicon (#21380)
Signed-off-by: Yeju Zhou <yejuzhou@outlook.com>
```
  9094d11c
- [Refactor] Remove `moe_align_block_size_triton` (#21335) · 56e544f2
  Wentao Ye authored Jul 26, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  56e544f2
- [BugFix] Fix shared storage connector load kv only load attention layer (#21428) · 97d6c30c
  WeiQing Chen authored Jul 26, 2025
```
Signed-off-by: David Chen <530634352@qq.com>
```
  97d6c30c
- [Misc] Improve memory profiling debug message (#21429) · a40a8506
  Ye (Charlotte) Qi authored Jul 26, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  a40a8506
- [Bug] Fix `has_flashinfer_moe` Import Error when it is not installed (#21634) · c215f5c8
  Wentao Ye authored Jul 26, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  c215f5c8
- Support encoder-only models without KV-Cache (#21270) · 1cd6eaba
  Maximilien de Bayser authored Jul 26, 2025
```
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
```
  1cd6eaba
- [Bugfix] Investigate Qwen2-VL failing test (#21527) · f27fdfc3
  Isotr0py authored Jul 26, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  f27fdfc3
- Migrate AyaVisionImagePixelInputs to TensorSchema for shape validation (#21622) · de10ff0b
  Benji Beck authored Jul 26, 2025
```
Signed-off-by: Benji Beck <benjibeck@meta.com>
```
  de10ff0b
- Migrate AriaImagePixelInputs to TensorSchema for shape validation (#21620) · 9d197280
  Benji Beck authored Jul 26, 2025
```
Signed-off-by: Benji Beck <benjibeck@meta.com>
```
  9d197280
- [Take 2] Correctly kill vLLM processes after benchmarks (#21646) · e98def43
  Huy Do authored Jul 26, 2025
```
Signed-off-by: Huy Do <huydhn@gmail.com>
```
  e98def43
- [Misc] remove unused try-except in pooling config check (#21618) · 05c1126f
  Reid authored Jul 26, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
```
  05c1126f
- Support Intern-S1 (#21628) · 875af38e
  Lyu Han authored Jul 26, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.me>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Roger Wang <hey@rogerw.me>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  875af38e
- [TPU][Test] Divide TPU v1 Test into 2 parts. (#21431) · 7728dd77
  QiliangCui authored Jul 25, 2025
  
  7728dd77
- [Bugfix] Fix isinstance check for tensor types in _load_prompt_embeds to use... · 2f6e6b33
  Alexandre JUAN authored Jul 26, 2025
```
[Bugfix] Fix isinstance check for tensor types in _load_prompt_embeds to use dtype comparison (#21612)
Signed-off-by: Alexandre Juan <a.juan@netheos.net>
```
  2f6e6b33
- Correctly kill vLLM processes after finishing serving benchmarks (#21641) · a55c9509
  Huy Do authored Jul 25, 2025
```
Signed-off-by: Huy Do <huydhn@gmail.com>
```
  a55c9509
- [Docs] add offline serving multi-modal video input expamle Qwen2.5-VL (#21530) · 97349fe2
  WeiQing Chen authored Jul 26, 2025
```
Signed-off-by: David Chen <530634352@qq.com>
```
  97349fe2
- [Model] Ultravox: Support Llama 4 and Gemma 3 backends (#17818) · 62965de5
  Farzad Abdolhosseini authored Jul 26, 2025
```
Signed-off-by: Farzad Abdolhosseini <farzad@fixie.ai>
Signed-off-by: Patrick Li <patrick8289@gmail.com>
Co-authored-by: Patrick Li <patrick8289@gmail.com>
```
  62965de5
- [Feature] Add support for MoE models in the calibration-free RTN-based quantization (#20766) · 7ae75fa6
  Alex Kogan authored Jul 25, 2025
```
Signed-off-by: Alex Kogan <alex.kogan@oracle.com>
```
  7ae75fa6
- [TPU] Update ptxla nightly version to 20250724 (#21555) · f1b286b2
  Chengji Yao authored Jul 25, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  f1b286b2
- [Bugfix] Always set RAY_ADDRESS for Ray actor before spawn (#21540) · c7742d61
  Rui Qiao authored Jul 25, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  c7742d61
- [Bugfix] Fix sync_and_slice_intermediate_tensors (#21537) · cea96a01
  Rui Qiao authored Jul 25, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  cea96a01
- Add interleaved RoPE test for Llama4 (Maverick) (#21478) · 2eddd437
  Yong Hoon Shin authored Jul 25, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  2eddd437
- [Perf] Cuda Kernel for Int8 Per Token Group Quant (#21476) · 75d29cf4
  Wentao Ye authored Jul 25, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  75d29cf4
- Add Unsloth to RLHF.md (#21636) · 41d3082c
  Daniel Han authored Jul 25, 2025
  
  41d3082c
25 Jul, 2025 15 commits
- [TPU][Test] Rollback PR-21550. (#21619) · 7cfea0df
  QiliangCui authored Jul 25, 2025
```
Signed-off-by: Qiliang Cui <derrhein@gmail.com>
```
  7cfea0df
- [Docs] add auto-round quantization readme (#21600) · 5ac3168e
  Wenhua Cheng authored Jul 25, 2025
```
Signed-off-by: Wenhua Cheng <wenhua.cheng@intel.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  5ac3168e
- [CI] Unifying Dockerfiles for ARM and X86 Builds (#21343) · 396ee941
  Kebe authored Jul 25, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
  396ee941
- Add support for Prithvi in Online serving mode (#21518) · e189b50f
  mgazz authored Jul 25, 2025
```
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  e189b50f
- [Kernel] Improve machete memory bound perf (#21556) · 136d750f
  czhu-cohere authored Jul 25, 2025
```
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
```
  136d750f
- [ROCm][AITER] Enable fp8 kv cache on rocm aiter backend. (#20295) · b3caeb82
  who who who authored Jul 25, 2025
```
Signed-off-by: fsx950223 <fsx950223@outlook.com>
Signed-off-by: amd-ruitang3 <Rui.Tang2@amd.com>
Co-authored-by: amd-ruitang3 <Rui.Tang2@amd.com>
```
  b3caeb82
- [Model] Replace Mamba2 RMSNorm Gated with Fused Triton Kernel (#20839) · eab2f398
  Chih-Chieh Yang authored Jul 25, 2025
```
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
Signed-off-by: Yu Chin Fabian Lim <fabian.lim@gmail.com>
Signed-off-by: Chih-Chieh Yang <7364402+cyang49@users.noreply.github.com>
Co-authored-by: Yu Chin Fabian Lim <fabian.lim@gmail.com>
```
  eab2f398
- [Frontend] Add request_id to the Request object so they can be controlled... · 9fe98d42
  kourosh hakhamaneshi authored Jul 25, 2025
```
[Frontend] Add request_id to the Request object so they can be controlled better via external load balancers (#21009)
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
```
  9fe98d42
- [MODEL] New model support for naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B (#20931) · 29c6fbe5
  bigshanedogg authored Jul 25, 2025
```
Signed-off-by: bigshanedogg <bigshane319@gmail.com>
```
  29c6fbe5
- [Model] Fix Ernie4.5MoE e_score_correction_bias parameter (#21586) · c72f049c
  xyxinyang authored Jul 25, 2025
```
Signed-off-by: zhouchong <zhouchong03@baidu.com>
Co-authored-by: zhouchong <zhouchong03@baidu.com>
```
  c72f049c
- [Bugfix][Logprobs] Fix logprobs op to support more backend (#21591) · f3a683b7
  Mengqing Cao authored Jul 25, 2025
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  f3a683b7
- [V1] Get supported tasks from model runner instead of model config (#21585) · 46d81d69
  Cyrus Leung authored Jul 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  46d81d69
- [Quantization] Enable BNB support for more MoE models (#21370) · 5c3f2628
  Jee Jee Li authored Jul 25, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  5c3f2628
- [Bugfix] GGUF: fix AttributeError: 'PosixPath' object has no attribute 'startswith' (#21579) · 7311f744
  Kebe authored Jul 25, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
  7311f744
- Add H20-3e fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct (#21598) · 8ed01e32
  Xu Wenqing authored Jul 25, 2025
```
Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
```
  8ed01e32