Commits · a33ea28b1be64f0b57e9eb90389dd36715c60ecb · OpenDAS / vllm_cscc · GitLab

29 Jul, 2025 23 commits
- Add `flashinfer_python` to CUDA wheel requirements (#21389) · a33ea28b
  Michael Goin authored Jul 29, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  a33ea28b
- [Doc] update Contributing page's testing section (#18272) · 7b49cb1c
  David Xia authored Jul 29, 2025
```
Signed-off-by: David Xia <david@davidxia.com>
```
  7b49cb1c
- [Doc] Add FusedMoE Modular Kernel Documentation (#21623) · f03e9cf2
  Varun Sundar Rabindranath authored Jul 29, 2025
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  f03e9cf2
- [Docs] use `uv` in GPU installation docs (#20277) · 37f86d90
  David Xia authored Jul 29, 2025
```
Signed-off-by: David Xia <david@davidxia.com>
```
  37f86d90
- [Bugfix] Fix workspace buffer None issue for Flashinfer TRTLLM Backend (#21525) · 58b11b24
  elvischenv authored Jul 29, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  58b11b24
- [Bugfix]fix mixed bits and visual language model quantization in AutoRound (#21802) · ad341c51
  Wenhua Cheng authored Jul 29, 2025
```
Signed-off-by: Wenhua Cheng <wenhua.cheng@intel.com>
```
  ad341c51
- [TPU] Add an optimization doc on TPU (#21155) · 759b87ef
  Brittany authored Jul 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  759b87ef
- [Docs] Merge design docs for a V1 only future (#21832) · f693b067
  Harry Mellor authored Jul 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  f693b067
- [Bugfix] VLLM_V1 supports passing other compilation levels (#19340) · 04e38500
  Richard Zou authored Jul 29, 2025
```
Signed-off-by: Richard Zou <zou3519@gmail.com>
```
  04e38500
- [Doc] Update compatibility matrix for pooling and multimodal models (#21831) · ab714131
  Cyrus Leung authored Jul 29, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ab714131
- [KVCache] Make KVCacheSpec hashable (#21791) · 755fa8b6
  Chen Zhang authored Jul 29, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  755fa8b6
- [Docs] Fix the outdated URL for installing from vLLM binaries (#21523) · 24704191
  Kay Yan authored Jul 29, 2025
```
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  24704191
- [Model] Refactor JambaForCausalLM (#21394) · 61a6905a
  Jee Jee Li authored Jul 29, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  61a6905a
- [V0 deprecation] Guided decoding (#21347) · 37efc63b
  Reza Barazesh authored Jul 29, 2025
```
Signed-off-by: Reza Barazesh <rezabarazesh@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  37efc63b
- [Model]: Fused MoE for nomic-embed-text-v2-moe (#18321) · a4528f0c
  Isotr0py authored Jul 29, 2025
```
Signed-off-by: isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  a4528f0c
- [Doc] Link to RFC for pooling optimizations (#21806) · a2480251
  Cyrus Leung authored Jul 29, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a2480251
- [Misc] Rework process titles (#21780) · 7234fe26
  Nick Hill authored Jul 29, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  7234fe26
- Migrate InternVLImageInputs and InternVLVideoInputs to TensorSchema (#21684) · f1e2c095
  Benji Beck authored Jul 28, 2025
```
Signed-off-by: Benji Beck <benjibeck@meta.com>
```
  f1e2c095
- [AMD][CI/Build][Bugfix] Guarding CUDA specific functions by ifndef ROCM (#21766) · 12a223ef
  Gregory Shtrasberg authored Jul 28, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  12a223ef
- skip fusedmoe layer for start_load_kv (#21378) · e18f0851
  Calvin Chen authored Jul 29, 2025
```
Signed-off-by: calvin chen <wen.chen@dynamia.ai>
```
  e18f0851
- [CI] Parallelize Kernels MoE Test (#21764) · afa26075
  Michael Goin authored Jul 28, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  afa26075
- [Refactor] Merge Compressed Tensor FP8 `CompressedTensorsW8A8Fp8MoEMethod` and... · 48b763d6
  Wentao Ye authored Jul 28, 2025
```
[Refactor] Merge Compressed Tensor FP8 `CompressedTensorsW8A8Fp8MoEMethod` and `CompressedTensorsW8A8Fp8MoECutlassMethod` (#21775)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  48b763d6
- [Docs] Minimize spacing for supported_hardware.md table (#21779) · 947e982e
  Michael Goin authored Jul 28, 2025
  
  947e982e
28 Jul, 2025 17 commits
- [Kernel] SM90 CUTLASS FP8 GEMM: add support for swap AB + kernel tuning (#20396) · c6c9122d
  lyrisz authored Jul 28, 2025
```
Signed-off-by: Faqin Zhong <faqin.zhong@gmail.com>
Co-authored-by: Duncan Moss <djm.moss@gmail.com>
```
  c6c9122d
- [Perf] Disable chunked local attention by default with llama4 (#21761) · 8aa1485f
  Lucas Wilkinson authored Jul 28, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  8aa1485f
- [Feat]: Add support for Dynamic Quant 4 bit CPU kleidiai kernels (#17112) · 89ac266b
  Nikhil Gupta authored Jul 28, 2025
```
Signed-off-by: Nikhil Gupta <nikhil.gupta2@arm.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  89ac266b
- [Bugfix] DeepGEMM is not enabled on B200 due to `_lazy_init()` (#21472) · c6f36cfa
  Clayton Coleman authored Jul 28, 2025
```
Signed-off-by: Clayton Coleman <smarterclayton@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  c6f36cfa
- Revert "[V1] Exception Handling when Loading KV Cache from Remote Store" (#21778) · b18b417f
  Kuntai Du authored Jul 28, 2025
```
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
```
  b18b417f
- [AMD][CI/Build] Fix the AMD issue caused by inappropriate of symbol exposure (#21647) · 9ba1c88a
  Lu Fang authored Jul 28, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  9ba1c88a
- [Bug] Enforce contiguous input for `dynamic_scaled_fp8_quant` and... · e0e58f97
  Wentao Ye authored Jul 28, 2025
```
[Bug] Enforce contiguous input for `dynamic_scaled_fp8_quant` and `static_scaled_fp8_quant` (#21773)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  e0e58f97
- [AMD][BugFix] Fix omission of wvSplitK kernel for small batch sizes (1-4) due... · b361f14e
  rasmith authored Jul 28, 2025
```
[AMD][BugFix] Fix omission  of wvSplitK kernel for small batch sizes (1-4) due to torch.compile (#21350)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  b361f14e
- update flashinfer to v0.2.9rc2 (#21701) · 01c753ed
  weiliang authored Jul 29, 2025
```
Signed-off-by: Weiliang Liu <weiliangl@nvidia.com>
```
  01c753ed
- Use `metavar` to list the choices for a CLI arg when custom values are also accepted (#21760) · 94b71ae1
  Harry Mellor authored Jul 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  94b71ae1
- [P/D] Log warnings related to prefill KV expiry (#21753) · 7d44c691
  Nick Hill authored Jul 28, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  7d44c691
- [Bugfix] Fix granite speech shape validation (#21762) · e17a4d3b
  Cyrus Leung authored Jul 29, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e17a4d3b
- [XPU] IPEX-optimized Punica Wrapper on XPU (#21703) · ec261b02
  Chaojun Zhang authored Jul 29, 2025
```
Signed-off-by: chzhang <chaojun.zhang@intel.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ec261b02
- [CI/Build] Fix plugin tests (#21758) · 04fe61aa
  Cyrus Leung authored Jul 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  04fe61aa
- [Bugfix] Mistral crashes on tool with no description (#21167) · 25708d31
  Michard Hugo authored Jul 28, 2025
```
Signed-off-by: HugoMichard <hugo@harfanglab.fr>
```
  25708d31
- [Misc] Reduce logs for model resolution (#21765) · 0e18a5d0
  Cyrus Leung authored Jul 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0e18a5d0
- [Logs] Change flashinfer sampler logs to once (#21759) · 34a20c49
  Michael Goin authored Jul 28, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  34a20c49