Commits · ecff8309a3ca5159ac09ac9a7976516b9301f64d · OpenDAS / vllm_cscc · GitLab

27 Mar, 2025 11 commits
- [ROCm] Env variable to trigger custom PA (#15557) · ecff8309
  Gregory Shtrasberg authored Mar 27, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  ecff8309
- Allow torchao quantization in SiglipMLP (#15575) · dcf2a590
  Jerry Zhang authored Mar 26, 2025
  
  dcf2a590
- [V1] Refactor num_computed_tokens logic (#15307) · 54aa6194
  Cody Yu authored Mar 26, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  54aa6194
- [moe][quant] add weight name case for offset (#15515) · fb22be58
  Mengqing Cao authored Mar 27, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  fb22be58
- [Doc] Update V1 user guide for fp8 kv cache support (#15585) · 7f301dd8
  Wei Zeng authored Mar 26, 2025
```
Signed-off-by: weizeng <weizeng@roblox.com>
```
  7f301dd8
- [misc] LoRA: Remove unused long context test data (#15558) · 8095341a
  Varun Sundar Rabindranath authored Mar 26, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  8095341a
- add platform check back (#15578) · 69db16a4
  Chenyaaang authored Mar 26, 2025
```
Signed-off-by: Chenyaaang <llccyy1212@gmail.com>
```
  69db16a4
- Add automatic tpu label to mergify.yml (#15560) · ce78f9af
  Michael Goin authored Mar 26, 2025
  
  ce78f9af
- [Kernel] CUTLASS grouped gemm fp8 MoE kernel (#13972) · 9239bf71
  ElizaWszola authored Mar 27, 2025
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
```
  9239bf71
- Support FIPS enabled machines with MD5 hashing (#15299) · 7a6d45bc
  Matthew Vine authored Mar 26, 2025
```
Signed-off-by: Matthew Vine <32849887+MattTheCuber@users.noreply.github.com>
```
  7a6d45bc
- [TPU] support disabling xla compilation cache (#15567) · e74ff409
  Chengji Yao authored Mar 26, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  e74ff409
26 Mar, 2025 24 commits
- Use Cache Hinting for fused_moe kernel (#15511) · 7a888271
  Wes authored Mar 26, 2025
  
  7a888271
- [V1] TPU CI - Fix test_compilation.py (#15570) · 9d119a86
  Alexander Matveev authored Mar 26, 2025
```
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
```
  9d119a86
- [V1] TPU - Revert to exponential padding by default (#15565) · b2e85e26
  Alexander Matveev authored Mar 26, 2025
```
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
```
  b2e85e26
- Applying some fixes for K8s agents in CI (#15493) · dd8a29da
  Alexei-V-Ivanov-AMD authored Mar 26, 2025
```
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
```
  dd8a29da
- Support SHA256 as hash function in prefix caching (#15297) · 27df5199
  marko authored Mar 26, 2025
```
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
```
  27df5199
- [V1][Sampler] Faster top-k only implementation (#15478) · 35fad35a
  Nick Hill authored Mar 26, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  35fad35a
- [Refactor] Remove unnecessary backend parameter in structured output interface (#15317) · 733e7c9e
  Aaron Pham authored Mar 26, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  733e7c9e
- Fix weight loading for some models in Transformers backend (#15544) · 0af4d764
  Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0af4d764
- multi-node offline DP+EP example (#15484) · e64afa45
  youkaichao authored Mar 26, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  e64afa45
- [Model] Add Reasoning Parser for Granite Models (#14202) · 1711b929
  Alex Brooks authored Mar 26, 2025
```
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
```
  1711b929
- Improve validation of TP in Transformers backend (#15540) · c091c0a5
  Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c091c0a5
- Apply torchfix (#15532) · 1aa162e0
  cyyever authored Mar 26, 2025
```
Signed-off-by: cyy <cyyever@outlook.com>
```
  1aa162e0
- Separate base model from `TransformersModel` (#15467) · cf5c8f16
  Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  cf5c8f16
- [Misc] improve example script output (#15528) · 4ec2cee0
  Reid authored Mar 26, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  4ec2cee0
- [Misc] Enhance warning information to user-defined chat template (#15408) · 99f536f8
  wwl2755 authored Mar 26, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  99f536f8
- [FEAT][ROCm] Integrate Fused MoE Kernels from AITER (#14967) · 5ebf6674
  vllmellm authored Mar 26, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  5ebf6674
- [Feature] Enhance EAGLE Architecture with Proper RMS Norms (#14990) · 781d0562
  Bryan Lu authored Mar 26, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  781d0562
- Fix raw_request extraction in load_aware_call decorator (#15382) · 5aefd6ac
  daniel-salib authored Mar 25, 2025
```
Signed-off-by: Daniel Salib <danielsalib@meta.com>
```
  5aefd6ac
- [misc] LoRA - Skip LoRA kernels when not required (#15152) · 6c663dfd
  Varun Sundar Rabindranath authored Mar 25, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  6c663dfd
- [BugFix] Fix nightly MLA failure (FA2 + MLA chunked prefill, i.e. V1,... · 33437bc6
  Lucas Wilkinson authored Mar 25, 2025
```
[BugFix] Fix nightly MLA failure (FA2 + MLA chunked prefill, i.e. V1, producing bad results) (#15492)
Signed-off-by: LucasWilkinson <lwilkinson@neuralmagic.com>
```
  33437bc6
- [Misc] Warn about v0 in benchmark_paged_attn.py (#15495) · 23114d33
  Tyler Michael Smith authored Mar 25, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  23114d33
- [Model] Support multi-image for Molmo (#15438) · 997c8811
  Cyrus Leung authored Mar 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  997c8811
- Transformers backend already supports V1 (#15463) · e42389f9
  Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e42389f9
- [CI/Build] LoRA: Delete long context tests (#15503) · ff38f0a3
  Varun Sundar Rabindranath authored Mar 25, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  ff38f0a3
25 Mar, 2025 5 commits
- [Core] LoRA: V1 Scheduler optimization (#15422) · a5cfbab3
  Varun Sundar Rabindranath authored Mar 25, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  a5cfbab3
- [core] add bucket padding to tpu_model_runner (#14995) · ac3cd6e8
  Chenyaaang authored Mar 25, 2025
```
Signed-off-by: Chenyaaang <llccyy1212@gmail.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  ac3cd6e8
- [V1] Support long_prefill_token_threshold in v1 scheduler (#15419) · 082ab86f
  Lu Fang authored Mar 25, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  082ab86f
- [V1][Minor] Use `SchedulerInterface` type for engine scheduler field (#15499) · 6aa196c8
  Nick Hill authored Mar 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  6aa196c8
- [TPU][V1] Fix Sampler recompilation (#15309) · a0dd7dcd
  Nicolò Lucchesi authored Mar 25, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  a0dd7dcd