Commits · 355f66348c3ddb0a2c3217372f2ee47fb961d58f · OpenDAS / vllm_cscc · GitLab

28 Mar, 2025 11 commits
- [V1] Remove legacy input registry (#15673) · 355f6634
  Cyrus Leung authored Mar 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  355f6634
- [Bugfix] Fix `mm_hashes` forgetting to be passed (#15668) · 8693e47e
  Cyrus Leung authored Mar 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8693e47e
- Refactor error handling for multiple exceptions in preprocessing (#15650) · cec8c7d7
  Jason (Siyu) Zhu authored Mar 27, 2025
```
Signed-off-by: JasonZhu1313 <jasonchu13@outlook.com>
```
  cec8c7d7
- [Quantization][FP8] Adding support for fp8 gemm layer input in fp8 (#14578) · 4d0ec372
  Gregory Shtrasberg authored Mar 27, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  4d0ec372
- [Misc]add coding benchmark for speculative decoding (#15303) · e7f720ea
  Chen Xia authored Mar 27, 2025
```
Signed-off-by: CXIAAAAA <cxia0209@gmail.com>
```
  e7f720ea
- Revert "Use Cache Hinting for fused_moe kernel (#15511)" (#15645) · 4ae17bf1
  Wes authored Mar 27, 2025
```
Signed-off-by: Wes Medford <wryanmedford@gmail.com>
```
  4ae17bf1
- [CI][TPU] Temporarily Disable Quant Test on TPU (#15649) · 8a49eea7
  Robert Shaw authored Mar 27, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  8a49eea7
- [Doc] Fix dead links in Job Board (#15637) · b4245a48
  wwl2755 authored Mar 27, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  b4245a48
- [Bugfix] Fix failure to launch in Tensor Parallel TP mode on macOS. (#14948) · 4e0f6076
  Kebe authored Mar 28, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  4e0f6076
- [Quantization][V1] BitsAndBytes support V1 (#15611) · 726efc6a
  Jee Jee Li authored Mar 28, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  726efc6a
- [TPU] Lazy Import (#15656) · bd45912b
  Robert Shaw authored Mar 27, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  bd45912b
27 Mar, 2025 29 commits
- [V1] AsyncLLM data parallel (#13923) · 15dac210
  Nick Hill authored Mar 27, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  15dac210
- [CI] Update rules for applying `tpu` label. (#15634) · 112b3e5b
  Russell Bryant authored Mar 27, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  112b3e5b
- Correct PowerPC to modern IBM Power (#15635) · 32d66927
  cnorman authored Mar 27, 2025
```
Signed-off-by: Christy Norman <christy@linux.vnet.ibm.com>
```
  32d66927
- [Bugfix][TPU][V1] Fix recompilation (#15553) · 4098b722
  Nicolò Lucchesi authored Mar 27, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  4098b722
- Use absolute placement for Ask AI button (#15628) · 46450b8d
  Harry Mellor authored Mar 27, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  46450b8d
- [Misc] Avoid direct access of global `mm_registry` in `compute_encoder_budget` (#15621) · 13ac9cab
  Cyrus Leung authored Mar 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  13ac9cab
- [Feature] Add middleware to log API Server responses (#15593) · 66aa4c0b
  Yuan Tang authored Mar 27, 2025
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  66aa4c0b
- [Misc] Replace `is_encoder_decoder_inputs` with `split_enc_dec_inputs` (#15620) · 24718153
  Cyrus Leung authored Mar 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  24718153
- [Doc] Link to onboarding tasks (#15629) · 07bf813f
  Cyrus Leung authored Mar 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  07bf813f
- [Bugfix] Fix use_cascade_attention handling for Alibi-based models on vllm/v1 (#15211) · 8958217a
  Hiroaki Sugiyama authored Mar 27, 2025
```
Signed-off-by: h-sugi <h.sugi@ieee.org>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8958217a
- [Model] MiniCPM-V/O supports V1 (#15487) · ac5bc615
  Cyrus Leung authored Mar 27, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ac5bc615
- [Doc] update --system for transformers installation in docker doc (#15616) · 8063dfc6
  Reid authored Mar 27, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  8063dfc6
- Fix incorrect filenames in vllm_compile_cache.py (#15494) · 6278bc82
  Richard Zou authored Mar 27, 2025
```
Signed-off-by: <zou3519@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  6278bc82
- [Misc] Use model_redirect to redirect the model name to a local folder. (#14116) · 3f532cb6
  wang.yuqi authored Mar 27, 2025
  
  3f532cb6
- [Misc] Clean up `scatter_patch_features` (#15559) · e6c9053f
  Cyrus Leung authored Mar 27, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e6c9053f
- [Quantization] Fp8 Channelwise Dynamic Per Token GroupedGEMM (#15587) · 43ed4143
  Robert Shaw authored Mar 27, 2025
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>
```
  43ed4143
- [Misc] Consolidate LRUCache implementations (#15481) · f4c98b4d
  Bella kira authored Mar 27, 2025
```
Signed-off-by: Bella kira <2374035698@qq.com>
```
  f4c98b4d
- [TPU] Avoid Triton Import (#15589) · e1e0fd75
  Robert Shaw authored Mar 27, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  e1e0fd75
- [Misc] Restrict ray version dependency and update PP feature warning in V1 (#15556) · df8d3d12
  Rui Qiao authored Mar 26, 2025
  
  df8d3d12
- [TPU] [V1] fix cases when max_num_reqs is set smaller than MIN_NUM_SEQS (#15583) · 619d3de8
  Chengji Yao authored Mar 26, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  619d3de8
- [ROCm] Env variable to trigger custom PA (#15557) · ecff8309
  Gregory Shtrasberg authored Mar 27, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  ecff8309
- Allow torchao quantization in SiglipMLP (#15575) · dcf2a590
  Jerry Zhang authored Mar 26, 2025
  
  dcf2a590
- [V1] Refactor num_computed_tokens logic (#15307) · 54aa6194
  Cody Yu authored Mar 26, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  54aa6194
- [moe][quant] add weight name case for offset (#15515) · fb22be58
  Mengqing Cao authored Mar 27, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  fb22be58
- [Doc] Update V1 user guide for fp8 kv cache support (#15585) · 7f301dd8
  Wei Zeng authored Mar 26, 2025
```
Signed-off-by: weizeng <weizeng@roblox.com>
```
  7f301dd8
- [misc] LoRA: Remove unused long context test data (#15558) · 8095341a
  Varun Sundar Rabindranath authored Mar 26, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  8095341a
- add platform check back (#15578) · 69db16a4
  Chenyaaang authored Mar 26, 2025
```
Signed-off-by: Chenyaaang <llccyy1212@gmail.com>
```
  69db16a4
- Add automatic tpu label to mergify.yml (#15560) · ce78f9af
  Michael Goin authored Mar 26, 2025
  
  ce78f9af
- [Kernel] CUTLASS grouped gemm fp8 MoE kernel (#13972) · 9239bf71
  ElizaWszola authored Mar 27, 2025
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
```
  9239bf71