Commits · c8525f06fcc99dbc3564eaed9985edf16d287b47 · OpenDAS / vllm_cscc · GitLab

04 Mar, 2025 10 commits
- [V0][Metrics] Deprecate some questionable request time metrics (#14135) · c8525f06
  Mark McLoughlin authored Mar 04, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  c8525f06
- [V1][BugFix] Fix remaining sync engine client shutdown errors/hangs (#13869) · 5db6b2c9
  Nick Hill authored Mar 04, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  5db6b2c9
- [Bugfix] Restrict MacOS CPU detection (#14210) · 6247bae6
  Michael Goin authored Mar 04, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  6247bae6
- [sleep mode] error out with expandable_segments (#14189) · 71c4b405
  youkaichao authored Mar 04, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  71c4b405
- [platform] add debug logging during inferring the device type (#14195) · ac65bc92
  youkaichao authored Mar 04, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ac65bc92
- Use math.prod instead of np.prod for trivial ops (#14142) · 66233af7
  Zhanwen Chen authored Mar 04, 2025
  
  66233af7
- [core] Pass all driver env vars to ray workers unless excluded (#14099) · bf13d409
  Rui Qiao authored Mar 03, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  bf13d409
- [Misc] Remove lru_cache in NvmlCudaPlatform (#14156) · 989f4f43
  Cody Yu authored Mar 03, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  989f4f43
- [core] moe fp8 block quant tuning support (#14068) · bb5b6403
  Divakar Verma authored Mar 03, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  bb5b6403
- [Model] Add support for GraniteMoeShared models (#13313) · c060b714
  Travis Johnson authored Mar 03, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  c060b714
03 Mar, 2025 14 commits
- [v1] Add comments to the new ragged paged attention Pallas kernel (#14155) · 79e4937c
  iefgnoix authored Mar 03, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  79e4937c
- [Kernel] Optimize moe intermediate_cache usage (#13625) · 19d98e0c
  Michael Goin authored Mar 03, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  19d98e0c
- [Bugfix] Allow shared_experts skip quantization for DeepSeekV2/V3 (#14100) · 2b04c209
  Michael Goin authored Mar 03, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  2b04c209
- [WIP][[V1][Metrics] Implement max_num_generation_tokens, request_params_n,... · ae122b1c
  Mark McLoughlin authored Mar 03, 2025
```
[WIP][[V1][Metrics] Implement max_num_generation_tokens,  request_params_n, and request_params_max_tokens metrics (#14055)
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  ae122b1c
- [V1] Simplify stats logging (#14082) · 872db2be
  Nick Hill authored Mar 03, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  872db2be
- [V0][Metrics] Deprecate some KV/prefix cache metrics (#14136) · 2dfdfed8
  Mark McLoughlin authored Mar 03, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  2dfdfed8
- [V0][Metrics] Remove unimplemented `vllm:tokens_total` (#14134) · c41d2715
  Mark McLoughlin authored Mar 03, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  c41d2715
- Fix `head_dim` not existing in all model configs (Transformers backend) (#14141) · 91373a0d
  Harry Mellor authored Mar 03, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  91373a0d
- [ROCm] Faster Custom Paged Attention kernels (#12348) · 848a6438
  TJian authored Mar 04, 2025
  
  848a6438
- [V1] Refactor parallel sampling support (#13774) · 4167252e
  Mark McLoughlin authored Mar 03, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  4167252e
- [Build] Make sure local main branch is synced when VLLM_USE_PRECOMPILED=1 (#13921) · f35f8e22
  Cody Yu authored Mar 03, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  f35f8e22
- [Misc][Platform] Move use allgather to platform (#14010) · b87c21fc
  Mengqing Cao authored Mar 03, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  b87c21fc
- [Misc] duplicate code in deepseek_v2 (#14106) · e584b85a
  wang.yuqi authored Mar 03, 2025
  
  e584b85a
- Update deprecated Python 3.8 typing (#13971) · cf069aa8
  Harry Mellor authored Mar 03, 2025
  
  cf069aa8
02 Mar, 2025 3 commits
- [v0][structured output] Support reasoning output (#12955) · bf33700e
  Ce Gao authored Mar 03, 2025
```
Signed-off-by: Ce Gao <cegao@tensorchord.ai>
```
  bf33700e
- [Misc] Accurately capture the time of loading weights (#14063) · 82fbeae9
  Jun Duan authored Mar 01, 2025
```
Signed-off-by: Jun Duan <jun.duan.phd@outlook.com>
```
  82fbeae9
- [Model] Add LoRA support for TransformersModel (#13770) · cc5e8f6d
  Jee Jee Li authored Mar 02, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  cc5e8f6d
01 Mar, 2025 8 commits
- [v1] Add `__repr__` to KVCacheBlock to avoid recursive print (#14081) · d54990da
  Chen Zhang authored Mar 02, 2025
  
  d54990da
- [v1][Bugfix] Only cache blocks that are not in the prefix cache (#14073) · b9f1d429
  Chen Zhang authored Mar 01, 2025
  
  b9f1d429
- [ROCm][V1][Bugfix] Add get_builder_cls method to the ROCmAttentionBackend class (#14065) · b28246f6
  Sage Moore authored Feb 28, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  b28246f6
- [V1][Minor] Do not print attn backend twice (#13985) · 3b5567a2
  Woosuk Kwon authored Feb 28, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3b5567a2
- [Doc] Consolidate `whisper` and `florence2` examples (#14050) · fdcc4053
  Isotr0py authored Mar 01, 2025
  
  fdcc4053
- [Bugfix][V1][Minor] Fix shutting_down flag checking in V1 MultiprocExecutor (#14053) · 02296f42
  Li, Jiang authored Mar 01, 2025
  
  02296f42
- [Misc][Kernel]: Add GPTQAllSpark Quantization (#12931) · 6a92ff93
  YajieWang authored Mar 01, 2025
  
  6a92ff93
- [Bugfix] Add file lock for ModelScope download (#14060) · 6a84164a
  Jee Jee Li authored Mar 01, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  6a84164a
28 Feb, 2025 5 commits
- [torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename... · bd56c983
  Luka Govedič authored Feb 28, 2025
```
[torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename RedundantReshapesPass to NoopEliminationPass (#10902)
Signed-off-by: luka <luka@neuralmagic.com>
```
  bd56c983
- [core] Bump ray to 2.43 (#13994) · 084bbac8
  Rui Qiao authored Feb 28, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  084bbac8
- [v1] Move block pool operations to a separate class (#13973) · 28943d36
  Chen Zhang authored Mar 01, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  28943d36
- [v1] Cleanup the BlockTable in InputBatch (#13977) · e7bd944e
  Chen Zhang authored Mar 01, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  e7bd944e
- [V1][TPU] Integrate the new ragged paged attention kernel with vLLM v1 on TPU (#13379) · c3b6559a
  iefgnoix authored Feb 28, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  c3b6559a