Commits · 4f5b059f146adeecd153fa781cf21863ed6679d8 · OpenDAS / vllm_cscc · GitLab

04 Mar, 2025 21 commits
- Clean up unused padding_idx variables across many model definitions (#13240) · 4f5b059f
  Tyler Michael Smith authored Mar 04, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  4f5b059f
- [Security] Serialize using safetensors instead of pickle in Mooncake Pipe (#14228) · 288ca110
  Kuntai Du authored Mar 04, 2025
```
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
```
  288ca110
- [v1][Metrics] Add design doc (#12745) · c2bd2196
  Mark McLoughlin authored Mar 04, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  c2bd2196
- [Docs] Update Dockerfile dependency image (#14215) · 550c7ba3
  Michael Goin authored Mar 04, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  550c7ba3
- [Frontend] Do `prompt_logprobs` clamping for chat as well as completions (#14225) · e5b2f160
  Harry Mellor authored Mar 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e5b2f160
- Fix performance when `--generation-config` is not `None` (#14223) · 9badee53
  Harry Mellor authored Mar 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  9badee53
- [TPU][Profiler] Support start_profile/stop_profile in TPU worker (#13988) · beebf474
  Siyuan Liu authored Mar 04, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  beebf474
- add cutlass support for blackwell fp8 gemm (#13798) · f89978ad
  kushanam authored Mar 04, 2025
  
  f89978ad
- [V1][Molmo] Fix get_multimodal_embeddings() in molmo.py (#14161) · b3cf368d
  lkchen authored Mar 04, 2025
  
  b3cf368d
- [V0][Metrics] Deprecate some questionable request time metrics (#14135) · c8525f06
  Mark McLoughlin authored Mar 04, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  c8525f06
- [V1][BugFix] Fix remaining sync engine client shutdown errors/hangs (#13869) · 5db6b2c9
  Nick Hill authored Mar 04, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  5db6b2c9
- [Bugfix] Restrict MacOS CPU detection (#14210) · 6247bae6
  Michael Goin authored Mar 04, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  6247bae6
- [doc] add "Failed to infer device type" to faq (#14200) · 3610fb49
  youkaichao authored Mar 04, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3610fb49
- [sleep mode] error out with expandable_segments (#14189) · 71c4b405
  youkaichao authored Mar 04, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  71c4b405
- [platform] add debug logging during inferring the device type (#14195) · ac65bc92
  youkaichao authored Mar 04, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ac65bc92
- Fix benchmark_moe.py tuning for CUDA devices (#14164) · f78c0be8
  Michael Goin authored Mar 04, 2025
  
  f78c0be8
- Use math.prod instead of np.prod for trivial ops (#14142) · 66233af7
  Zhanwen Chen authored Mar 04, 2025
  
  66233af7
- [core] Pass all driver env vars to ray workers unless excluded (#14099) · bf13d409
  Rui Qiao authored Mar 03, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  bf13d409
- [Misc] Remove lru_cache in NvmlCudaPlatform (#14156) · 989f4f43
  Cody Yu authored Mar 03, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  989f4f43
- [core] moe fp8 block quant tuning support (#14068) · bb5b6403
  Divakar Verma authored Mar 03, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  bb5b6403
- [Model] Add support for GraniteMoeShared models (#13313) · c060b714
  Travis Johnson authored Mar 03, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  c060b714
03 Mar, 2025 17 commits
- [v1] Add comments to the new ragged paged attention Pallas kernel (#14155) · 79e4937c
  iefgnoix authored Mar 03, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  79e4937c
- [Docs] Add GPTQModel (#14056) · cd1d3c3d
  Qubitium-ModelCloud authored Mar 04, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  cd1d3c3d
- [Kernel] Optimize moe intermediate_cache usage (#13625) · 19d98e0c
  Michael Goin authored Mar 03, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  19d98e0c
- [Bugfix] Allow shared_experts skip quantization for DeepSeekV2/V3 (#14100) · 2b04c209
  Michael Goin authored Mar 03, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  2b04c209
- [WIP][[V1][Metrics] Implement max_num_generation_tokens, request_params_n,... · ae122b1c
  Mark McLoughlin authored Mar 03, 2025
```
[WIP][[V1][Metrics] Implement max_num_generation_tokens,  request_params_n, and request_params_max_tokens metrics (#14055)
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  ae122b1c
- [V1] Simplify stats logging (#14082) · 872db2be
  Nick Hill authored Mar 03, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  872db2be
- [V0][Metrics] Deprecate some KV/prefix cache metrics (#14136) · 2dfdfed8
  Mark McLoughlin authored Mar 03, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  2dfdfed8
- [V0][Metrics] Remove unimplemented `vllm:tokens_total` (#14134) · c41d2715
  Mark McLoughlin authored Mar 03, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  c41d2715
- Fix `head_dim` not existing in all model configs (Transformers backend) (#14141) · 91373a0d
  Harry Mellor authored Mar 03, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  91373a0d
- [ROCm] Faster Custom Paged Attention kernels (#12348) · 848a6438
  TJian authored Mar 04, 2025
  
  848a6438
- Improve the docs for `TransformersModel` (#14147) · 98175b28
  Harry Mellor authored Mar 03, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  98175b28
- [V1] Refactor parallel sampling support (#13774) · 4167252e
  Mark McLoughlin authored Mar 03, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  4167252e
- [Build] Make sure local main branch is synced when VLLM_USE_PRECOMPILED=1 (#13921) · f35f8e22
  Cody Yu authored Mar 03, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  f35f8e22
- [Misc][Platform] Move use allgather to platform (#14010) · b87c21fc
  Mengqing Cao authored Mar 03, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  b87c21fc
- [Misc] duplicate code in deepseek_v2 (#14106) · e584b85a
  wang.yuqi authored Mar 03, 2025
  
  e584b85a
- [Bugfix] Explicitly include "omp.h" for MacOS to avoid installation failure (#14051) · 09e56f92
  Sheng Yao authored Mar 03, 2025
  
  09e56f92
- Update deprecated Python 3.8 typing (#13971) · cf069aa8
  Harry Mellor authored Mar 03, 2025
  
  cf069aa8
02 Mar, 2025 2 commits
- [v0][structured output] Support reasoning output (#12955) · bf33700e
  Ce Gao authored Mar 03, 2025
```
Signed-off-by: Ce Gao <cegao@tensorchord.ai>
```
  bf33700e
- [Doc] Source building add clone step (#14086) · bc6ccb98
  qux-bbb authored Mar 02, 2025
```
Signed-off-by: qux-bbb <1147635419@qq.com>
```
  bc6ccb98