Commits · 6a417b8600d4d1e57698a91b71a38446e8fc5c45 · OpenDAS / vllm_cscc · GitLab

20 Feb, 2025 20 commits
- fix neuron performance issue (#13589) · 6a417b86
  ajayvohra2005 authored Feb 20, 2025
  
  6a417b86
- [V1][Minor] Print KV cache size in token counts (#13596) · d3ea5011
  Woosuk Kwon authored Feb 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d3ea5011
- Update `pre-commit`'s `isort` version to remove warnings (#13614) · 34aad515
  Harry Mellor authored Feb 20, 2025
  
  34aad515
- [Bugfix] Fix deepseekv3 grouped topk error (#13474) · ed6e9075
  chenxiaobing authored Feb 20, 2025
```
Signed-off-by: Chen-XiaoBing <chenxb002@whu.edu.cn>
```
  ed6e9075
- Merge similar examples in `offline_inference` into single `basic` example (#12737) · 992e5c3d
  Harry Mellor authored Feb 20, 2025
  
  992e5c3d
- [Kernel] LoRA - Refactor sgmv kernels (#13110) · b69692a2
  Varun Sundar Rabindranath authored Feb 20, 2025
  
  b69692a2
- [2/n][ci] S3: Use full model path (#13564) · a64a8443
  Kevin H. Luu authored Feb 20, 2025
```
Signed-off-by: <>
```
  a64a8443
- [ci] Fix spec decode test (#13600) · aa1e62d0
  Kevin H. Luu authored Feb 20, 2025
  
  aa1e62d0
- [CI/Build] Use uv in the Dockerfile (#13566) · 497bc831
  Michael Goin authored Feb 20, 2025
  
  497bc831
- [API Server] Add port number range validation (#13506) · 3738e6fa
  Yuan Tang authored Feb 20, 2025
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  3738e6fa
- [ROCm] MI300A compile targets deprecation (#13560) · 0023cd2b
  Gregory Shtrasberg authored Feb 20, 2025
  
  0023cd2b
- [Misc] add mm_processor_kwargs to extra_body for Qwen2.5-VL (#13533) · 041e2947
  燃 authored Feb 20, 2025
  
  041e2947
- [Misc] Warn if the vLLM version can't be retrieved (#13501) · 96216678
  Alex Brooks authored Feb 19, 2025
```
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
```
  96216678
- [bugfix] spec decode worker get tp group only when initialized (#13578) · 8c755c3b
  Simon Mo authored Feb 19, 2025
  
  8c755c3b
- [core] add sleep and wake up endpoint and v1 support (#12987) · ba811639
  youkaichao authored Feb 20, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: cennn <2523403608@qq.com>
Co-authored-by: cennn <2523403608@qq.com>
```
  ba811639
- [ROCm][MoE] mi300 mixtral8x7B perf for specific BS (#13577) · 0d243f2a
  Divakar Verma authored Feb 19, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  0d243f2a
- [ci] Add AWS creds for AMD (#13572) · 88f6ba32
  Kevin H. Luu authored Feb 19, 2025
  
  88f6ba32
- [Misc] Qwen2.5 VL support LoRA (#13261) · 512368e3
  Jee Jee Li authored Feb 20, 2025
  
  512368e3
- [3/n][CI] Load Quantization test models with S3 (#13570) · 473f51cf
  Kevin H. Luu authored Feb 19, 2025
```
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
```
  473f51cf
- [BugFix] Avoid error traceback in logs when V1 `LLM` terminates (#13565) · a4c402a7
  Nick Hill authored Feb 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  a4c402a7
19 Feb, 2025 20 commits
- [Misc] Avoid calling unnecessary `hf_list_repo_files` for local model path (#13348) · 550d97eb
  Isotr0py authored Feb 20, 2025
```
Signed-off-by: isotr0py <2037008807@qq.com>
```
  550d97eb
- [MISC] Logging the message about Ray teardown (#13502) · fbbe1fba
  Cody Yu authored Feb 19, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com>
```
  fbbe1fba
- Fix copyright year to auto get current year (#13561) · 01c184b8
  Wilson Wu authored Feb 20, 2025
  
  01c184b8
- [doc] clarify multi-node serving doc (#13558) · ad5a35c2
  youkaichao authored Feb 19, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ad5a35c2
- [Bugfix] Fix device ordinal for multi-node spec decode (#13269) · 5ae9f26a
  shangmingc authored Feb 19, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  5ae9f26a
- [VLM][Bugfix] Pass processor kwargs properly on init (#13516) · 377d10bd
  Cyrus Leung authored Feb 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  377d10bd
- [doc] clarify profiling is only for developers (#13554) · 52ce14d3
  youkaichao authored Feb 19, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  52ce14d3
- [CI/Build] force writing version file (#13544) · 81dabf24
  Daniele authored Feb 19, 2025
```
Signed-off-by: Daniele Trifirò <dtrifiro@redhat.com>
```
  81dabf24
- [Feature] Pluggable platform-specific scheduler (#13161) · 42333026
  Yannick Schnider authored Feb 19, 2025
```
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
```
  42333026
- [V1][Core] Generic mechanism for handling engine utility (#13060) · caf7ff44
  Nick Hill authored Feb 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  caf7ff44
- [Model][Speculative Decoding] DeepSeek MTP spec decode (#12755) · f525c0be
  Lucia Fang authored Feb 19, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  f525c0be
- [Bugfix] Fix Positive Feature Layers in Llava Models (#13514) · 983a40a8
  Alex Brooks authored Feb 19, 2025
```
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
```
  983a40a8
- use device param in load_model method (#13037) · fdc5df6f
  Zhe Zhang authored Feb 19, 2025
  
  fdc5df6f
- [perf-benchmark] Fix ECR path for premerge benchmark (#13512) · 3b05cd45
  Kevin H. Luu authored Feb 18, 2025
```
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
```
  3b05cd45
- [1/n][CI] Load models in CI from S3 instead of HF (#13205) · d5d214ac
  Kevin H. Luu authored Feb 18, 2025
```
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
```
  d5d214ac
- [Doc] Add clarification note regarding paligemma (#13511) · fd84857f
  Roger Wang authored Feb 18, 2025
  
  fd84857f
- [ROCm][MoE configs] mi325 mixtral & mi300 qwen_moe (#13503) · 8aada19d
  Divakar Verma authored Feb 19, 2025
  
  8aada19d
- [perf-benchmark] Allow premerge ECR (#13509) · 9aa95b0e
  Kevin H. Luu authored Feb 18, 2025
```
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
```
  9aa95b0e
- [Hardware][Gaudi][Feature] Support Contiguous Cache Fetch (#12139) · d0a7a276
  Yu-Zhou authored Feb 19, 2025
```
Signed-off-by: yuzhou <yuzhou@habana.ai>
Signed-off-by: zhouyu5 <yu.zhou@intel.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  d0a7a276
- [Misc] Remove dangling references to `--use-v2-block-manager` (#13492) · 00b69c2d
  Harry Mellor authored Feb 19, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  00b69c2d