Commits · 46e678bcff350518bee8b92c826f21c5cdd0d08f · OpenDAS / vllm_cscc · GitLab

24 Apr, 2025 2 commits
- [Minor] Use larger batch sizes for A100/B100/B200/MI300x (#17073) · 46e678bc
  Woosuk Kwon authored Apr 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  46e678bc
- [Quantization]add prefix for commandA quantized model (#17017) · 6b2427f9
  Chen Xia authored Apr 23, 2025
  
  6b2427f9
23 Apr, 2025 21 commits
- [V1][Spec Decode] Always use argmax for sampling draft tokens (#16899) · 41fb013d
  Woosuk Kwon authored Apr 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  41fb013d
- [BugFix][V1] Fix int32 token index overflow when preparing input ids (#16806) · 32d4b669
  Yong Hoon Shin authored Apr 23, 2025
  
  32d4b669
- [Frontend] Support guidance:no-additional-properties for compatibility with xgrammar (#15949) · 3cde34a4
  Travis Johnson authored Apr 23, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  3cde34a4
- Use `@property` and private field for `data_parallel_rank_local` (#17053) · bdb36603
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  bdb36603
- `CacheConfig.block_size` should always be `int` when used (#17052) · f3a21e9c
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  f3a21e9c
- Improve Transformers backend model loading QoL (#17039) · 8e630d68
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8e630d68
- Ensure that `pid` passed to `kill_process_tree` is `int` for `mypy` (#17051) · 53c0fa1e
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  53c0fa1e
- Mistral-format support for compressed-tensors (#16803) · aa72d9a4
  Michael Goin authored Apr 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  aa72d9a4
- [Bugfix] Fix AssertionError: skip_special_tokens=False is not supported for... · 8c87a9ad
  Chauncey authored Apr 23, 2025
```
[Bugfix] Fix AssertionError: skip_special_tokens=False is not supported for Mistral tokenizers (#16964)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  8c87a9ad
- [Misc] Improve readability of get_open_port function. (#17024) · ec69124e
  huafeng authored Apr 23, 2025
```
Signed-off-by: gitover22 <qidizou88@gmail.com>
```
  ec69124e
- [BugFix] llama4 fa3 fix - RuntimeError: scheduler_metadata must have shape (metadata_size) (#16998) · d0da99fb
  Lucas Wilkinson authored Apr 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  d0da99fb
- [V1] Avoid socket errors during shutdown when requests are in in-flight (#16807) · b2f195c4
  Nick Hill authored Apr 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  b2f195c4
- [Bugfix] Triton FA function takes no keyword arguments (#16902) · 047797ef
  vllmellm authored Apr 23, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  047797ef
- [INTEL-HPU][v0] Port delayed sampling to upstream (#16949) · 56a73526
  Chendi.Xue authored Apr 22, 2025
```
Signed-off-by: Michal Adamczyk <michal.adamczyk@intel.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
```
  56a73526
- [misc] tune some env vars for GB200 (#16992) · e1cf90e0
  youkaichao authored Apr 23, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  e1cf90e0
- Revert "[Misc] Add S3 environment variables for better support of MinIO." (#17021) · 6bc1e30e
  Chauncey authored Apr 23, 2025
  
  6bc1e30e
- [BugFix] Revert ROCm Custom Paged Attention Env Flag Check (#17022) · 7e081ba7
  vllmellm authored Apr 23, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  7e081ba7
- [V1][DP] More robust DP/EP dummy request coordination (#16277) · 1e013fa3
  Nick Hill authored Apr 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  1e013fa3
- [Kernel][ROCM] Upstream prefix prefill speed up for vLLM V1 (#13305) · bc7c4d20
  Aleksandr Malyshev authored Apr 22, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: root <root@banff-cyxtera-s65-4.amd.com>
Signed-off-by: maleksan85 <maleksan@amd.com>
Signed-off-by: <>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: root <root@banff-cyxtera-s65-4.amd.com>
```
  bc7c4d20
- [Bugfix] validate urls object for multimodal content parts (#16990) · 36fe7876
  Guillaume Calmettes authored Apr 23, 2025
```
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  36fe7876
- [Core][V1][TPU] Enable structured decoding on TPU V1 (#16499) · 83d93371
  Chenyaaang authored Apr 22, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  83d93371
22 Apr, 2025 17 commits
- [BugFix] Remove default multiproc executor `collective_rpc` timeout (#17000) · 5175b884
  Nick Hill authored Apr 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  5175b884
- Add assertion for no objects while hashing hf_config (#16930) · 7f58fb97
  Richard Zou authored Apr 22, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  7f58fb97
- [FEAT][ROCm]: Support AITER MLA (#15893) · 30bc3e0f
  vllmellm authored Apr 23, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
```
  30bc3e0f
- [frontend] enhance tool_calls type check (#16882) · f3441071
  Reid authored Apr 22, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  f3441071
- [Misc] Add S3 environment variables for better support of MinIO. (#16977) · 68d4c332
  Chauncey authored Apr 22, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  68d4c332
- [BugFix] Pass in correct VLLM config in FlashInfer backend (#13207) (#16973) · f961d7f6
  Zhengyuan Su (苏政渊) authored Apr 22, 2025
```
Signed-off-by: 苏政渊 <suzhengyuan@moonshot.cn>
Co-authored-by: 苏政渊 <suzhengyuan@moonshot.cn>
```
  f961d7f6
- Improve configs - `SpeculativeConfig` (#16971) · d0591104
  Harry Mellor authored Apr 22, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d0591104
- [Bugfix] Fix distributed bug again in Qwen2.5-VL & Qwen2.5-Omni (#16974) · 571e8dd6
  Yang Fan authored Apr 22, 2025
```
Signed-off-by: fyabc <suyang.fy@alibaba-inc.com>
```
  571e8dd6
- [FEAT][ROCm] Integrate Paged Attention Kernel from AITER (#15001) · 0e237f00
  vllmellm authored Apr 22, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  0e237f00
- [Doc] Improve documentation for multimodal CLI args (#16960) · 8f7bace7
  Cyrus Leung authored Apr 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f7bace7
- [BugFix] Fix incremental detokenization perf issue (#16963) · e4d61442
  Nick Hill authored Apr 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  e4d61442
- [Kernel] Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS (#6036) · 8d32dc60
  Lei Wang authored Apr 22, 2025
```
Signed-off-by: xinyuxiao <xinyuxiao2024@gmail.com>
Co-authored-by: xinyuxiao <xinyuxiao2024@gmail.com>
```
  8d32dc60
- [V1] Remove pre-allocation for KV cache (#16941) · c4ab9f3e
  Woosuk Kwon authored Apr 22, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  c4ab9f3e
- [Model] Use autoweightloader for mamba (#16950) · 2689d5c0
  Flora Feng authored Apr 22, 2025
```
Signed-off-by: sfeng33 <4florafeng@gmail.com>
```
  2689d5c0
- [Bugfix] Fix the issue where llm.generate cannot be called repeatedly after... · acba33a0
  Chauncey authored Apr 22, 2025
```
[Bugfix] Fix the issue where llm.generate cannot be called repeatedly after setting GuidedDecodingParams (#16767)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
```
  acba33a0
- [Perf] Optimize `_update_states` for GPU model runner (#16910) · a114bf20
  SnowCharm authored Apr 22, 2025
```
Signed-off-by: snowcharm <snowcharmqq@gmail.com>
```
  a114bf20
- [Bugfix] Fix f-string for Python 3.9-3.11 (#16962) · d6da9322
  Cyrus Leung authored Apr 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d6da9322