Commits · 2f54045508930a741f3638cc3c1174fdd672087f · OpenDAS / vllm_cscc · GitLab

25 Apr, 2025 6 commits
- [Bugfix][Misc] Use TritonPlaceholderModule to defensively import triton (#15099) · 2f540455
  Mengqing Cao authored Apr 25, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  2f540455
- [Misc] Clean up redundant code in uniproc_executor.py (#16762) · 5aa6efb9
  Lifu Huang authored Apr 24, 2025
```
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
```
  5aa6efb9
- Move missed `SchedulerConfig` args into scheduler config group in `EngineArgs` (#17131) · 6ca02344
  Harry Mellor authored Apr 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6ca02344
- fix float16 support for kimi-vl (#17156) · 69bff9bc
  Zaida Zhou authored Apr 25, 2025
```
Co-authored-by: zhouzaida <zhouzaida@msh.team>
```
  69bff9bc
- [FEAT] [ROCm]: AITER Fused MOE V1 Support (#16752) · eef36472
  vllmellm authored Apr 25, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  eef36472
- Use custom address for listening socket (#15988) · 0d6e187e
  jglaser authored Apr 24, 2025
```
Signed-off-by: Jens Glaser <glaserj@ornl.gov>
```
  0d6e187e
24 Apr, 2025 18 commits
- Better error message for missing mistral params.json (#17132) · 9420a1fc
  Michael Goin authored Apr 24, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  9420a1fc
- Add chat template for Llama 4 models (#16428) · 05e1fbfc
  Maximilien de Bayser authored Apr 24, 2025
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  05e1fbfc
- Add collective_rpc to llm engine (#16999) · fe921763
  Yinghai Lu authored Apr 24, 2025
```
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
```
  fe921763
- Improve configs - `LoRAConfig` + `PromptAdapterConfig` (#16980) · 0fa939e2
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0fa939e2
- [V1][Spec Decoding] Add num_drafts and num_accepted_tokens_per_position metrics (#16665) · 340d7b1b
  Mark McLoughlin authored Apr 24, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  340d7b1b
- [Frontend] Using matryoshka_dimensions control the allowed output dimensions. (#16970) · 67309a1c
  wang.yuqi authored Apr 24, 2025
  
  67309a1c
- [V1][Structured Output] Clear xgrammar compiler object when engine core shut... · b724afe3
  Shanshan Shen authored Apr 24, 2025
```
[V1][Structured Output] Clear xgrammar compiler object when engine core shut down to avoid nanobind leaked warning (#16954)
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  b724afe3
- Improve static type checking in `LoRAModelRunnerMixin` (#17104) · 21f4f1c9
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  21f4f1c9
- [Misc] Remove OLMo2 config copy (#17066) · b0c1f620
  Isotr0py authored Apr 24, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b0c1f620
- [V1][PP] Optimization: continue scheduling prefill chunks (#17080) · c0dfd975
  Rui Qiao authored Apr 24, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  c0dfd975
- Simplify `TokenizerGroup` (#16790) · 0a05ed57
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0a05ed57
- [Chore] Remove Sampler from Model Code (#17084) · b411418f
  Woosuk Kwon authored Apr 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b411418f
- [Bugfix][Core] add seq_id_to_seq_group clearing to avoid memory leak when s… (#16472) · 6167c0e5
  张宇 authored Apr 24, 2025
```
Signed-off-by: 开哲 <kaizhe.zy@alibaba-inc.com>
Co-authored-by: 开哲 <kaizhe.zy@alibaba-inc.com>
```
  6167c0e5
- Addendum Fix to support FIPS enabled machines with MD5 hashing (#17043) · ed2e4646
  Areeb Syed authored Apr 24, 2025
```
Signed-off-by: sydarb <areebsyed237@gmail.com>
```
  ed2e4646
- More informative error when using Transformers backend (#16988) · 2c8ed8ee
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  2c8ed8ee
- [Bugfix] Enable V1 usage stats (#16986) · ed50f466
  Michael Goin authored Apr 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  ed50f466
- [Minor] Use larger batch sizes for A100/B100/B200/MI300x (#17073) · 46e678bc
  Woosuk Kwon authored Apr 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  46e678bc
- [Quantization]add prefix for commandA quantized model (#17017) · 6b2427f9
  Chen Xia authored Apr 23, 2025
  
  6b2427f9
23 Apr, 2025 16 commits
- [V1][Spec Decode] Always use argmax for sampling draft tokens (#16899) · 41fb013d
  Woosuk Kwon authored Apr 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  41fb013d
- [BugFix][V1] Fix int32 token index overflow when preparing input ids (#16806) · 32d4b669
  Yong Hoon Shin authored Apr 23, 2025
  
  32d4b669
- [Frontend] Support guidance:no-additional-properties for compatibility with xgrammar (#15949) · 3cde34a4
  Travis Johnson authored Apr 23, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  3cde34a4
- Use `@property` and private field for `data_parallel_rank_local` (#17053) · bdb36603
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  bdb36603
- `CacheConfig.block_size` should always be `int` when used (#17052) · f3a21e9c
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  f3a21e9c
- Improve Transformers backend model loading QoL (#17039) · 8e630d68
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8e630d68
- Ensure that `pid` passed to `kill_process_tree` is `int` for `mypy` (#17051) · 53c0fa1e
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  53c0fa1e
- Mistral-format support for compressed-tensors (#16803) · aa72d9a4
  Michael Goin authored Apr 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  aa72d9a4
- [Bugfix] Fix AssertionError: skip_special_tokens=False is not supported for... · 8c87a9ad
  Chauncey authored Apr 23, 2025
```
[Bugfix] Fix AssertionError: skip_special_tokens=False is not supported for Mistral tokenizers (#16964)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  8c87a9ad
- [Misc] Improve readability of get_open_port function. (#17024) · ec69124e
  huafeng authored Apr 23, 2025
```
Signed-off-by: gitover22 <qidizou88@gmail.com>
```
  ec69124e
- [BugFix] llama4 fa3 fix - RuntimeError: scheduler_metadata must have shape (metadata_size) (#16998) · d0da99fb
  Lucas Wilkinson authored Apr 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  d0da99fb
- [V1] Avoid socket errors during shutdown when requests are in in-flight (#16807) · b2f195c4
  Nick Hill authored Apr 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  b2f195c4
- [Bugfix] Triton FA function takes no keyword arguments (#16902) · 047797ef
  vllmellm authored Apr 23, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  047797ef
- [INTEL-HPU][v0] Port delayed sampling to upstream (#16949) · 56a73526
  Chendi.Xue authored Apr 22, 2025
```
Signed-off-by: Michal Adamczyk <michal.adamczyk@intel.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
```
  56a73526
- [misc] tune some env vars for GB200 (#16992) · e1cf90e0
  youkaichao authored Apr 23, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  e1cf90e0
- Revert "[Misc] Add S3 environment variables for better support of MinIO." (#17021) · 6bc1e30e
  Chauncey authored Apr 23, 2025
  
  6bc1e30e