Commits · ad971af8c7dc2c006b58853207b36942569a10ee · OpenDAS / vllm_cscc

08 Apr, 2025 6 commits
- [Bugfix] fix use-ep bug to enable ep by dp/tp size > 1 (#16161) · ad971af8
  zxfan-cpu authored Apr 08, 2025
  
  ad971af8
- [V1] Scatter and gather placeholders in the model runner (#16076) · f2ebb6f5
  Roger Wang authored Apr 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
```
  f2ebb6f5
- Update BASE_IMAGE to 2.22 release of Neuron (#16218) · 1d012112
  Satyajith Chilappagari authored Apr 07, 2025
  
  1d012112
- [Misc] Update compressed-tensors to version 0.9.3 (#16196) · f94ab12f
  Miles Williams authored Apr 08, 2025
```
Signed-off-by: Miles Williams <42222518+mlsw@users.noreply.github.com>
```
  f94ab12f
- [core] do not send error across process (#16174) · a865bc1c
  youkaichao authored Apr 08, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  a865bc1c
- [ROCm][Bugfix][FP8] Make fp8 quant respect fused modules mapping (#16031) · 21802c4b
  Michael Goin authored Apr 07, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  21802c4b
07 Apr, 2025 23 commits

Torchao (#14231) · 652907b3
Driss Guessous authored Apr 07, 2025
```
Signed-off-by: drisspg <drisspguessous@gmail.com>
```
652907b3
[Bugfix][V0] XGrammar structured output supports Enum (#15878) · 24f1c01e
leon-seidel authored Apr 08, 2025
```
Signed-off-by: Leon Seidel <leon.seidel@fau.de>
```
24f1c01e

[Misc] add description attribute in CLI (#15921) · fad6e253

Reid authored Apr 08, 2025


Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>

fad6e253

[V1][BugFix] Exit properly if engine core fails during startup (#16137) · 7f6d47c1
Nick Hill authored Apr 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
7f6d47c1
[Bugfix] Fix guidance backend for Qwen models (#16210) · 3147586e
Benjamin Chislett authored Apr 07, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
3147586e
[Misc] Move Llama 4 projector call into encoder execution (#16201) · ed636d99
Roger Wang authored Apr 07, 2025

ed636d99

[Misc] Human-readable `max-model-len` cli arg (#16181) · 090c856d

Nicolò Lucchesi authored Apr 07, 2025


Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

090c856d

Print the warning only once (#16193) · ad434d4c
Gregory Shtrasberg authored Apr 07, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
ad434d4c
[V1] Revert the default `max_num_seqs` to V0 values for most hardware (#16158) · 66d433b9
Cyrus Leung authored Apr 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
66d433b9
[Bugfix] Re-enable support for `ChatGLMForConditionalGeneration` (#16187) · 027b204f
Cyrus Leung authored Apr 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
027b204f

Upstream Llama4 Support to Main (#16113) · 55dcce91

Lu Fang authored Apr 07, 2025


Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

55dcce91

[Doc]Update image to latest version (#16186) · 8017c8db
Robin authored Apr 07, 2025
```
Signed-off-by: WangErXiao <863579016@qq.com>
```
8017c8db
[Misc] improve example mlpspeculator and llm_engine_example (#16175) · dc3529db
Reid authored Apr 07, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
dc3529db

[Model] Add Qwen3 and Qwen3MoE (#15289) · 7699258e

YamPengLi authored Apr 07, 2025


Signed-off-by: YamPengLi <yampayne.lyp@alibaba-inc.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

7699258e

[V1][Structured Output] Add `supports_structured_output()` method to Platform (#16148) · e9ba99f2
Shanshan Shen authored Apr 07, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
e9ba99f2
[VLM] Florence-2 supports online serving (#16164) · 7c803687
Isotr0py authored Apr 07, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
7c803687
doc: fix some typos in doc (#16154) · 95d63f38
yihong authored Apr 07, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
95d63f38
[CI] Set max transformers version for Ultravox model test (#16149) · bb8dab82
Roger Wang authored Apr 06, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
bb8dab82
[Bugfix] Make dummy encoder prompt padding alternative and add missing warnings (#16129) · fc0f8776
Isotr0py authored Apr 07, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
fc0f8776
[Misc] Update Mistral-3.1 example (#16147) · 0a573867
Cyrus Leung authored Apr 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
0a573867
[V1][Minor] Minor simplification for get_computed_blocks (#16139) · 3749e287
Woosuk Kwon authored Apr 06, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
3749e287

[Metrics] Add bucket for `request_latency`, `time_to_first_token` and... · 86fc2321

Kay Yan authored Apr 07, 2025


[Metrics] Add bucket for `request_latency`, `time_to_first_token` and `time_per_output_token` (#15202)
Signed-off-by: Kay Yan <kay.yan@daocloud.io>

86fc2321

Fix requires-python (#16132) · 2549c0df
Martin Hoyer authored Apr 07, 2025

2549c0df

06 Apr, 2025 11 commits
- [V1][Minor] Optimize get_cached_block (#16135) · b10e5198
  Woosuk Kwon authored Apr 06, 2025
  
  b10e5198
- [TPU] Update PyTorch/XLA (#16130) · 9bde5ba1
  Chengji Yao authored Apr 06, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  9bde5ba1
- [Misc] update requires-python in pyproject.toml (#16116) · 72c8f1ad
  Reid authored Apr 06, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  72c8f1ad
- [Bugfix] add hf_token to EngineArgs (#16093) · da224daa
  paolovic authored Apr 06, 2025
```
Signed-off-by: paolovic <paul-philipp.luley@uzh.ch>
Co-authored-by: paolovic <paul-philipp.luley@uzh.ch>
```
  da224daa
- [Bugfix] LoRA : Fix the order in which the kernels process LoRAs (#16040) · 3a100b92
  Varun Sundar Rabindranath authored Apr 06, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  3a100b92
- [Model] use AutoWeightsLoader for stablelm,starcoder2,zamba2 (#16103) · 242a637a
  rongfu.leng authored Apr 06, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  242a637a
- [Misc] Improve model redirect to accept json dictionary (#16119) · c2a96715
  Isotr0py authored Apr 06, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c2a96715
- [Doc][Bugfix] Add missing EOF in k8s deploy doc (#16025) · d5ae4f7f
  Paul Schweigert authored Apr 06, 2025
  
  d5ae4f7f
- [Misc] refactor example eagle (#16100) · b6c502a1
  Reid authored Apr 06, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  b6c502a1
- [CI][V1] Fix passing `tokenizer` as kwarg to `validate_guidance_grammar` (#16117) · 9ca710e5
  Roger Wang authored Apr 06, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  9ca710e5
- [Frontend] Fix typo in tool chat templates for llama3.2 and toolace (#14501) · eb07c8cb
  Ben Jackson authored Apr 06, 2025
```
Signed-off-by: Ben Jackson <ben@ben.com>
```
  eb07c8cb