Commits · d55244df31969e7df435603b5d7014939e60881b · OpenDAS / vllm_cscc

"docs/vscode:/vscode.git/clone" did not exist on "dc3529dbf65786fe25cce8144c76260266061c9d"

09 Apr, 2025 7 commits
- [Misc] Fix test_sharded_state_loader.py(#16004) (#16005) · 24f6b9a7
  Accelerator1996 authored Apr 09, 2025
```
Signed-off-by: lvfei.lv <lvfei.lv@alibaba-inc.com>
```
  24f6b9a7
- [BugFix] Fix fusion test and add them to CI (#16287) · 9cdde472
  Luka Govedič authored Apr 09, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  9cdde472
- [TPU] Update PyTorch/XLA (#16288) · b1eb4ca1
  Chengji Yao authored Apr 08, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  b1eb4ca1
- [CI][Bugfix] Fix bad tolerance for test_batch_base64_embedding (#16221) · 87b4ac56
  Michael Goin authored Apr 08, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  87b4ac56
- [Feature] Estimate max-model-len use available KV cache memory (#16168) · 4716377f
  rongfu.leng authored Apr 09, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  4716377f
- [Model] Add smolvlm support (#16017) · 102bf967
  Chauncey authored Apr 09, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  102bf967
- [CI/Build] Fix CI LoRA failure (#16270) · 86c3369e
  Jee Jee Li authored Apr 09, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  86c3369e
08 Apr, 2025 7 commits
- [Bugfix] Proper input validation for multi-modal encoder-decoder models (#16156) · 4ebc0b96
  Cyrus Leung authored Apr 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  4ebc0b96
- [New Model]: jinaai/jina-embeddings-v3 (#16120) · 1f5d13ab
  wang.yuqi authored Apr 08, 2025
  
  1f5d13ab
- [Bugfix] Remove triton do_bench fast_flush arg (#16256) · e11880de
  Kebe authored Apr 08, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
  e11880de
- [V1] Add `disable_chunked_mm_input` arg to disable partial mm input prefill (#15837) · 8e5314a4
  Michael Goin authored Apr 08, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  8e5314a4
- [Bugfix] Fix and reorganize broken GGUF tests and bump gguf version (#16194) · f6b32efb
  Isotr0py authored Apr 08, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  f6b32efb
- [Bugfix] Do not skip "empty" parts of chats that are parsable (#16219) · b99733d0
  Michael Goin authored Apr 07, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b99733d0
- [V1] Scatter and gather placeholders in the model runner (#16076) · f2ebb6f5
  Roger Wang authored Apr 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
```
  f2ebb6f5
07 Apr, 2025 10 commits

Torchao (#14231) · 652907b3
Driss Guessous authored Apr 07, 2025
```
Signed-off-by: drisspg <drisspguessous@gmail.com>
```
652907b3
[Bugfix][V0] XGrammar structured output supports Enum (#15878) · 24f1c01e
leon-seidel authored Apr 08, 2025
```
Signed-off-by: Leon Seidel <leon.seidel@fau.de>
```
24f1c01e
[V1][BugFix] Exit properly if engine core fails during startup (#16137) · 7f6d47c1
Nick Hill authored Apr 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
7f6d47c1

[Misc] Human-readable `max-model-len` cli arg (#16181) · 090c856d

Nicolò Lucchesi authored Apr 07, 2025


Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

090c856d

[V1] Revert the default `max_num_seqs` to V0 values for most hardware (#16158) · 66d433b9
Cyrus Leung authored Apr 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
66d433b9
[Bugfix] Re-enable support for `ChatGLMForConditionalGeneration` (#16187) · 027b204f
Cyrus Leung authored Apr 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
027b204f

Upstream Llama4 Support to Main (#16113) · 55dcce91

Lu Fang authored Apr 07, 2025


Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

55dcce91

[Model] Add Qwen3 and Qwen3MoE (#15289) · 7699258e

YamPengLi authored Apr 07, 2025


Signed-off-by: YamPengLi <yampayne.lyp@alibaba-inc.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

7699258e

[CI] Set max transformers version for Ultravox model test (#16149) · bb8dab82
Roger Wang authored Apr 06, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
bb8dab82
[Bugfix] Make dummy encoder prompt padding alternative and add missing warnings (#16129) · fc0f8776
Isotr0py authored Apr 07, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
fc0f8776

05 Apr, 2025 1 commit
- [Misc] Auto detect bitsandbytes pre-quantized models (#16027) · 4285e423
  Tristan Leclercq authored Apr 05, 2025
```
Signed-off-by: Tristan Leclercq <tristanleclercq@gmail.com>
```
  4285e423
04 Apr, 2025 3 commits
- Revert "[V1] Scatter and gather placeholders in the model runner" (#16075) · af51d80f
  Roger Wang authored Apr 04, 2025
  
  af51d80f
- [V1] Scatter and gather placeholders in the model runner (#15712) · f5722a50
  Cyrus Leung authored Apr 05, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  f5722a50
- [V1][Spec Decode] Avoid logging useless nan metrics (#16023) · a35a8a83
  Mark McLoughlin authored Apr 04, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  a35a8a83
03 Apr, 2025 7 commits
- [Bugfix] Fix function names in test_block_fp8.py (#16033) · dcc56d62
  bnellnm authored Apr 03, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  dcc56d62
- [TPU] Switch Test to Non-Sliding Window (#15981) · f15e70d9
  Robert Shaw authored Apr 03, 2025
```
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
```
  f15e70d9
- [TPU] Support sliding window and logit soft capping in the paged attention kernel for TPU. (#15732) · b6be6f8d
  iefgnoix authored Apr 03, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  b6be6f8d
- [Minor] Fused experts refactor (#15914) · 15ba07ef
  bnellnm authored Apr 03, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  15ba07ef
- [Neuron][kernel] Fuse kv cache into a single tensor (#15911) · d2b58ca2
  Liangfu Chen authored Apr 03, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
  d2b58ca2
- [ROCM][KERNEL] Paged attention for V1 (#15720) · e73ff24e
  Aleksandr Malyshev authored Apr 02, 2025
```
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: root <root@banff-cyxtera-s65-4.amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: root <root@banff-cyxtera-s65-4.amd.com>
```
  e73ff24e
- [V1][TPU] TPU-optimized top-p implementation (avoids scattering). (#15736) · 1b84eff0
  Hyesoo Yang authored Apr 02, 2025
```
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: root <root@t1v-n-822696b7-w-0.us-central2-b.c.tpu-prod-env-large-adhoc.internal>
```
  1b84eff0
02 Apr, 2025 5 commits

[Frontend] Implement Tool Calling with `tool_choice='required'` (#13483) · cefb9e5a

Matthias Matt authored Apr 02, 2025


Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Signed-off-by: Matt, Matthias <matthias.matt@tuwien.ac.at>
Co-authored-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: mgoin <michael@neuralmagic.com>

cefb9e5a

[Metrics] Hide deprecated metrics (#15458) · 98d7367b
Mark McLoughlin authored Apr 02, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
98d7367b

[Bugfix] Fix the issue where the model name is empty string, causing no... · 594a8b90

Chauncey authored Apr 02, 2025


[Bugfix] Fix the issue where the model name is empty string, causing no response with the model name. (#15938)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>

594a8b90

[V1] Fix json_object support with xgrammar (#15488) · 14e53ed1
Russell Bryant authored Apr 02, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
14e53ed1
[core] Add tags parameter to wake_up() (#15500) · ddb94c26
Eric Tang authored Apr 02, 2025
```
Signed-off-by: Eric <erictang000@gmail.com>
```
ddb94c26