Commits · 3e887d2e0c1fcd65efbfe02db8a824c761fe4d41 · OpenDAS / vllm_cscc

02 May, 2025 10 commits
- permute/unpermute kernel for moe optimization (#14568) · 3e887d2e
  Caleb_Du authored May 03, 2025
```
Signed-off-by: Caleb_Du <Caleb_Du@zju.edu.cn>
```
  3e887d2e
- [Misc] Clean up input processing (#17582) · cb234955
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cb234955
- [Security] Fix image hash collision (#17378) · 99404f53
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  99404f53
- Automatically tell users that dict args must be valid JSON in CLI (#17577) · 785d75a0
  Harry Mellor authored May 02, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  785d75a0
- [Misc] Rename assets for testing (#17575) · d7543862
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d7543862
- [BugFix] Fix Memory Leak (#17567) · c777df79
  Robert Shaw authored May 02, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  c777df79
- [Core] [Bugfix] Add Input Embeddings (#15428) · cc2a77d7
  Andrew Sansom authored May 02, 2025
```
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: 临景 <linjing.yx@alibaba-inc.com>
Co-authored-by: Bryce1010 <bryceyx@gmail.com>
Co-authored-by: Nan2018 <nan@protopia.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cc2a77d7
- Add `pt_load_map_location` to allow loading to cuda (#16869) · 109e15a3
  Jerry Zhang authored May 01, 2025
```
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
```
  109e15a3
- [Misc] Continue refactoring model tests (#17573) · f89d0e11
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f89d0e11
- [CI] Actually run tests/kv_transfer/test_disagg.py in CI (#17555) · 292fc59d
  Michael Goin authored May 01, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  292fc59d
01 May, 2025 11 commits
- [Model] Refactor Ovis2 to support original tokenizer (#17537) · 88c83041
  Isotr0py authored May 02, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  88c83041
- [torch.compile] Add torch inductor pass for fusing silu_and_mul with... · 460a2b11
  Sage Moore authored May 01, 2025
```
[torch.compile] Add torch inductor pass for fusing silu_and_mul with subsequent scaled_fp8_quant operations (#10867)
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  460a2b11
- [Feature][Frontend]: Deprecate --enable-reasoning (#17452) · 98060b00
  Chauncey authored May 01, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  98060b00
- Fix more broken speculative decode tests (#17450) · b74d888c
  Huy Do authored May 01, 2025
```
Signed-off-by: Huy Do <huydhn@gmail.com>
```
  b74d888c
- [Misc] Clean up test docstrings and names (#17521) · 48e925fa
  Cyrus Leung authored May 01, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  48e925fa
- [Core] Enable IPv6 with vllm.utils.make_zmq_socket() (#16506) · fbefc8a7
  Russell Bryant authored May 01, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  fbefc8a7
- [BugFix] fix speculative decoding memory leak when speculation is disabled (#15506) · 13cf6b62
  Noah Yoshida authored Apr 30, 2025
```
Signed-off-by: Noah Yoshida <noahcy117@gmail.com>
```
  13cf6b62
- [CI/Build] Reorganize models tests (#17459) · afb4429b
  Cyrus Leung authored May 01, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  afb4429b
- [CI][Bugfix] Fix failing V1 Test due to missing 'cache_salt' arg (#17500) · aa4502e7
  Michael Goin authored Apr 30, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  aa4502e7
- [CI][TPU] Skip structured outputs+spec decode tests on TPU (#17510) · 17b4d85f
  Michael Goin authored Apr 30, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  17b4d85f
- [CI][TPU] Skip Multimodal test (#17488) · dbc18e78
  Siyuan Liu authored Apr 30, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  dbc18e78
30 Apr, 2025 10 commits

[v1][Spec Decode] Make sliding window compatible with eagle prefix caching (#17398) · 81ecf425
Chen Zhang authored May 01, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
81ecf425
[V1] Allow turning off pickle fallback in vllm.v1.serial_utils (#17427) · 947f2f53
Russell Bryant authored Apr 30, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
947f2f53

[V1][Metrics] add support for kv event publishing (#16750) · 0be6d05b

Alec authored Apr 30, 2025


Signed-off-by: alec-flowers <aflowers@nvidia.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>

0be6d05b

[Core] Prevent side-channel attacks via cache salting (#17045) · 77073c77
Marko Rosenmueller authored Apr 30, 2025
```
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
```
77073c77
[TPU][V1][CI] Update regression test baseline for v6 CI (#17064) · a7d5b016
Nicolò Lucchesi authored Apr 30, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
a7d5b016

[MODEL ADDITION] Ovis2 Model Addition (#15826) · 54072f31

Marco authored Apr 30, 2025


Signed-off-by: Marco <121761685+mlinmg@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

54072f31

Fix some speculative decode tests with tl.dot (#17371) · 88fcf00d
Huy Do authored Apr 29, 2025
```
Signed-off-by: Huy Do <huydhn@gmail.com>
```
88fcf00d
Improve configs - `ModelConfig` (#17130) · 13698db6
Harry Mellor authored Apr 30, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
13698db6

Truncation control for embedding models (#14776) · 1c2bc7ea

Gabriel Marinho authored Apr 29, 2025


Signed-off-by: Gabriel Marinho <gmarinho@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>

1c2bc7ea

[V1][Feature] Enable Speculative Decoding with Structured Outputs (#14702) · 34120f5a
Benjamin Chislett authored Apr 29, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
```
34120f5a

29 Apr, 2025 9 commits
- Remove Bamba 9B from CI (#17407) · 7489ec0b
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  7489ec0b
- Remove Falcon3 2x7B from CI (#17404) · 0350809f
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0350809f
- Simplify (and fix) passing of guided decoding backend options (#17008) · a6977dbd
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a6977dbd
- [Bugfix] add qwen3 reasoning-parser fix content is None when disable … (#17369) · a39203f9
  mofanke authored Apr 30, 2025
```
Signed-off-by: mofanke <mofanke@gmail.com>
```
  a39203f9
- Improve literal dataclass field conversion to argparse argument (#17391) · 2ef5d106
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  2ef5d106
- [Frontend] Support `chat_template_kwargs` in `LLM.chat` (#17356) · 88ad9ec6
  Cyrus Leung authored Apr 29, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  88ad9ec6
- [Bugfix] Clean up MiniMax-VL and fix processing (#17354) · 00ee37ef
  Cyrus Leung authored Apr 29, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  00ee37ef
- [Doc] Fix QWen3MOE info (#17381) · 890f104c
  Jee Jee Li authored Apr 29, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  890f104c
- [Misc]Use a platform independent interface to obtain the device attributes (#17100) · bdb2cdda
  ponix-j authored Apr 29, 2025
  
  bdb2cdda