Commits · 1b15df2546e97c409668da92954d8802c48d13af · OpenDAS / vllm_cscc

19 May, 2025 2 commits

Feature/vllm/input embedding completion api (#17590) · 221cfc2f

Nan Qin authored May 18, 2025


Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Nan2018 <nan@protopia.ai>
Co-authored-by: 临景 <linjing.yx@alibaba-inc.com>
Co-authored-by: Bryce1010 <bryceyx@gmail.com>
Co-authored-by: Andrew Sansom <andrew@protopia.ai>
Co-authored-by: Andrew Sansom <qthequartermasterman@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

221cfc2f

[Spec Decode][V0] Fix spec decode correctness test in V0 eagle/medusa (#18175) · 9da1095d
wwl2755 authored May 18, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
9da1095d

17 May, 2025 1 commit
- Support sequence parallelism combined with pipeline parallelism (#18243) · 9ab2c02f
  cascade authored May 17, 2025
```
Signed-off-by: cascade812 <cascade812@outlook.com>
```
  9ab2c02f
16 May, 2025 8 commits
- [Bugfix] fix `an illegal memory access was encountered` of marlin kernel + act_order (#18245) · e73b7dfd
  Jinzhen Lin authored May 17, 2025
  
  e73b7dfd
- [Sampler] Adapt to FlashInfer 0.2.3 sampler API (#15777) · 7fdfa015
  Bowen Wang authored May 16, 2025
```
Signed-off-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  7fdfa015
- [Misc] Consolidate Audio tests into multimodal common generation tests (#18214) · 390ec889
  Isotr0py authored May 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  390ec889
- [Misc] Add Ray Prometheus logger to V1 (#17925) · 54181767
  Seiji Eicher authored May 16, 2025
```
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
```
  54181767
- [Feature] Support Pipeline Parallism in torchrun SPMD offline inference for V1 (#17827) · 3d2779c2
  Lucia Fang authored May 15, 2025
```
Signed-off-by: Lucia Fang <fanglu@fb.com>
```
  3d2779c2
- Throw better error for when running into k8s service discovery issue (#18209) · 6b31c84a
  Will Eaton authored May 16, 2025
```
Signed-off-by: Will Eaton <weaton@redhat.com>
```
  6b31c84a
- Allow users to pass arbitrary JSON keys from CLI (#18208) · b18201fe
  Harry Mellor authored May 16, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  b18201fe
- [Bugfix] fix rotary embedding test for _get_padded_tensor_shape (#18229) · 4e1c6a02
  Lucas Wilkinson authored May 15, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  4e1c6a02
15 May, 2025 11 commits
- [Bugfix] Fix test_eagle test (#18223) · 8795eb99
  Lucia Fang authored May 15, 2025
```
Signed-off-by: Lucia Fang <fanglu@fb.com>
```
  8795eb99
- Adding "Basic Models Test" and "Multi-Modal Models Test (Extended) 3" in AMD Pipeline (#18106) · 566ec04c
  Alexei-V-Ivanov-AMD authored May 15, 2025
```
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  566ec04c
- add tools into TokenizeChatRequest (#18187) · 451da4bc
  hustxiayang authored May 15, 2025
```
Signed-off-by: yangxia <yangxiast@gmail.com>
```
  451da4bc
- fix: typos (#18151) · a9944aab
  omahs authored May 15, 2025
```
Signed-off-by: omahs <73983677+omahs@users.noreply.github.com>
```
  a9944aab
- [V1] Update zmq socket creation in nixl connector (#18148) · a8f5aec2
  Russell Bryant authored May 15, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  a8f5aec2
- [CI] don't skip fixed `test_kv_cache_events()` (#18183) · de71fec8
  David Xia authored May 15, 2025
```
Signed-off-by: David Xia <david@davidxia.com>
```
  de71fec8
- [UT] Add ut for none hash (#17892) · 420caf75
  Ning Xie authored May 15, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  420caf75
- Support custom implementations of VideoLoader backends. (#18091) · 4f07a640
  Chenheli Hua authored May 14, 2025
  
  4f07a640
- [Bugfix] Fix fp8 tests for triton_unified_attention for Triton 3.3 (#18013) · e6b8e65d
  Thomas Parnell authored May 15, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  e6b8e65d
- [V1][Metrics] Remove unused code (#18158) · 65334ef3
  Mark McLoughlin authored May 15, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  65334ef3
- [v1] Support multiple KV cache groups in GPU model runner (#17945) · e60f550b
  Chen Zhang authored May 15, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  e60f550b
14 May, 2025 18 commits
- [V1] Support multiple kv connectors (#17564) · 2142035b
  Michael Goin authored May 14, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  2142035b
- [CI] Fix race condition in test_kv_cache_events test (#18169) · 78aa341d
  Russell Bryant authored May 14, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  78aa341d
- Add support for loading torchao models with `AOPerModuleConfig` (#17826) · 79747367
  Jerry Zhang authored May 14, 2025
```
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
```
  79747367
- [V1] Structured Outputs + Thinking compatibility (#16577) · 2fc9075b
  Aaron Pham authored May 14, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
```
  2fc9075b
- [Kernel] Have rotary embeddings support tensors (#18046) · d93c976a
  Lucas Wilkinson authored May 14, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  d93c976a
- [CI] Disable Failing Tests (#18165) · 85686500
  Robert Shaw authored May 14, 2025
  
  85686500
- Modularize fused experts and integrate PPLX kernels (#15956) · f9c069c8
  bnellnm authored May 14, 2025
  
  f9c069c8
- [KVConnector] Keep KVTransferParams as a dict (#18033) · 59dd311c
  Nick Hill authored May 14, 2025
  
  59dd311c
- [Bugfix] Fix chat utils tests (#18139) · d066e520
  Cyrus Leung authored May 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d066e520
- [Model] GritLM supports other attention backends (#18109) · d62a076e
  Cyrus Leung authored May 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d62a076e
- [Bugfix] Fix LoRA test (#18123) · 259127f8
  Jee Jee Li authored May 14, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  259127f8
- [FEAT] [ROCm]: Add AITER CK 2 Stages MoE support (#17110) · 612c2edb
  TJian authored May 14, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  612c2edb
- [Misc] replace does not exist model (#18119) · 82e7f9bb
  rongfu.leng authored May 14, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  82e7f9bb
- [Bugfix] Fix entrypoints audio test failure (#18111) · 8f5dc414
  Cyrus Leung authored May 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f5dc414
- [New Model]: support GTE NewModel (#17986) · 63ad6222
  wang.yuqi authored May 14, 2025
  
  63ad6222
- [Fix] Move "model_config" as keyword args in chat_utils.py (#18098) · 6685890d
  lkchen authored May 13, 2025
```
Signed-off-by: Linkun <github@lkchen.net>
```
  6685890d
- [AMD][torch.compile] Enable silu+fp8_quant fusion for rocm (#18082) · 7b2f28de
  Charlie Fu authored May 14, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
```
  7b2f28de
- [FEAT] [ROCm] [V1]: Add AITER biased group topk for DeepSeekV3 (#17955) · 2d912fb6
  vllmellm authored May 14, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  2d912fb6