Commits · 3097ce3a329bfe19ce3add0dbf5aaa46d7b99e88 · OpenDAS / vllm_cscc · GitLab

22 Apr, 2025 15 commits
- [Doc] Update ai_accelerator/hpu-gaudi.inc.md (#16956) · 3097ce3a
  Michael Yao authored Apr 22, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  3097ce3a
- [Bugfix] Fix f-string for Python 3.9-3.11 (#16962) · d6da9322
  Cyrus Leung authored Apr 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d6da9322
- Support S3 Sharded loading with RunAI Model Streamer (#16317) · 71ce4404
  omer-dayan authored Apr 22, 2025
```
Signed-off-by: Omer Dayan (SW-GPU) <omer@run.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  71ce4404
- [Performance][ROCm] Add skinny gemms for unquantized linear on ROCm (#15830) · 188b7f9b
  Charlie Fu authored Apr 21, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
```
  188b7f9b
- [V1] Remove additional_config check (#16710) · b9b47469
  wangxiyuan authored Apr 22, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  b9b47469
- [Kernel] Add expert_map support to Cutlass FP8 MOE (#16861) · 7b8a2ab7
  Varun Sundar Rabindranath authored Apr 21, 2025
```
Signed-off-by: varun sundar rabindranath <vsundarr@redhat.com>
Co-authored-by: varun sundar rabindranath <vsundarr@redhat.com>
```
  7b8a2ab7
- [Misc] Remove the chunked prefill warning for LoRA (#16925) · c9acbf11
  Jee Jee Li authored Apr 22, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c9acbf11
- [ROCm] Add aiter tkw1 kernel for Llama4 fp8 (#16727) · 5b794cae
  kliuae authored Apr 22, 2025
```
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  5b794cae
- [Bugfix]: fix issue with n>1 sampling on v1 requests overriding each other (#16863) · 0e425449
  Jeffrey Li authored Apr 21, 2025
```
Signed-off-by: Jeffrey Li <jeffrey.dot.li@gmail.com>
```
  0e425449
- [BugFix][Spec Decode] No in-place update to draft probs (#16952) · 1311913f
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1311913f
- [Doc] Remove unnecessary V1 flag (#16924) · 29f395c9
  Cyrus Leung authored Apr 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  29f395c9
- [TPU][V1] Enable Top-P (#16843) · fa3bba2a
  Nicolò Lucchesi authored Apr 22, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  fa3bba2a
- [V1] V1 FlashInfer Attention (#16684) · 986537f1
  Michael Goin authored Apr 21, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Aurick Qiao <qiao@aurick.net>
```
  986537f1
- [TPU][V1] Capture multimodal encoder during model compilation (#15051) · 21020752
  Nicolò Lucchesi authored Apr 22, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Siyuan Liu <lsiyuan@google.com>
```
  21020752
- Update Qwen1.5-MoE-W4A16-compressed-tensors.yaml (#16946) · 71eda0bb
  Michael Goin authored Apr 21, 2025
  
  71eda0bb
21 Apr, 2025 15 commits
- [TPU][V1] Implicitly adjust page size when there's SMEM OOM (#16871) · 471fe656
  Chengji Yao authored Apr 21, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  471fe656
- [V1][Spec Decode] Handle draft tokens beyond max_model_len (#16087) · 3a0fba5c
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3a0fba5c
- [Core] Speed up decode by remove synchronizing operation in sampler (#16436) · 299ebb62
  Chanh Nguyen authored Apr 21, 2025
```
Signed-off-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
```
  299ebb62
- [Doc] mention how to install in CPU editable mode (#16923) · f728ab8e
  David Xia authored Apr 21, 2025
```
Signed-off-by: David Xia <david@davidxia.com>
```
  f728ab8e
- [doc] install required python3-dev apt package (#16888) · 63e26fff
  David Xia authored Apr 21, 2025
```
Signed-off-by: David Xia <david@davidxia.com>
```
  63e26fff
- [XPU][Bugfix] minor fix for XPU (#15591) · fe3462c7
  Yan Ma authored Apr 22, 2025
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  fe3462c7
- Raise error for data-parallel with benchmark_throughput (#16737) · 3b34fd52
  Kartik Ramesh authored Apr 21, 2025
```
Signed-off-by: Kartik Ramesh <kartikx2000@gmail.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  3b34fd52
- [Bugfix] Fix GLM rotary_dim issue and support v1 (#16912) · 55d6d3fd
  Isotr0py authored Apr 21, 2025
```
Signed-off-by: isotr0py <2037008807@qq.com>
```
  55d6d3fd
- [Misc] Refactor platform to get device specific stream and event (#14411) · 7272bfae
  Shanshan Shen authored Apr 21, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  7272bfae
- [Misc] fix collect_env version parse (#15267) · d9ac9e3d
  wangxiyuan authored Apr 21, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  d9ac9e3d
- Restore buffers when wake up from level 2 sleep (#16564) (#16889) · d41faaf9
  Han Zhang authored Apr 21, 2025
```
Signed-off-by: Han <zh950713@gmail.com>
```
  d41faaf9
- [Doc] Split dummy_processor_inputs() in Multimodal Docs (#16915) · b34f3343
  Alex Brooks authored Apr 21, 2025
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  b34f3343
- [Bugfix] Fix distributed bug in Qwen2.5-VL & Qwen2.5-Omni (#16907) · 26c04065
  Yang Fan authored Apr 21, 2025
  
  26c04065
- [CI/CD][V1] Add spec decode tests to CI (#16900) · 4c41278b
  Woosuk Kwon authored Apr 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4c41278b
- [Bugfix] Fix v1/spec_decode/test_ngram.py (#16895) · bb3605db
  qizixi authored Apr 20, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  bb3605db
20 Apr, 2025 2 commits
- [easy] Pass compile_fx only the config patches (#16845) · fe742aef
  Richard Zou authored Apr 20, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  fe742aef
- Improve configs - `CacheConfig` (#16835) · 4b07d368
  Harry Mellor authored Apr 20, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  4b07d368
19 Apr, 2025 8 commits
- Serialize tensors using int8 views (#16866) · 87aaadef
  Staszek Paśko authored Apr 19, 2025
```
Signed-off-by: Staszek Pasko <staszek@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  87aaadef
- Log how much time loading a compiled artifact takes (#16848) · 682e0b6d
  Richard Zou authored Apr 19, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  682e0b6d
- [doc] update hyperlink (#16877) · d6195a74
  Reid authored Apr 20, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  d6195a74
- [VLM] Clean up models (#16873) · 205d84aa
  Cyrus Leung authored Apr 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  205d84aa
- [Model] Qwen2.5-Omni Cleanup (#16872) · 5124f5bf
  Roger Wang authored Apr 19, 2025
  
  5124f5bf
- [Model] Refactor Phi-4-multimodal to use merged processor and support V1 (#15477) · 83f3c3bd
  Isotr0py authored Apr 19, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  83f3c3bd
- [V1][Misc] stop update prefix cache stats when logs_stats is disabled (#16460) · d9737ca1
  vie-serendipity authored Apr 19, 2025
```
Signed-off-by: vie-serendipity <2733147505@qq.com>
```
  d9737ca1
- [Misc] Benchmarks for audio models (#16505) · 9d4ca19d
  Nicolò Lucchesi authored Apr 19, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  9d4ca19d