Commits · 986537f1c3c8288635b1f00944409489ab1cf21b · OpenDAS / vllm_cscc · GitLab

22 Apr, 2025 3 commits
- [V1] V1 FlashInfer Attention (#16684) · 986537f1
  Michael Goin authored Apr 21, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Aurick Qiao <qiao@aurick.net>
```
  986537f1
- [TPU][V1] Capture multimodal encoder during model compilation (#15051) · 21020752
  Nicolò Lucchesi authored Apr 22, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Siyuan Liu <lsiyuan@google.com>
```
  21020752
- Update Qwen1.5-MoE-W4A16-compressed-tensors.yaml (#16946) · 71eda0bb
  Michael Goin authored Apr 21, 2025
  
  71eda0bb
21 Apr, 2025 15 commits
- [TPU][V1] Implicitly adjust page size when there's SMEM OOM (#16871) · 471fe656
  Chengji Yao authored Apr 21, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  471fe656
- [V1][Spec Decode] Handle draft tokens beyond max_model_len (#16087) · 3a0fba5c
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3a0fba5c
- [Core] Speed up decode by remove synchronizing operation in sampler (#16436) · 299ebb62
  Chanh Nguyen authored Apr 21, 2025
```
Signed-off-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
```
  299ebb62
- [Doc] mention how to install in CPU editable mode (#16923) · f728ab8e
  David Xia authored Apr 21, 2025
```
Signed-off-by: David Xia <david@davidxia.com>
```
  f728ab8e
- [doc] install required python3-dev apt package (#16888) · 63e26fff
  David Xia authored Apr 21, 2025
```
Signed-off-by: David Xia <david@davidxia.com>
```
  63e26fff
- [XPU][Bugfix] minor fix for XPU (#15591) · fe3462c7
  Yan Ma authored Apr 22, 2025
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  fe3462c7
- Raise error for data-parallel with benchmark_throughput (#16737) · 3b34fd52
  Kartik Ramesh authored Apr 21, 2025
```
Signed-off-by: Kartik Ramesh <kartikx2000@gmail.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  3b34fd52
- [Bugfix] Fix GLM rotary_dim issue and support v1 (#16912) · 55d6d3fd
  Isotr0py authored Apr 21, 2025
```
Signed-off-by: isotr0py <2037008807@qq.com>
```
  55d6d3fd
- [Misc] Refactor platform to get device specific stream and event (#14411) · 7272bfae
  Shanshan Shen authored Apr 21, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  7272bfae
- [Misc] fix collect_env version parse (#15267) · d9ac9e3d
  wangxiyuan authored Apr 21, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  d9ac9e3d
- Restore buffers when wake up from level 2 sleep (#16564) (#16889) · d41faaf9
  Han Zhang authored Apr 21, 2025
```
Signed-off-by: Han <zh950713@gmail.com>
```
  d41faaf9
- [Doc] Split dummy_processor_inputs() in Multimodal Docs (#16915) · b34f3343
  Alex Brooks authored Apr 21, 2025
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  b34f3343
- [Bugfix] Fix distributed bug in Qwen2.5-VL & Qwen2.5-Omni (#16907) · 26c04065
  Yang Fan authored Apr 21, 2025
  
  26c04065
- [CI/CD][V1] Add spec decode tests to CI (#16900) · 4c41278b
  Woosuk Kwon authored Apr 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4c41278b
- [Bugfix] Fix v1/spec_decode/test_ngram.py (#16895) · bb3605db
  qizixi authored Apr 20, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  bb3605db
20 Apr, 2025 2 commits
- [easy] Pass compile_fx only the config patches (#16845) · fe742aef
  Richard Zou authored Apr 20, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  fe742aef
- Improve configs - `CacheConfig` (#16835) · 4b07d368
  Harry Mellor authored Apr 20, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  4b07d368
19 Apr, 2025 12 commits
- Serialize tensors using int8 views (#16866) · 87aaadef
  Staszek Paśko authored Apr 19, 2025
```
Signed-off-by: Staszek Pasko <staszek@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  87aaadef
- Log how much time loading a compiled artifact takes (#16848) · 682e0b6d
  Richard Zou authored Apr 19, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  682e0b6d
- [doc] update hyperlink (#16877) · d6195a74
  Reid authored Apr 20, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  d6195a74
- [VLM] Clean up models (#16873) · 205d84aa
  Cyrus Leung authored Apr 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  205d84aa
- [Model] Qwen2.5-Omni Cleanup (#16872) · 5124f5bf
  Roger Wang authored Apr 19, 2025
  
  5124f5bf
- [Model] Refactor Phi-4-multimodal to use merged processor and support V1 (#15477) · 83f3c3bd
  Isotr0py authored Apr 19, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  83f3c3bd
- [V1][Misc] stop update prefix cache stats when logs_stats is disabled (#16460) · d9737ca1
  vie-serendipity authored Apr 19, 2025
```
Signed-off-by: vie-serendipity <2733147505@qq.com>
```
  d9737ca1
- [Misc] Benchmarks for audio models (#16505) · 9d4ca19d
  Nicolò Lucchesi authored Apr 19, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  9d4ca19d
- [Frontend] Add sampling params to `v1/audio/transcriptions` endpoint (#16591) · 2ef0dc53
  Nicolò Lucchesi authored Apr 19, 2025
```
Signed-off-by: Jannis Schönleber <joennlae@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Jannis Schönleber <joennlae@gmail.com>
```
  2ef0dc53
- [rocm][MI300] llama4 maverick fp8 moe config tp8 (#16847) · 1d4680fa
  Divakar Verma authored Apr 19, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  1d4680fa
- [Model][VLM] Add Qwen2.5-Omni model support (thinker only) (#15130) · 2c1bd848
  Yang Fan authored Apr 19, 2025
```
Signed-off-by: fyabc <suyang.fy@alibaba-inc.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Xiong Wang <wangxiongts@163.com>
```
  2c1bd848
- [release] Publish neuron docker image (#16733) · 5c912120
  omrishiv authored Apr 19, 2025
```
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
```
  5c912120
18 Apr, 2025 8 commits
- [Doc] Updated Llama section in tool calling docs to have llama 3.2 config info (#16857) · 490b1698
  Justin Ho authored Apr 18, 2025
```
Signed-off-by: jmho <jaylenho734@gmail.com>
```
  490b1698
- [Misc] refactor examples series - Chat Completion Client With Tools (#16829) · 5a5e29de
  Reid authored Apr 19, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  5a5e29de
- [New Model]: Snowflake Arctic Embed (Family) (#16649) · 3d3ab368
  wang.yuqi authored Apr 18, 2025
  
  3d3ab368
- Fix `nullable_kvs` fallback (#16837) · 686623c5
  Harry Mellor authored Apr 18, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  686623c5
- [Misc] Clean up Kimi-VL (#16833) · aadb6565
  Cyrus Leung authored Apr 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  aadb6565
- [Model] use AutoWeightsLoader for BigCode, GPT-J (#16823) · 87e067de
  Jonghyun Choe authored Apr 18, 2025
```
Signed-off-by: Jonghyun Choe <andy.choe729@gmail.com>
```
  87e067de
- [Docs] Fix a link and grammar issue in production-stack.md (#16809) · 26507f89
  Michael Yao authored Apr 18, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  26507f89
- [Doc] add podman setup instructions for official image (#16796) · 9c1d5b45
  Nathan Weinberg authored Apr 18, 2025
```
Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
```
  9c1d5b45