Commits · 0535e5fe6c38a25bad71d92bb7a396f04fd1aee5 · OpenDAS / vllm_cscc · GitLab

08 Nov, 2024 16 commits
- Fix edge case Mistral tokenizer (#10152) · 0535e5fe
  Patrick von Platen authored Nov 08, 2024
  
  0535e5fe
- [CI/Build] Update CPU tests to include all "standard" tests (#5481) · b489fc3c
  Cyrus Leung authored Nov 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b489fc3c
- [V1]Enable APC by default only for text models (#10148) · 208ce622
  Roger Wang authored Nov 08, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  208ce622
- [Model] Expose size to Idefics3 as mm_processor_kwargs (#10146) · 1ff4aed5
  Isotr0py authored Nov 08, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  1ff4aed5
- [Bugfix][XPU] Fix xpu tp by introducing XpuCommunicator (#10144) · f10797c0
  Yan Ma authored Nov 08, 2024
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  f10797c0
- [Misc] Fix typo in #5895 (#10145) · f4c2187e
  Cyrus Leung authored Nov 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f4c2187e
- Add hf_transfer to testing image (#10096) · aea6ad62
  Michael Goin authored Nov 08, 2024
  
  aea6ad62
- Fixes a typo about 'max_decode_seq_len' which causes crashes with cuda graph. (#9285) · da07a9ea
  Tao He authored Nov 08, 2024
```
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
```
  da07a9ea
- [Doc] Move CONTRIBUTING to docs site (#9924) · 3a7f15a3
  Russell Bryant authored Nov 08, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  3a7f15a3
- [Misc] Fix ImportError causing by triton (#9493) · 7371749d
  Mengqing Cao authored Nov 08, 2024
  
  7371749d
- [Bugfix] Add error handling when server cannot respond any valid tokens (#5895) · ad39bd64
  DearPlanet authored Nov 08, 2024
  
  ad39bd64
- [Doc] Update FAQ links in spec_decode.rst (#9662) · 40d0e741
  whyiug authored Nov 08, 2024
```
Signed-off-by: whyiug <whyiug@hotmail.com>
```
  40d0e741
- [CI/Build] Give PR cleanup job PR write access (#10139) · 6bb52b0f
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  6bb52b0f
- [V1] Prefix caching (take 2) (#9972) · 201fc077
  Cody Yu authored Nov 07, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  201fc077
- [V1] Add all_token_ids attribute to Request (#10135) · 42b4f46b
  Woosuk Kwon authored Nov 07, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  42b4f46b
- [Misc] report relevant env vars in collect_env.py tool (#9293) · 073a4727
  Jiangtao Hu authored Nov 07, 2024
  
  073a4727
07 Nov, 2024 24 commits
- Bump actions/checkout from 4.2.1 to 4.2.2 (#9746) · 93bff421
  dependabot[bot] authored Nov 07, 2024
```
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
```
  93bff421
- Online video support for VLMs (#10020) · 28b2877d
  litianjian authored Nov 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  28b2877d
- Bump actions/setup-python from 5.2.0 to 5.3.0 (#9745) · 97b8475b
  dependabot[bot] authored Nov 07, 2024
```
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
```
  97b8475b
- [CI/Build] Automate PR body text cleanup (#10082) · a2f1f3b0
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  a2f1f3b0
- [CI/Build] Add shell script linting using shellcheck (#7925) · 3be5b26a
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  3be5b26a
- [CI/Build] Always run mypy (#10122) · de0e61a3
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  de0e61a3
- [Feature] [Spec decode]: Combine chunked prefill with speculative decoding (#9291) · 9d43afcc
  Nicolò Lucchesi authored Nov 07, 2024
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  9d43afcc
- [Frontend] Tool calling parser for Granite 3.0 models (#9027) · ae62fd17
  Maximilien de Bayser authored Nov 07, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  ae62fd17
- [Misc] Add Gamma-Distribution Request Generation Support for Serving Benchmark. (#10105) · a62bc010
  Atlas authored Nov 07, 2024
```
Signed-off-by: Mozhou <spli161006@gmail.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  a62bc010
- [Bugfix] Make image processor respect `mm_processor_kwargs` for Qwen2-VL (#10112) · 999df95b
  Jiahao Li authored Nov 07, 2024
```
Signed-off-by: Jiahao Li <liplus17@163.com>
```
  999df95b
- [Hardware][CPU][bugfix] Fix half dtype support on AVX2-only target (#10108) · a6f332d0
  Li, Jiang authored Nov 07, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a6f332d0
- [Frontend] Fix multiple values for keyword argument error (#10075) (#10076) · 0dfba97b
  Lei Yang authored Nov 07, 2024
```
Signed-off-by: Lei <ylxx@live.com>
```
  0dfba97b
- Adds method to read the pooling types from model's files (#9506) · aa9078fa
  Flávia Béo authored Nov 07, 2024
```
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
```
  aa9078fa
- [CI/Build] Improve mypy + python version matrix (#10041) · e036e527
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  e036e527
- [Core][Distributed] Refactor ipc buffer init in CustomAllreduce (#10030) · 6192e9b8
  Hanzhi Zhou authored Nov 06, 2024
```
Signed-off-by: Hanzhi Zhou <hanzhi713@gmail.com>
```
  6192e9b8
- Doc: Improve benchmark documentation (#9927) · d7263a1b
  Rafael Vasquez authored Nov 07, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  d7263a1b
- [CI/Build] re-add codespell to CI (#10083) · 104d7296
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  104d7296
- [Misc] Consolidate ModelConfig code related to HF config (#10104) · db7db4aa
  Cyrus Leung authored Nov 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  db7db4aa
- [V1][BugFix] Fix Generator construction in greedy + seed case (#10097) · 1fa020c5
  Nick Hill authored Nov 07, 2024
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  1fa020c5
- [doc] add back Python 3.8 ABI (#10100) · e7b84c39
  youkaichao authored Nov 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  e7b84c39
- [Hardware][CPU] Update torch 2.5 (#9911) · a4b3e0c1
  Li, Jiang authored Nov 07, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a4b3e0c1
- [Frontend] Adjust try/except blocks in API impl (#10056) · 29862b88
  Nick Hill authored Nov 07, 2024
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  29862b88
- [Misc][XPU] Upgrade to Pytorch 2.5 for xpu backend (#9823) · d3859f18
  Yan Ma authored Nov 07, 2024
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: yan ma <yan.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  d3859f18
- [Bugfix] Fix FP8 torch._scaled_mm fallback for torch>2.5 with CUDA<12.4 (#10095) · 4ab32566
  Michael Goin authored Nov 06, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  4ab32566