Commits · ad39bd640cdaaf2963cd07a7cc912c1dde516ed0 · OpenDAS / vllm_cscc · GitLab

08 Nov, 2024 6 commits
- [Bugfix] Add error handling when server cannot respond any valid tokens (#5895) · ad39bd64
  DearPlanet authored Nov 08, 2024
  
  ad39bd64
- [Doc] Update FAQ links in spec_decode.rst (#9662) · 40d0e741
  whyiug authored Nov 08, 2024
```
Signed-off-by: whyiug <whyiug@hotmail.com>
```
  40d0e741
- [CI/Build] Give PR cleanup job PR write access (#10139) · 6bb52b0f
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  6bb52b0f
- [V1] Prefix caching (take 2) (#9972) · 201fc077
  Cody Yu authored Nov 07, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  201fc077
- [V1] Add all_token_ids attribute to Request (#10135) · 42b4f46b
  Woosuk Kwon authored Nov 07, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  42b4f46b
- [Misc] report relevant env vars in collect_env.py tool (#9293) · 073a4727
  Jiangtao Hu authored Nov 07, 2024
  
  073a4727
07 Nov, 2024 25 commits
- Bump actions/checkout from 4.2.1 to 4.2.2 (#9746) · 93bff421
  dependabot[bot] authored Nov 07, 2024
```
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
```
  93bff421
- Online video support for VLMs (#10020) · 28b2877d
  litianjian authored Nov 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  28b2877d
- Bump actions/setup-python from 5.2.0 to 5.3.0 (#9745) · 97b8475b
  dependabot[bot] authored Nov 07, 2024
```
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
```
  97b8475b
- [CI/Build] Automate PR body text cleanup (#10082) · a2f1f3b0
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  a2f1f3b0
- [CI/Build] Add shell script linting using shellcheck (#7925) · 3be5b26a
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  3be5b26a
- [CI/Build] Always run mypy (#10122) · de0e61a3
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  de0e61a3
- [Feature] [Spec decode]: Combine chunked prefill with speculative decoding (#9291) · 9d43afcc
  Nicolò Lucchesi authored Nov 07, 2024
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  9d43afcc
- [Frontend] Tool calling parser for Granite 3.0 models (#9027) · ae62fd17
  Maximilien de Bayser authored Nov 07, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  ae62fd17
- [Misc] Add Gamma-Distribution Request Generation Support for Serving Benchmark. (#10105) · a62bc010
  Atlas authored Nov 07, 2024
```
Signed-off-by: Mozhou <spli161006@gmail.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  a62bc010
- [Bugfix] Make image processor respect `mm_processor_kwargs` for Qwen2-VL (#10112) · 999df95b
  Jiahao Li authored Nov 07, 2024
```
Signed-off-by: Jiahao Li <liplus17@163.com>
```
  999df95b
- [Hardware][CPU][bugfix] Fix half dtype support on AVX2-only target (#10108) · a6f332d0
  Li, Jiang authored Nov 07, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a6f332d0
- [Frontend] Fix multiple values for keyword argument error (#10075) (#10076) · 0dfba97b
  Lei Yang authored Nov 07, 2024
```
Signed-off-by: Lei <ylxx@live.com>
```
  0dfba97b
- Adds method to read the pooling types from model's files (#9506) · aa9078fa
  Flávia Béo authored Nov 07, 2024
```
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
```
  aa9078fa
- [CI/Build] Improve mypy + python version matrix (#10041) · e036e527
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  e036e527
- [Core][Distributed] Refactor ipc buffer init in CustomAllreduce (#10030) · 6192e9b8
  Hanzhi Zhou authored Nov 06, 2024
```
Signed-off-by: Hanzhi Zhou <hanzhi713@gmail.com>
```
  6192e9b8
- Doc: Improve benchmark documentation (#9927) · d7263a1b
  Rafael Vasquez authored Nov 07, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  d7263a1b
- [CI/Build] re-add codespell to CI (#10083) · 104d7296
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  104d7296
- [Misc] Consolidate ModelConfig code related to HF config (#10104) · db7db4aa
  Cyrus Leung authored Nov 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  db7db4aa
- [V1][BugFix] Fix Generator construction in greedy + seed case (#10097) · 1fa020c5
  Nick Hill authored Nov 07, 2024
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  1fa020c5
- [doc] add back Python 3.8 ABI (#10100) · e7b84c39
  youkaichao authored Nov 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  e7b84c39
- [Hardware][CPU] Update torch 2.5 (#9911) · a4b3e0c1
  Li, Jiang authored Nov 07, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a4b3e0c1
- [Frontend] Adjust try/except blocks in API impl (#10056) · 29862b88
  Nick Hill authored Nov 07, 2024
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  29862b88
- [Misc][XPU] Upgrade to Pytorch 2.5 for xpu backend (#9823) · d3859f18
  Yan Ma authored Nov 07, 2024
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: yan ma <yan.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  d3859f18
- [Bugfix] Fix FP8 torch._scaled_mm fallback for torch>2.5 with CUDA<12.4 (#10095) · 4ab32566
  Michael Goin authored Nov 06, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  4ab32566
- [core][distributed] add stateless_init_process_group (#10072) · 719c1ca4
  youkaichao authored Nov 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  719c1ca4
06 Nov, 2024 9 commits
- [CI/Build] Always run the ruff workflow (#10092) · 74f2f8a0
  Russell Bryant authored Nov 06, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  74f2f8a0
- [V1] Make v1 more testable (#9888) · d58268c5
  Joe Runde authored Nov 06, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  d58268c5
- [CI/Build] change conflict PR comment from mergify (#10080) · 87bd7e05
  Russell Bryant authored Nov 06, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  87bd7e05
- [CI/Build] Drop Python 3.8 support (#10038) · 098f94de
  Russell Bryant authored Nov 06, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  098f94de
- Remove ScaledActivation for AWQ (#10057) · 399c7986
  Michael Goin authored Nov 06, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  399c7986
- [Model][LoRA]LoRA support added for Qwen2VLForConditionalGeneration (#10022) · 406d4cc4
  Eric authored Nov 06, 2024
```
Signed-off-by: ericperfect <ericperfectttt@gmail.com>
```
  406d4cc4
- [Model] Add Idefics3 support (#9767) · a5bba7d2
  Jee Jee Li authored Nov 06, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: B-201 <Joy25810@foxmail.com>
```
  a5bba7d2
- [Model][LoRA]LoRA support added for LlamaEmbeddingModel (#10071) · 2003cc35
  Jee Jee Li authored Nov 06, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  2003cc35
- [Hotfix] Fix ruff errors (#10073) · 6a585a23
  Woosuk Kwon authored Nov 06, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6a585a23