Commits · 42b4f46b71572e21582fd12c498ec3b0b78ada7b · OpenDAS / vllm_cscc

08 Nov, 2024 2 commits
- [V1] Add all_token_ids attribute to Request (#10135) · 42b4f46b
  Woosuk Kwon authored Nov 07, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  42b4f46b
- [Misc] report relevant env vars in collect_env.py tool (#9293) · 073a4727
  Jiangtao Hu authored Nov 07, 2024
  
  073a4727
07 Nov, 2024 25 commits
- Bump actions/checkout from 4.2.1 to 4.2.2 (#9746) · 93bff421
  dependabot[bot] authored Nov 07, 2024
```
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
```
  93bff421
- Online video support for VLMs (#10020) · 28b2877d
  litianjian authored Nov 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  28b2877d
- Bump actions/setup-python from 5.2.0 to 5.3.0 (#9745) · 97b8475b
  dependabot[bot] authored Nov 07, 2024
```
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
```
  97b8475b
- [CI/Build] Automate PR body text cleanup (#10082) · a2f1f3b0
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  a2f1f3b0
- [CI/Build] Add shell script linting using shellcheck (#7925) · 3be5b26a
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  3be5b26a
- [CI/Build] Always run mypy (#10122) · de0e61a3
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  de0e61a3
- [Feature] [Spec decode]: Combine chunked prefill with speculative decoding (#9291) · 9d43afcc
  Nicolò Lucchesi authored Nov 07, 2024
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  9d43afcc
- [Frontend] Tool calling parser for Granite 3.0 models (#9027) · ae62fd17
  Maximilien de Bayser authored Nov 07, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  ae62fd17
- [Misc] Add Gamma-Distribution Request Generation Support for Serving Benchmark. (#10105) · a62bc010
  Atlas authored Nov 07, 2024
```
Signed-off-by: Mozhou <spli161006@gmail.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  a62bc010
- [Bugfix] Make image processor respect `mm_processor_kwargs` for Qwen2-VL (#10112) · 999df95b
  Jiahao Li authored Nov 07, 2024
```
Signed-off-by: Jiahao Li <liplus17@163.com>
```
  999df95b
- [Hardware][CPU][bugfix] Fix half dtype support on AVX2-only target (#10108) · a6f332d0
  Li, Jiang authored Nov 07, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a6f332d0
- [Frontend] Fix multiple values for keyword argument error (#10075) (#10076) · 0dfba97b
  Lei Yang authored Nov 07, 2024
```
Signed-off-by: Lei <ylxx@live.com>
```
  0dfba97b
- Adds method to read the pooling types from model's files (#9506) · aa9078fa
  Flávia Béo authored Nov 07, 2024
```
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
```
  aa9078fa
- [CI/Build] Improve mypy + python version matrix (#10041) · e036e527
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  e036e527
- [Core][Distributed] Refactor ipc buffer init in CustomAllreduce (#10030) · 6192e9b8
  Hanzhi Zhou authored Nov 06, 2024
```
Signed-off-by: Hanzhi Zhou <hanzhi713@gmail.com>
```
  6192e9b8
- Doc: Improve benchmark documentation (#9927) · d7263a1b
  Rafael Vasquez authored Nov 07, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  d7263a1b
- [CI/Build] re-add codespell to CI (#10083) · 104d7296
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  104d7296
- [Misc] Consolidate ModelConfig code related to HF config (#10104) · db7db4aa
  Cyrus Leung authored Nov 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  db7db4aa
- [V1][BugFix] Fix Generator construction in greedy + seed case (#10097) · 1fa020c5
  Nick Hill authored Nov 07, 2024
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  1fa020c5
- [doc] add back Python 3.8 ABI (#10100) · e7b84c39
  youkaichao authored Nov 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  e7b84c39
- [Hardware][CPU] Update torch 2.5 (#9911) · a4b3e0c1
  Li, Jiang authored Nov 07, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a4b3e0c1
- [Frontend] Adjust try/except blocks in API impl (#10056) · 29862b88
  Nick Hill authored Nov 07, 2024
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  29862b88
- [Misc][XPU] Upgrade to Pytorch 2.5 for xpu backend (#9823) · d3859f18
  Yan Ma authored Nov 07, 2024
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: yan ma <yan.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  d3859f18
- [Bugfix] Fix FP8 torch._scaled_mm fallback for torch>2.5 with CUDA<12.4 (#10095) · 4ab32566
  Michael Goin authored Nov 06, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  4ab32566
- [core][distributed] add stateless_init_process_group (#10072) · 719c1ca4
  youkaichao authored Nov 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  719c1ca4
06 Nov, 2024 13 commits

[CI/Build] Always run the ruff workflow (#10092) · 74f2f8a0
Russell Bryant authored Nov 06, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
74f2f8a0
[V1] Make v1 more testable (#9888) · d58268c5
Joe Runde authored Nov 06, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
d58268c5
[CI/Build] change conflict PR comment from mergify (#10080) · 87bd7e05
Russell Bryant authored Nov 06, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
87bd7e05

[CI/Build] Drop Python 3.8 support (#10038) · 098f94de

Russell Bryant authored Nov 06, 2024


Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

098f94de

Remove ScaledActivation for AWQ (#10057) · 399c7986
Michael Goin authored Nov 06, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
399c7986
[Model][LoRA]LoRA support added for Qwen2VLForConditionalGeneration (#10022) · 406d4cc4
Eric authored Nov 06, 2024
```
Signed-off-by: ericperfect <ericperfectttt@gmail.com>
```
406d4cc4

[Model] Add Idefics3 support (#9767) · a5bba7d2

Jee Jee Li authored Nov 06, 2024


Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: B-201 <Joy25810@foxmail.com>

a5bba7d2

[Model][LoRA]LoRA support added for LlamaEmbeddingModel (#10071) · 2003cc35
Jee Jee Li authored Nov 06, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
2003cc35
[Hotfix] Fix ruff errors (#10073) · 6a585a23
Woosuk Kwon authored Nov 06, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
6a585a23

[Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) · a02a50e6

Konrad Zawora authored Nov 06, 2024


Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Bob Zhu <bob.zhu@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
Co-authored-by: Marceli Fylcek <mfylcek@habana.ai>
Co-authored-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
Co-authored-by: Vivek Goel <vgoel@habana.ai>
Co-authored-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Dominika Olszewska <dolszewska@habana.ai>
Co-authored-by: barak goldberg <149692267+bgoldberg-habana@users.noreply.github.com>
Co-authored-by: Michal Szutenberg <37601244+szutenberg@users.noreply.github.com>
Co-authored-by: Jan Kaniecki <jkaniecki@habana.ai>
Co-authored-by: Agata Dobrzynie...