Commits · 6192e9b8fef8492c3e52bd65c7d954a1ef9b40c8 · OpenDAS / vllm_cscc · GitLab

07 Nov, 2024 11 commits
- [Core][Distributed] Refactor ipc buffer init in CustomAllreduce (#10030) · 6192e9b8
  Hanzhi Zhou authored Nov 06, 2024
```
Signed-off-by: Hanzhi Zhou <hanzhi713@gmail.com>
```
  6192e9b8
- Doc: Improve benchmark documentation (#9927) · d7263a1b
  Rafael Vasquez authored Nov 07, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  d7263a1b
- [CI/Build] re-add codespell to CI (#10083) · 104d7296
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  104d7296
- [Misc] Consolidate ModelConfig code related to HF config (#10104) · db7db4aa
  Cyrus Leung authored Nov 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  db7db4aa
- [V1][BugFix] Fix Generator construction in greedy + seed case (#10097) · 1fa020c5
  Nick Hill authored Nov 07, 2024
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  1fa020c5
- [doc] add back Python 3.8 ABI (#10100) · e7b84c39
  youkaichao authored Nov 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  e7b84c39
- [Hardware][CPU] Update torch 2.5 (#9911) · a4b3e0c1
  Li, Jiang authored Nov 07, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a4b3e0c1
- [Frontend] Adjust try/except blocks in API impl (#10056) · 29862b88
  Nick Hill authored Nov 07, 2024
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  29862b88
- [Misc][XPU] Upgrade to Pytorch 2.5 for xpu backend (#9823) · d3859f18
  Yan Ma authored Nov 07, 2024
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: yan ma <yan.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  d3859f18
- [Bugfix] Fix FP8 torch._scaled_mm fallback for torch>2.5 with CUDA<12.4 (#10095) · 4ab32566
  Michael Goin authored Nov 06, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  4ab32566
- [core][distributed] add stateless_init_process_group (#10072) · 719c1ca4
  youkaichao authored Nov 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  719c1ca4
06 Nov, 2024 23 commits
- [CI/Build] Always run the ruff workflow (#10092) · 74f2f8a0
  Russell Bryant authored Nov 06, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  74f2f8a0
- [V1] Make v1 more testable (#9888) · d58268c5
  Joe Runde authored Nov 06, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  d58268c5
- [CI/Build] change conflict PR comment from mergify (#10080) · 87bd7e05
  Russell Bryant authored Nov 06, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  87bd7e05
- [CI/Build] Drop Python 3.8 support (#10038) · 098f94de
  Russell Bryant authored Nov 06, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  098f94de
- Remove ScaledActivation for AWQ (#10057) · 399c7986
  Michael Goin authored Nov 06, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  399c7986
- [Model][LoRA]LoRA support added for Qwen2VLForConditionalGeneration (#10022) · 406d4cc4
  Eric authored Nov 06, 2024
```
Signed-off-by: ericperfect <ericperfectttt@gmail.com>
```
  406d4cc4
- [Model] Add Idefics3 support (#9767) · a5bba7d2
  Jee Jee Li authored Nov 06, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: B-201 <Joy25810@foxmail.com>
```
  a5bba7d2
- [Model][LoRA]LoRA support added for LlamaEmbeddingModel (#10071) · 2003cc35
  Jee Jee Li authored Nov 06, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  2003cc35
- [Hotfix] Fix ruff errors (#10073) · 6a585a23
  Woosuk Kwon authored Nov 06, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6a585a23
- [Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) · a02a50e6
  Konrad Zawora authored Nov 06, 2024
```
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Bob Zhu <bob.zhu@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
Co-authored-by: Marceli Fylcek <mfylcek@habana.ai>
Co-authored-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
Co-authored-by: Vivek Goel <vgoel@habana.ai>
Co-authored-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Dominika Olszewska <dolszewska@habana.ai>
Co-authored-by: barak goldberg <149692267+bgoldberg-habana@users.noreply.github.com>
Co-authored-by: Michal Szutenberg <37601244+szutenberg@users.noreply.github.com>
Co-authored-by: Jan Kaniecki <jkaniecki@habana.ai>
Co-authored-by: Agata Dobrzynie...
```
  a02a50e6
- [CI/Build] Fix large_gpu_mark reason (#10070) · a5fda50a
  Isotr0py authored Nov 06, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  a5fda50a
- [CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
  Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  21063c11
- [distributed] add function to create ipc buffers directly (#10064) · 4be3a451
  youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4be3a451
- [V1] Integrate Piecewise CUDA graphs (#10058) · 40899855
  Woosuk Kwon authored Nov 05, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  40899855
- [Bugfix] Remove CustomChatCompletionContentPartParam multimodal input type (#10054) · 9d59b755
  zifeitong authored Nov 05, 2024
```
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
```
  9d59b755
- [Bugfix] Gpt-j-6B patch kv_scale to k_scale path (#10063) · ea928f60
  arakowsk-amd authored Nov 05, 2024
```
Signed-off-by: Alex Rakowski <alex.rakowski@amd.com>
Signed-off-by: Alex Rakowski <182798202+arakowsk-amd@users.noreply.github.com>
```
  ea928f60
- [Bugfix] Fix edge-case crash when using chat with the Mistral Tekken Tokenizer (#10051) · 2bcbae70
  Travis Johnson authored Nov 05, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  2bcbae70
- [Model][OpenVINO] Fix regressions from #8346 (#10045) · ffc0f2b4
  Peter Salas authored Nov 05, 2024
```
Signed-off-by: Peter Salas <peter@fixie.ai>
```
  ffc0f2b4
- [Misc] Sort the list of embedding models (#10037) · 82bfc38d
  Cyrus Leung authored Nov 06, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  82bfc38d
- [v1] reduce graph capture time for piecewise cudagraph (#10059) · c4cacbaa
  youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c4cacbaa
- [Bugfix][SpecDecode] kv corruption with bonus tokens in spec decode (#9730) · 0c63c34f
  Sungjae Lee authored Nov 06, 2024
```
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  0c63c34f
- [Bugfix] Fix pickle of input when async output processing is on (#9931) · 966e3169
  Wallas Henrique authored Nov 05, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  966e3169
- [Bugfix] Properly propagate trust_remote_code settings (#10047) · 43300bd9
  zifeitong authored Nov 05, 2024
```
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
```
  43300bd9
05 Nov, 2024 6 commits
- [bugfix] fix weak ref in piecewise cudagraph and tractable test (#10048) · ca9844b3
  youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ca9844b3
- [CI] Prune back the number of tests in tests/kernels/* (#9932) · 235366fe
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  235366fe
- [CI] Prune tests/models/decoder_only/language/* tests (#9940) · 02462465
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  02462465
- [Misc] Modify BNB parameter name (#9997) · b9c64c0c
  Jee Jee Li authored Nov 06, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  b9c64c0c
- [Feature] Update benchmark_throughput.py to support image input (#9851) · d2e80332
  lkchen authored Nov 05, 2024
```
Signed-off-by: Linkun Chen <github+anyscale@lkchen.net>
Co-authored-by: Linkun Chen <github+anyscale@lkchen.net>
```
  d2e80332
- [Model] Support quantization of PixtralHFTransformer for PixtralHF (#9921) · a53046b1
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  a53046b1