Commits · a4b3e0c1e999d214c6355b16a1c68250e6c030e2 · OpenDAS / vllm_cscc

"vllm/vscode:/vscode.git/clone" did not exist on "d3cf61b89bc53aa7709932ab43e7630b9a71f2b3"

07 Nov, 2024 5 commits
- [Hardware][CPU] Update torch 2.5 (#9911) · a4b3e0c1
  Li, Jiang authored Nov 07, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a4b3e0c1
- [Frontend] Adjust try/except blocks in API impl (#10056) · 29862b88
  Nick Hill authored Nov 07, 2024
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  29862b88
- [Misc][XPU] Upgrade to Pytorch 2.5 for xpu backend (#9823) · d3859f18
  Yan Ma authored Nov 07, 2024
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: yan ma <yan.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  d3859f18
- [Bugfix] Fix FP8 torch._scaled_mm fallback for torch>2.5 with CUDA<12.4 (#10095) · 4ab32566
  Michael Goin authored Nov 06, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  4ab32566
- [core][distributed] add stateless_init_process_group (#10072) · 719c1ca4
  youkaichao authored Nov 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  719c1ca4
06 Nov, 2024 23 commits

[CI/Build] Always run the ruff workflow (#10092) · 74f2f8a0
Russell Bryant authored Nov 06, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
74f2f8a0
[V1] Make v1 more testable (#9888) · d58268c5
Joe Runde authored Nov 06, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
d58268c5
[CI/Build] change conflict PR comment from mergify (#10080) · 87bd7e05
Russell Bryant authored Nov 06, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
87bd7e05

[CI/Build] Drop Python 3.8 support (#10038) · 098f94de

Russell Bryant authored Nov 06, 2024


Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

098f94de

Remove ScaledActivation for AWQ (#10057) · 399c7986
Michael Goin authored Nov 06, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
399c7986
[Model][LoRA]LoRA support added for Qwen2VLForConditionalGeneration (#10022) · 406d4cc4
Eric authored Nov 06, 2024
```
Signed-off-by: ericperfect <ericperfectttt@gmail.com>
```
406d4cc4

[Model] Add Idefics3 support (#9767) · a5bba7d2

Jee Jee Li authored Nov 06, 2024


Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: B-201 <Joy25810@foxmail.com>

a5bba7d2

[Model][LoRA]LoRA support added for LlamaEmbeddingModel (#10071) · 2003cc35
Jee Jee Li authored Nov 06, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
2003cc35
[Hotfix] Fix ruff errors (#10073) · 6a585a23
Woosuk Kwon authored Nov 06, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
6a585a23

[Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) · a02a50e6

Konrad Zawora authored Nov 06, 2024


Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Bob Zhu <bob.zhu@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
Co-authored-by: Marceli Fylcek <mfylcek@habana.ai>
Co-authored-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
Co-authored-by: Vivek Goel <vgoel@habana.ai>
Co-authored-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Dominika Olszewska <dolszewska@habana.ai>
Co-authored-by: barak goldberg <149692267+bgoldberg-habana@users.noreply.github.com>
Co-authored-by: Michal Szutenberg <37601244+szutenberg@users.noreply.github.com>
Co-authored-by: Jan Kaniecki <jkaniecki@habana.ai>
Co-authored-by: Agata Dobrzyniewicz <160237065+adobrzyniewicz-habana@users.noreply.github.com>
Co-authored-by: Krzysztof Wisniewski <kwisniewski@habana.ai>
Co-authored-by: Dudi Lester <160421192+dudilester@users.noreply.github.com>
Co-authored-by: Ilia Taraban <tarabanil@gmail.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Michał Kuligowski <mkuligowski@habana.ai>
Co-authored-by: Jakub Maksymczuk <jmaksymczuk@habana.ai>
Co-authored-by: Tomasz Zielinski <85164140+tzielinski-habana@users.noreply.github.com>
Co-authored-by: Sun Choi <schoi@habana.ai>
Co-authored-by: Iryna Boiko <iboiko@habana.ai>
Co-authored-by: Bob Zhu <41610754+czhu15@users.noreply.github.com>
Co-authored-by: hlin99 <73271530+hlin99@users.noreply.github.com>
Co-authored-by: Zehao Huang <zehao.huang@intel.com>
Co-authored-by: Andrzej Kotłowski <Andrzej.Kotlowski@intel.com>
Co-authored-by: Yan Tomsinsky <73292515+Yantom1@users.noreply.github.com>
Co-authored-by: Nir David <ndavid@habana.ai>
Co-authored-by: Yu-Zhou <yu.zhou@intel.com>
Co-authored-by: Ruheena Suhani Shaik <rsshaik@habana.ai>
Co-authored-by: Karol Damaszke <kdamaszke@habana.ai>
Co-authored-by: Marcin Swiniarski <mswiniarski@habana.ai>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Jacek Czaja <jacek.czaja@intel.com>
Co-authored-by: Jacek Czaja <jczaja@habana.ai>
Co-authored-by: Yuan <yuan.zhou@outlook.com>

a02a50e6

[CI/Build] Fix large_gpu_mark reason (#10070) · a5fda50a
Isotr0py authored Nov 06, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
a5fda50a
[CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
21063c11
[distributed] add function to create ipc buffers directly (#10064) · 4be3a451
youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
4be3a451
[V1] Integrate Piecewise CUDA graphs (#10058) · 40899855
Woosuk Kwon authored Nov 05, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
40899855
[Bugfix] Remove CustomChatCompletionContentPartParam multimodal input type (#10054) · 9d59b755
zifeitong authored Nov 05, 2024
```
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
```
9d59b755

[Bugfix] Gpt-j-6B patch kv_scale to k_scale path (#10063) · ea928f60

arakowsk-amd authored Nov 05, 2024


Signed-off-by: Alex Rakowski <alex.rakowski@amd.com>
Signed-off-by: Alex Rakowski <182798202+arakowsk-amd@users.noreply.github.com>

ea928f60

[Bugfix] Fix edge-case crash when using chat with the Mistral Tekken Tokenizer (#10051) · 2bcbae70
Travis Johnson authored Nov 05, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
2bcbae70
[Model][OpenVINO] Fix regressions from #8346 (#10045) · ffc0f2b4
Peter Salas authored Nov 05, 2024
```
Signed-off-by: Peter Salas <peter@fixie.ai>
```
ffc0f2b4
[Misc] Sort the list of embedding models (#10037) · 82bfc38d
Cyrus Leung authored Nov 06, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
82bfc38d
[v1] reduce graph capture time for piecewise cudagraph (#10059) · c4cacbaa
youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
c4cacbaa
[Bugfix][SpecDecode] kv corruption with bonus tokens in spec decode (#9730) · 0c63c34f
Sungjae Lee authored Nov 06, 2024
```
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
0c63c34f
[Bugfix] Fix pickle of input when async output processing is on (#9931) · 966e3169
Wallas Henrique authored Nov 05, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
966e3169
[Bugfix] Properly propagate trust_remote_code settings (#10047) · 43300bd9
zifeitong authored Nov 05, 2024
```
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
```
43300bd9

05 Nov, 2024 12 commits
- [bugfix] fix weak ref in piecewise cudagraph and tractable test (#10048) · ca9844b3
  youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ca9844b3
- [CI] Prune back the number of tests in tests/kernels/* (#9932) · 235366fe
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  235366fe
- [CI] Prune tests/models/decoder_only/language/* tests (#9940) · 02462465
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  02462465
- [Misc] Modify BNB parameter name (#9997) · b9c64c0c
  Jee Jee Li authored Nov 06, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  b9c64c0c
- [Feature] Update benchmark_throughput.py to support image input (#9851) · d2e80332
  lkchen authored Nov 05, 2024
```
Signed-off-by: Linkun Chen <github+anyscale@lkchen.net>
Co-authored-by: Linkun Chen <github+anyscale@lkchen.net>
```
  d2e80332
- [Model] Support quantization of PixtralHFTransformer for PixtralHF (#9921) · a53046b1
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  a53046b1
- [CI/Build] Limit github CI jobs based on files changed (#9928) · 731aec5b
  Russell Bryant authored Nov 05, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  731aec5b
- [Misc] Add logging for CUDA memory (#10027) · 09d35503
  Chenghao (Alan) Yang authored Nov 05, 2024
```
Signed-off-by: Chenghao Yang <yangalan1996@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Chenghao Yang <yangalan1996@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  09d35503
- Refactor TPU requirements file and pin build dependencies (#10010) · cd34029e
  Richard Liu authored Nov 05, 2024
```
Signed-off-by: Richard Liu <ricliu@google.com>
```
  cd34029e
- [Frontend] Fix tcp port reservation for api server (#10012) · 5952d811
  Russell Bryant authored Nov 05, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  5952d811
- [Misc] vllm CLI flags should be ordered for better user readability (#10017) · 93dee88f
  Chauncey authored Nov 05, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  93dee88f
- [BugFix] Lazy import ray (#10021) · 7a83b1ae
  Gene Der Su authored Nov 05, 2024
  
  7a83b1ae