Commits · efbe8544489c8ee9bb0cf72985c5aae89c315a76 · OpenDAS / vllm_cscc

11 Feb, 2025 1 commit
- Set `torch_dtype` in `TransformersModel` (#13088) · ad977635
  Harry Mellor authored Feb 11, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  ad977635
06 Feb, 2025 1 commit
- Improve `TransformersModel` UX (#12785) · 1a6fcad4
  Harry Mellor authored Feb 06, 2025
  
  1a6fcad4
05 Feb, 2025 1 commit
- Refactor `Linear` handling in `TransformersModel` (#12727) · 249824c3
  Harry Mellor authored Feb 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  249824c3
03 Feb, 2025 2 commits

[Misc] Fix improper placement of SPDX header in scripts (#12694) · 33e0602e
Russell Bryant authored Feb 03, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
33e0602e

[Model]: Add `transformers` backend support (#11330) · a1a2aaad

Arthur authored Feb 03, 2025

# Adds support for `transformers` as a backend

Following https://github.com/huggingface/transformers/pull/35235

, a
bunch of models should already be supported, we are ramping up support
for more models.

Thanks @Isotr0py for the TP support, and @hmellor for his help as well!
This includes: 
- `trust_remote_code=True` support: any model on the hub, if it
implements attention the correct way can be natively supported!!
- tensor parallel support

---------
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <41363108+Isotr0py@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

a1a2aaad