Commits · c091c0a58898b8a0a76e18bd6724732d80fcfc28 · OpenDAS / vllm_cscc

26 Mar, 2025 3 commits

Improve validation of TP in Transformers backend (#15540) · c091c0a5
Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
c091c0a5

Separate base model from `TransformersModel` (#15467) · cf5c8f16

Harry Mellor authored Mar 26, 2025


Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

cf5c8f16

Transformers backend already supports V1 (#15463) · e42389f9
Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
e42389f9

25 Mar, 2025 1 commit

Add pipeline parallel support to `TransformersModel` (#12832) · 97cfa65d

Harry Mellor authored Mar 25, 2025


Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

97cfa65d

05 Mar, 2025 1 commit
- [LoRA] Remove linear hack outside transformers backend (#14177) · e17e4488
  Isotr0py authored Mar 05, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  e17e4488
03 Mar, 2025 1 commit
- Fix `head_dim` not existing in all model configs (Transformers backend) (#14141) · 91373a0d
  Harry Mellor authored Mar 03, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  91373a0d
02 Mar, 2025 1 commit
- [Model] Add LoRA support for TransformersModel (#13770) · cc5e8f6d
  Jee Jee Li authored Mar 02, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  cc5e8f6d
25 Feb, 2025 1 commit
- Remove unused kwargs from model definitions (#13555) · cdc1fa12
  Harry Mellor authored Feb 25, 2025
  
  cdc1fa12
22 Feb, 2025 1 commit
- [Misc] Reduce LoRA-related static variable (#13166) · 105b8ce4
  Jee Jee Li authored Feb 22, 2025
  
  105b8ce4
18 Feb, 2025 1 commit
- [Model] Enable quantization support for `transformers` backend (#12960) · 67ef8f66
  Isotr0py authored Feb 18, 2025
  
  67ef8f66
11 Feb, 2025 1 commit
- Set `torch_dtype` in `TransformersModel` (#13088) · ad977635
  Harry Mellor authored Feb 11, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  ad977635
06 Feb, 2025 1 commit
- Improve `TransformersModel` UX (#12785) · 1a6fcad4
  Harry Mellor authored Feb 06, 2025
  
  1a6fcad4
05 Feb, 2025 1 commit
- Refactor `Linear` handling in `TransformersModel` (#12727) · 249824c3
  Harry Mellor authored Feb 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  249824c3
03 Feb, 2025 2 commits

[Misc] Fix improper placement of SPDX header in scripts (#12694) · 33e0602e
Russell Bryant authored Feb 03, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
33e0602e

[Model]: Add `transformers` backend support (#11330) · a1a2aaad

Arthur authored Feb 03, 2025

# Adds support for `transformers` as a backend

Following https://github.com/huggingface/transformers/pull/35235

, a
bunch of models should already be supported, we are ramping up support
for more models.

Thanks @Isotr0py for the TP support, and @hmellor for his help as well!
This includes: 
- `trust_remote_code=True` support: any model on the hub, if it
implements attention the correct way can be natively supported!!
- tensor parallel support

---------
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <41363108+Isotr0py@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

a1a2aaad