Commits · f690372b6803c43870aefa631ab207ea3d163b1d · OpenDAS / vllm_cscc

19 Mar, 2025 1 commit
- [Core] Update dtype detection and defaults (#14858) · f690372b
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f690372b
18 Mar, 2025 3 commits

[Bugfix] Fix LoRA extra vocab size (#15047) · 46c759c1
Jee Jee Li authored Mar 19, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
46c759c1

[MODEL] Add support for Zamba2 models (#13185) · 452e8fd9

yury-tokpanov authored Mar 18, 2025


Signed-off-by: Yury Tokpanov <yury@zyphra.com>
Signed-off-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

452e8fd9

[Mistral-Small 3.1] Update docs and tests (#14977) · f863ffc9
Patrick von Platen authored Mar 18, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
f863ffc9

17 Mar, 2025 4 commits

[CI/Build] Use `AutoModelForImageTextToText` to load VLMs in tests (#14945) · b89fb2a4
Cyrus Leung authored Mar 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
b89fb2a4

[Bugfix][ROCm] running new process using spawn method for rocm in tests. (#14810) · 2bb0e1a7

vllmellm authored Mar 17, 2025


Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

2bb0e1a7

[Misc] Replace os environ to monkeypatch in test suite (#14516) · a73e183e

Sibi authored Mar 17, 2025


Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>

a73e183e

[CI] Nightly Tests (#14898) · bb3aeddf

Robert Shaw authored Mar 16, 2025


Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

bb3aeddf

16 Mar, 2025 1 commit

[VLM] Clean up Phi-4-MM ViT implementation (#14812) · def232e1

Isotr0py authored Mar 16, 2025


Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

def232e1

15 Mar, 2025 4 commits

[VLM] Merged multi-modal processor for Pixtral (#12211) · 61c6a5a7

Rémi Delacourt authored Mar 15, 2025


Signed-off-by: remi <remi@mistral.ai>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

61c6a5a7

[VLM] Limit multimodal input cache by memory (#14805) · 3556a414
Cyrus Leung authored Mar 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
3556a414
[CPU] Support FP8 KV cache (#14741) · a2ae4965
Li, Jiang authored Mar 15, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
a2ae4965

[V1] V1 Enablement Oracle (#13726) · d4d93db2

Robert Shaw authored Mar 15, 2025


Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

d4d93db2

14 Mar, 2025 2 commits
- [Bugfix] Fix Aria test loading (#14823) · 613c5bb9
  Cyrus Leung authored Mar 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  613c5bb9
- [CI] Fix missing example model id in processor test (#14787) · 0c2af17c
  Roger Wang authored Mar 13, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  0c2af17c
13 Mar, 2025 3 commits

[Misc] Clean up processor tests (#14771) · 8e9ffd37
Cyrus Leung authored Mar 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
8e9ffd37
[Bugfix] Fix prompt format of GLM4V (#14539) · f53a0586
Cyrus Leung authored Mar 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
f53a0586

[VLM] Support pan-and-scan for Gemma3 multi-modal processor (#14672) · 38240392

Cyrus Leung authored Mar 13, 2025


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>

38240392

12 Mar, 2025 5 commits

[FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash... · 916836bb

TJian authored Mar 13, 2025


[FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash Attention to enable embedding models. (#14664)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>

916836bb

[Model] Add support for Gemma 3 (#14660) · c0c25e25

Woosuk Kwon authored Mar 12, 2025


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

c0c25e25

[Kernel] Add ModelOpt FP4 Checkpoint Support (#12520) · debd6bbf
Pavani Majety authored Mar 11, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
debd6bbf
[Core] Refactor `QKVCrossParallelLinear` implementation to support BNB 4-bit quantization (#14545) · e392d858
Isotr0py authored Mar 12, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
e392d858
[Model] Extend Ultravox to accept audio longer than 30s (#13631) · 80e78d02
Farzad Abdolhosseini authored Mar 11, 2025
```
Signed-off-by: Farzad Abdolhosseini <farzad@fixie.ai>
```
80e78d02

11 Mar, 2025 1 commit
- [Bugfix] Update `--hf-overrides` for `Alibaba-NLP/gte-Qwen2` (#14609) · af295e9b
  Cyrus Leung authored Mar 11, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  af295e9b
06 Mar, 2025 1 commit

[Model] Update Paligemma multimodal processing with PromptUpdate (#14015) · 17699280

kYLe authored Mar 06, 2025


Signed-off-by: Kyle Huang <kylhuang@nvidia.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

17699280

05 Mar, 2025 1 commit
- [Model] New model support for Phi-4-multimodal-instruct (#14119) · 0a995d54
  Congcong Chen authored Mar 04, 2025
  
  0a995d54
04 Mar, 2025 1 commit

[Model] Add support for GraniteMoeShared models (#13313) · c060b714

Travis Johnson authored Mar 03, 2025


Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

c060b714

03 Mar, 2025 1 commit
- Update deprecated Python 3.8 typing (#13971) · cf069aa8
  Harry Mellor authored Mar 03, 2025
  
  cf069aa8
28 Feb, 2025 2 commits
- Use smaller embedding model when not testing model specifically (#13891) · 76c89fca
  Harry Mellor authored Feb 28, 2025
  
  76c89fca
- [Bugfix] Check that number of images matches number of <|image|> tokens with mllama (#13911) · 73e0225e
  Travis Johnson authored Feb 27, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  73e0225e
27 Feb, 2025 1 commit
- [VLM] Support multimodal inputs for Florence-2 models (#13320) · edf309eb
  Isotr0py authored Feb 27, 2025
  
  edf309eb
26 Feb, 2025 3 commits
- [Bugfix] Add test example for Ultravox v0.5 (#13890) · 7b700ec8
  Cyrus Leung authored Feb 26, 2025
  
  7b700ec8
- [Misc] Fix input processing for Ultravox (#13871) · 7ca1da02
  Roger Wang authored Feb 25, 2025
  
  7ca1da02
- [Model] Support Grok1 (#13795) · 07c43530
  Michael Goin authored Feb 25, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  07c43530
23 Feb, 2025 2 commits
- [LMM] Implement merged multimodal processor for whisper (#13278) · ba5106e5
  Isotr0py authored Feb 23, 2025
  
  ba5106e5
- [ci] Use env var to control whether to use S3 bucket in CI (#13634) · 2c5e637b
  Kevin H. Luu authored Feb 22, 2025
  
  2c5e637b
19 Feb, 2025 4 commits
- [VLM][Bugfix] Pass processor kwargs properly on init (#13516) · 377d10bd
  Cyrus Leung authored Feb 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  377d10bd
- [Model][Speculative Decoding] DeepSeek MTP spec decode (#12755) · f525c0be
  Lucia Fang authored Feb 19, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  f525c0be
- [Bugfix] Fix Positive Feature Layers in Llava Models (#13514) · 983a40a8
  Alex Brooks authored Feb 19, 2025
```
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
```
  983a40a8
- [1/n][CI] Load models in CI from S3 instead of HF (#13205) · d5d214ac
  Kevin H. Luu authored Feb 18, 2025
```
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
```
  d5d214ac