Commits · b12e87f942eb7740c17ab546b964bc327afdda37 · OpenDAS / vllm_cscc

30 Dec, 2024 2 commits
- [platforms] enable platform plugins (#11602) · b12e87f9
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  b12e87f9
- [v1] fix compilation cache (#11598) · 3682e33f
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3682e33f
28 Dec, 2024 4 commits
- [V1] [5/N] API Server: unify `Detokenizer` and `EngineCore` input (#11545) · 4fb8e329
  Robert Shaw authored Dec 28, 2024
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  4fb8e329
- [bugfix] interleaving sliding window for cohere2 model (#11583) · 328841d0
  youkaichao authored Dec 29, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  328841d0
- [Model] Support InternLM2 Reward models (#11571) · d34be24b
  Isotr0py authored Dec 28, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d34be24b
- [V1] [4/N] API Server: ZMQ/MP Utilities (#11541) · df04dffa
  Robert Shaw authored Dec 27, 2024
  
  df04dffa
27 Dec, 2024 5 commits
- [MODEL] LoRA support for Jamba model (#11209) · 55509c21
  ErezSC42 authored Dec 27, 2024
```
Signed-off-by: Erez Schwartz <erezs@ai21.com>
```
  55509c21
- [VLM] Support caching in merged multi-modal processor (#11396) · 10141809
  Cyrus Leung authored Dec 28, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  10141809
- [Misc] Abstract the logic for reading and writing media content (#11527) · 7af553ea
  Cyrus Leung authored Dec 27, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7af553ea
- [CI] Fix broken CI (#11543) · 46d43594
  Robert Shaw authored Dec 26, 2024
  
  46d43594
- [V1] Use FlashInfer Sampling Kernel for Top-P & Top-K Sampling (#11394) · 371d04d3
  Woosuk Kwon authored Dec 27, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  371d04d3
26 Dec, 2024 5 commits

[Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (#11523) · 2072924d

Michael Goin authored Dec 26, 2024


Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: HandH1998 <1335248067@qq.com>

2072924d

[Misc] Add placeholder module (#11501) · eec906d8
Cyrus Leung authored Dec 26, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
eec906d8

[V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (#10681) · dcb1a944

sroy745 authored Dec 26, 2024


Signed-off-by: Sourashis Roy <sroy@roblox.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

dcb1a944

[Misc][LoRA] Fix LoRA weight mapper (#11495) · aa25985b
Jee Jee Li authored Dec 26, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
aa25985b
[Misc] Move some multimodal utils to modality-specific modules (#11494) · 51a624bf
Cyrus Leung authored Dec 26, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
51a624bf

25 Dec, 2024 2 commits
- [Misc] Update disaggregation benchmark scripts and test logs (#11456) · fc601665
  Jiaxin Shan authored Dec 24, 2024
```
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
```
  fc601665
- [V1] Unify VLLM_ENABLE_V1_MULTIPROCESSING handling in RayExecutor (#11472) · 9832e557
  Rui Qiao authored Dec 24, 2024
  
  9832e557
24 Dec, 2024 4 commits
- [Model] Automatic conversion of classification and reward models (#11469) · 3f3e92e1
  Cyrus Leung authored Dec 25, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3f3e92e1
- [Misc] Move weights mapper (#11443) · 196c34b0
  Jee Jee Li authored Dec 24, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  196c34b0
- [Bugfix] Fix Qwen2-VL LoRA weight loading (#11430) · b1b1038f
  Jee Jee Li authored Dec 24, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  b1b1038f
- [Frontend] Online Pooling API (#11457) · 9edca6bf
  Cyrus Leung authored Dec 24, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9edca6bf
23 Dec, 2024 4 commits
- [V1] TP Ray executor (#11107) · a491d6f5
  Rui Qiao authored Dec 23, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  a491d6f5
- [CI] Expand OpenAI test_chat.py guided decoding tests (#11048) · 63afbe92
  Michael Goin authored Dec 23, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  63afbe92
- [Misc] add w8a8 asym models (#11075) · 8cef6e02
  Dipika Sikka authored Dec 23, 2024
  
  8cef6e02
- [Bugfix] Fix CFGGuide and use outlines for grammars that can't convert to GBNF (#11389) · 5bfb30a5
  Michael Goin authored Dec 23, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  5bfb30a5
22 Dec, 2024 2 commits
- [Bugfix] Fix fully sharded LoRAs with Mixtral (#11390) · f1d1bf62
  Jason T. Greene authored Dec 22, 2024
```
Signed-off-by: Jason Greene <jason.greene@redhat.com>
```
  f1d1bf62
- [CI] Fix flaky entrypoint tests (#11403) · 29c74893
  Roger Wang authored Dec 21, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  29c74893
20 Dec, 2024 2 commits
- [Core] Loading model from S3 using RunAI Model Streamer as optional loader (#10192) · 995f5623
  omer-dayan authored Dec 20, 2024
```
Signed-off-by: OmerD <omer@run.ai>
```
  995f5623
- [Bugfix] Fix spec decoding when seed is none in a batch (#10863) · 86c2d8fd
  Wallas Henrique authored Dec 20, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  86c2d8fd
19 Dec, 2024 7 commits

[Model] Refactor Qwen2-VL to use merged multimodal processor (#11258) · e24113a8

Isotr0py authored Dec 20, 2024


Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

e24113a8

[Model] Add JambaForSequenceClassification model (#10860) · 6c7f8815

Yehoshua Cohen authored Dec 19, 2024


Signed-off-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

6c7f8815

[Feature] Add load generation config from model (#11164) · 5aef4980

Yanyi Liu authored Dec 19, 2024


Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

5aef4980

[VLM] Merged multimodal processor for Qwen2-Audio (#11303) · 6142ef0a
Cyrus Leung authored Dec 19, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
6142ef0a
[CI] Expand test_guided_generate to test all backends (#11313) · a30482f0
Michael Goin authored Dec 18, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
a30482f0
[Model] IBM Granite 3.1 (#11307) · 17ca9642
Travis Johnson authored Dec 18, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
17ca9642
[Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (#11311) · 5a9da2e6
Tyler Michael Smith authored Dec 18, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
5a9da2e6

18 Dec, 2024 3 commits

[Bugfix] fix minicpmv test (#11304) · ca5f54a9
Joe Runde authored Dec 18, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
ca5f54a9
[Bugfix] Fix broken phi3-v mm_processor_kwargs tests (#11263) · 996aa70f
Isotr0py authored Dec 19, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
996aa70f

[Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd

Dipika Sikka authored Dec 18, 2024


Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

60508ffd