Commits · b880ffb87e0bcde5e3693203b480df49e46d67bc · OpenDAS / vllm_cscc

"projects/vscode:/vscode.git/clone" did not exist on "ea9336c0c169497a22c6f6f3ae26e67039b08b6d"

19 Dec, 2024 7 commits

[Model] Refactor Qwen2-VL to use merged multimodal processor (#11258) · e24113a8

Isotr0py authored Dec 20, 2024


Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

e24113a8

[Model] Add JambaForSequenceClassification model (#10860) · 6c7f8815

Yehoshua Cohen authored Dec 19, 2024


Signed-off-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

6c7f8815

[Feature] Add load generation config from model (#11164) · 5aef4980

Yanyi Liu authored Dec 19, 2024


Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

5aef4980

[VLM] Merged multimodal processor for Qwen2-Audio (#11303) · 6142ef0a
Cyrus Leung authored Dec 19, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
6142ef0a
[CI] Expand test_guided_generate to test all backends (#11313) · a30482f0
Michael Goin authored Dec 18, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
a30482f0
[Model] IBM Granite 3.1 (#11307) · 17ca9642
Travis Johnson authored Dec 18, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
17ca9642
[Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (#11311) · 5a9da2e6
Tyler Michael Smith authored Dec 18, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
5a9da2e6

18 Dec, 2024 6 commits
- [Bugfix] fix minicpmv test (#11304) · ca5f54a9
  Joe Runde authored Dec 18, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  ca5f54a9
- [Bugfix] Fix broken phi3-v mm_processor_kwargs tests (#11263) · 996aa70f
  Isotr0py authored Dec 19, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  996aa70f
- [Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd
  Dipika Sikka authored Dec 18, 2024
```
Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  60508ffd
- [Bugfix] Fix guided decoding with tokenizer mode mistral (#11046) · 8b79f9e1
  Wallas Henrique authored Dec 18, 2024
  
  8b79f9e1
- [V1] Prefix caching for vision language models (#11187) · bf8717eb
  Cody Yu authored Dec 17, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  bf8717eb
- [Bugfix] Set temperature=0.7 in test_guided_choice_chat (#11264) · c77eb8a3
  Michael Goin authored Dec 17, 2024
  
  c77eb8a3
17 Dec, 2024 3 commits
- [Bugfix] Fix request cancellation without polling (#11190) · 2d1b9baa
  Joe Runde authored Dec 17, 2024
  
  2d1b9baa
- [Frontend] Add OpenAI API support for input_audio (#11027) · 66d4b167
  kYLe authored Dec 17, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  66d4b167
- [CI] Add test case with JSON schema using references + use xgrammar by default... · 0064f697
  Michael Goin authored Dec 16, 2024
```
[CI] Add test case with JSON schema using references + use xgrammar by default with OpenAI parse (#10935)
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  0064f697
16 Dec, 2024 3 commits
- [core] overhaul memory profiling and fix backward compatibility (#10511) · 551603fe
  youkaichao authored Dec 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  551603fe
- [Model] Refactor Ultravox to use merged input processor (#11198) · d927dbcd
  Isotr0py authored Dec 16, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d927dbcd
- [Model] Support Cohere2ForCausalLM (Cohere R7B) (#11203) · bddbbcb1
  Jani Monoses authored Dec 16, 2024
  
  bddbbcb1
15 Dec, 2024 1 commit
- [Misc] Clean up multi-modal processor (#11207) · b10609e6
  Cyrus Leung authored Dec 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b10609e6
14 Dec, 2024 2 commits
- [VLM] Fully dynamic prompt replacement in merged input processor (#11199) · 93abf23a
  Cyrus Leung authored Dec 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  93abf23a
- [Frontend] Add `logits_processors` as an extra completion argument (#11150) · 9c3dadd1
  Brad Hilton authored Dec 14, 2024
```
Signed-off-by: Brad Hilton <brad.hilton.nw@gmail.com>
```
  9c3dadd1
13 Dec, 2024 7 commits
- [Doc] Reorganize online pooling APIs (#11172) · 0920ab91
  Cyrus Leung authored Dec 14, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0920ab91
- [Core] support LoRA and prompt adapter in content-based hashing for Block... · c31d4a57
  Sungjae Lee authored Dec 14, 2024
```
[Core] support LoRA and prompt adapter in content-based hashing for Block Manager v2 prefix caching (#8240)
```
  c31d4a57
- [Frontend] Separate pooling APIs in offline inference (#11129) · eeec9e33
  Cyrus Leung authored Dec 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  eeec9e33
- [core] clean up cudagraph batchsize padding logic (#10996) · be39e3cd
  youkaichao authored Dec 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  be39e3cd
- [Bugfix] Use runner_type instead of task in GritLM (#11144) · 1efce686
  Pooya Davoodi authored Dec 12, 2024
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  1efce686
- [torch.compile] Dynamic fp8 + rms_norm fusion (#10906) · 30870b4f
  Luka Govedič authored Dec 12, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  30870b4f
- [Misc][V1] Fix type in v1 prefix caching (#11151) · 78ed8f57
  Cody Yu authored Dec 12, 2024
  
  78ed8f57
12 Dec, 2024 4 commits

[Misc][LoRA] Ensure Lora Adapter requests return adapter name (#11094) · 85362f02

Jiaxin Shan authored Dec 12, 2024


Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

85362f02

[core][distributed] initialization from StatelessProcessGroup (#10986) · 62de37a3
youkaichao authored Dec 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
62de37a3
[Model] Add support for embedding model GritLM (#10816) · 1da8f0e1
Pooya Davoodi authored Dec 11, 2024
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
1da8f0e1

[V1] VLM preprocessor hashing (#11020) · 4e116833

Alexander Matveev authored Dec 11, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

4e116833

11 Dec, 2024 5 commits
- [CI/Build] Split up VLM tests (#11083) · d1e21a97
  Cyrus Leung authored Dec 12, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d1e21a97
- [Misc] Split up pooling tasks (#10820) · 8f10d5e3
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f10d5e3
- [CI/Build] Check transformers v4.47 (#10991) · 2e33fe41
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  2e33fe41
- [Model] PP support for Mamba-like models (#10992) · ffa48c91
  Mor Zusman authored Dec 11, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  ffa48c91
- [Misc] LoRA + Chunked Prefill (#9057) · d5c5154f
  Aurick Qiao authored Dec 10, 2024
  
  d5c5154f
10 Dec, 2024 2 commits
- [Misc][LoRA] Add PEFTHelper for LoRA (#11003) · d05f8867
  Jee Jee Li authored Dec 10, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  d05f8867
- monitor metrics of tokens per step using cudagraph batchsizes (#11031) · ebf77806
  youkaichao authored Dec 09, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ebf77806