Commits · 28e0750847ded93158a66efdcbc869d87463b38f · OpenDAS / vllm_cscc

27 Jan, 2025 1 commit
- [Frontend] Rerank API (Jina- and Cohere-compatible API) (#12376) · 0034b09c
  Kyle Mistele authored Jan 26, 2025
```
Signed-off-by: Kyle Mistele <kyle@mistele.com>
```
  0034b09c
23 Jan, 2025 1 commit

[FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) · e97f802b

Gregory Shtrasberg authored Jan 23, 2025


Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>

e97f802b

22 Jan, 2025 1 commit
- [Model][Bugfix]: correct Aria model output (#12309) · 528dbcac
  zhou fan authored Jan 22, 2025
```
Signed-off-by: xffxff <1247714429@qq.com>
```
  528dbcac
20 Jan, 2025 2 commits
- [core][bugfix] configure env var during import vllm (#12209) · c222f479
  youkaichao authored Jan 20, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c222f479
- [Model] Upgrade Aria to transformers 4.48 (#12203) · b37d8279
  Cyrus Leung authored Jan 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b37d8279
18 Jan, 2025 1 commit
- [Model] Port deepseek-vl2 processor, remove dependency (#12169) · 02798eca
  Isotr0py authored Jan 18, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  02798eca
17 Jan, 2025 2 commits
- [Misc] Add deepseek_vl2 chat template (#12143) · d75ab55f
  Isotr0py authored Jan 17, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  d75ab55f
- [Bugfix] Fix a path bug in disaggregated prefill example script. (#12121) · ebc73f28
  Kuntai Du authored Jan 17, 2025
```
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
```
  ebc73f28
16 Jan, 2025 4 commits
- [Bugfix] Set enforce_eager automatically for mllama (#12127) · d06e8240
  Chen Zhang authored Jan 17, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  d06e8240
- [Model] Add support for deepseek-vl2-tiny model (#12068) · 62b06ba2
  Isotr0py authored Jan 17, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  62b06ba2
- [core] LLM.collective_rpc interface and RLHF example (#12084) · 92e793d9
  youkaichao authored Jan 16, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  92e793d9
- Support torchrun and SPMD-style offline inference (#12071) · bf53e0c7
  youkaichao authored Jan 16, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  bf53e0c7
13 Jan, 2025 1 commit
- [Model] Support GGUF models newly added in `transformers` 4.46.0 (#9685) · d14e98d9
  Isotr0py authored Jan 13, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d14e98d9
12 Jan, 2025 1 commit
- [Model] Initialize support for Deepseek-VL2 models (#11578) · f967e51f
  Isotr0py authored Jan 12, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  f967e51f
10 Jan, 2025 2 commits
- [Doc] Rename offline inference examples (#11927) · 482cdc49
  Harry Mellor authored Jan 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  482cdc49
- Replace "online inference" with "online serving" (#11923) · d85c47d6
  Harry Mellor authored Jan 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d85c47d6
09 Jan, 2025 1 commit
- [Misc] Provide correct Pixtral-HF chat template (#11891) · 9a228348
  Cyrus Leung authored Jan 10, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9a228348
08 Jan, 2025 3 commits
- [Doc] Move examples into categories (#11840) · aba8d6ee
  Harry Mellor authored Jan 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  aba8d6ee
- [Bugfix] Fix image input for Pixtral-HF (#11741) · 91445c7b
  Cyrus Leung authored Jan 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  91445c7b
- [Doc] Group examples into categories (#11782) · 5950f555
  Harry Mellor authored Jan 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  5950f555
07 Jan, 2025 1 commit
- [CI] Fix neuron CI and run offline tests (#11779) · 898cdf03
  Liangfu Chen authored Jan 06, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
  898cdf03
03 Jan, 2025 1 commit
- [Model] Whisper model implementation (#11280) · e1a5c2f0
  Aurick Qiao authored Jan 03, 2025
```
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
```
  e1a5c2f0
02 Jan, 2025 1 commit
- [Misc] Minimum requirements for SageMaker compatibility (#11576) · 68d37809
  Nathan Azrak authored Jan 03, 2025
  
  68d37809
31 Dec, 2024 1 commit

[V1][VLM] V1 support for selected single-image models. (#11632) · e7c7c5e8

Roger Wang authored Dec 31, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>

e7c7c5e8

28 Dec, 2024 1 commit
- [Doc] Update mllama example based on official doc (#11567) · a6073124
  Chen Zhang authored Dec 28, 2024
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  a6073124
26 Dec, 2024 1 commit
- [Doc] Add video example to openai client for multimodal (#11521) · b85a9778
  Isotr0py authored Dec 27, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  b85a9778
24 Dec, 2024 1 commit
- [Frontend] Online Pooling API (#11457) · 9edca6bf
  Cyrus Leung authored Dec 24, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9edca6bf
19 Dec, 2024 3 commits

[Model] Refactor Qwen2-VL to use merged multimodal processor (#11258) · e24113a8

Isotr0py authored Dec 20, 2024


Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

e24113a8

[Feature] Add load generation config from model (#11164) · 5aef4980

Yanyi Liu authored Dec 19, 2024


Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

5aef4980

[VLM] Merged multimodal processor for Qwen2-Audio (#11303) · 6142ef0a
Cyrus Leung authored Dec 19, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
6142ef0a

18 Dec, 2024 1 commit
- [V1] VLM - enable processor cache by default (#11305) · fdea8ec1
  Alexander Matveev authored Dec 18, 2024
```
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
```
  fdea8ec1
17 Dec, 2024 1 commit

[Frontend] Add OpenAI API support for input_audio (#11027) · 66d4b167

kYLe authored Dec 17, 2024


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

66d4b167

16 Dec, 2024 3 commits
- [misc] Layerwise profile updates (#10242) · efbce85f
  Varun Sundar Rabindranath authored Dec 16, 2024
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  efbce85f
- [Doc] Reorder vision language examples in alphabet order (#11228) · 2ca830db
  Isotr0py authored Dec 16, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  2ca830db
- [Model] Refactor Ultravox to use merged input processor (#11198) · d927dbcd
  Isotr0py authored Dec 16, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d927dbcd
15 Dec, 2024 1 commit
- [Misc] Clean up multi-modal processor (#11207) · b10609e6
  Cyrus Leung authored Dec 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b10609e6
14 Dec, 2024 1 commit
- [VLM] Fully dynamic prompt replacement in merged input processor (#11199) · 93abf23a
  Cyrus Leung authored Dec 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  93abf23a
13 Dec, 2024 3 commits
- [Doc] Reorganize online pooling APIs (#11172) · 0920ab91
  Cyrus Leung authored Dec 14, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0920ab91
- [Frontend] Separate pooling APIs in offline inference (#11129) · eeec9e33
  Cyrus Leung authored Dec 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  eeec9e33
- PaliGemma 2 support (#11142) · 7cd74091
  Jani Monoses authored Dec 13, 2024
  
  7cd74091