Commits · d93bf4da855a0c5e8d3c875def6b37c5e9d77763 · OpenDAS / vllm_cscc

29 Jan, 2025 3 commits

[Model] Refactoring of MiniCPM-V and add MiniCPM-o-2.6 support for vLLM (#12069) · d93bf4da

Alphi authored Jan 29, 2025


Signed-off-by: hzh <hezhihui_thu@163.com>
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Signed-off-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Oleg Mosalov <oleg@krai.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>
Signed-off-by: Chenguang Li <757486878@qq.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Shanshan Shen <467638484@qq.com>
Signed-off-by: elijah <f1renze.142857@gmail.com>
Signed-off-by: Yikun <yikunkero@gmail.com>
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Co-authored-by: shaochangxu <85155497+shaochangxu@users.noreply.github.com>
Co-authored-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: sixgod <evethwillbeok@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Rafael Vasquez <rafvasq21@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Akshat Tripathi <Akshat.tripathi6568@gmail.com>
Co-authored-by: Oleg Mosalov <oleg@krai.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Avshalom Manevich <12231371+avshalomman@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Yangcheng Li <liyangcheng.lyc@alibaba-inc.com>
Co-authored-by: Siyuan Li <94890248+liaoyanqing666@users.noreply.github.com>
Co-authored-by: Concurrensee <yida.wu@amd.com>
Co-authored-by: Chenguang Li <757486878@qq.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Alex Brooks <alex.brooks@ibm.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
Co-authored-by: elijah <30852919+e1ijah1@users.noreply.github.com>
Co-authored-by: Yikun Jiang <yikunkero@gmail.com>
Co-authored-by: Steve Luo <36296769+SunflowerAries@users.noreply.github.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: TJian <tunjian1996@gmail.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: maang-h <55082429+maang-h@users.noreply.github.com>
Co-authored-by: Elfie Guo <164945471+elfiegg@users.noreply.github.com>
Co-authored-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

d93bf4da

[Frontend] Support reasoning content for deepseek r1 (#12473) · a7e3eba6

Ce Gao authored Jan 29, 2025


Signed-off-by: Ce Gao <cegao@tensorchord.ai>
Co-authored-by: Rafael Vasquez <rafvasq21@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>

a7e3eba6

[TPU] Add example for profiling TPU inference (#12531) · fbb5bd4c
Michael Goin authored Jan 28, 2025
```
Signed-off-by: mgoin <mgoin@redhat.com>
```
fbb5bd4c

28 Jan, 2025 1 commit
- Fix bad path in prometheus example (#12481) · 426a5c36
  Michael Goin authored Jan 27, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  426a5c36
27 Jan, 2025 2 commits
- [Frontend] Support scores endpoint in run_batch (#12430) · 0cc6b383
  Pooya Davoodi authored Jan 26, 2025
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  0cc6b383
- [Frontend] Rerank API (Jina- and Cohere-compatible API) (#12376) · 0034b09c
  Kyle Mistele authored Jan 26, 2025
```
Signed-off-by: Kyle Mistele <kyle@mistele.com>
```
  0034b09c
23 Jan, 2025 1 commit

[FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) · e97f802b

Gregory Shtrasberg authored Jan 23, 2025


Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>

e97f802b

22 Jan, 2025 1 commit
- [Model][Bugfix]: correct Aria model output (#12309) · 528dbcac
  zhou fan authored Jan 22, 2025
```
Signed-off-by: xffxff <1247714429@qq.com>
```
  528dbcac
20 Jan, 2025 2 commits
- [core][bugfix] configure env var during import vllm (#12209) · c222f479
  youkaichao authored Jan 20, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c222f479
- [Model] Upgrade Aria to transformers 4.48 (#12203) · b37d8279
  Cyrus Leung authored Jan 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b37d8279
18 Jan, 2025 1 commit
- [Model] Port deepseek-vl2 processor, remove dependency (#12169) · 02798eca
  Isotr0py authored Jan 18, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  02798eca
17 Jan, 2025 2 commits
- [Misc] Add deepseek_vl2 chat template (#12143) · d75ab55f
  Isotr0py authored Jan 17, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  d75ab55f
- [Bugfix] Fix a path bug in disaggregated prefill example script. (#12121) · ebc73f28
  Kuntai Du authored Jan 17, 2025
```
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
```
  ebc73f28
16 Jan, 2025 4 commits
- [Bugfix] Set enforce_eager automatically for mllama (#12127) · d06e8240
  Chen Zhang authored Jan 17, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  d06e8240
- [Model] Add support for deepseek-vl2-tiny model (#12068) · 62b06ba2
  Isotr0py authored Jan 17, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  62b06ba2
- [core] LLM.collective_rpc interface and RLHF example (#12084) · 92e793d9
  youkaichao authored Jan 16, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  92e793d9
- Support torchrun and SPMD-style offline inference (#12071) · bf53e0c7
  youkaichao authored Jan 16, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  bf53e0c7
13 Jan, 2025 1 commit
- [Model] Support GGUF models newly added in `transformers` 4.46.0 (#9685) · d14e98d9
  Isotr0py authored Jan 13, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d14e98d9
12 Jan, 2025 1 commit
- [Model] Initialize support for Deepseek-VL2 models (#11578) · f967e51f
  Isotr0py authored Jan 12, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  f967e51f
10 Jan, 2025 2 commits
- [Doc] Rename offline inference examples (#11927) · 482cdc49
  Harry Mellor authored Jan 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  482cdc49
- Replace "online inference" with "online serving" (#11923) · d85c47d6
  Harry Mellor authored Jan 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d85c47d6
09 Jan, 2025 1 commit
- [Misc] Provide correct Pixtral-HF chat template (#11891) · 9a228348
  Cyrus Leung authored Jan 10, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9a228348
08 Jan, 2025 3 commits
- [Doc] Move examples into categories (#11840) · aba8d6ee
  Harry Mellor authored Jan 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  aba8d6ee
- [Bugfix] Fix image input for Pixtral-HF (#11741) · 91445c7b
  Cyrus Leung authored Jan 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  91445c7b
- [Doc] Group examples into categories (#11782) · 5950f555
  Harry Mellor authored Jan 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  5950f555
07 Jan, 2025 1 commit
- [CI] Fix neuron CI and run offline tests (#11779) · 898cdf03
  Liangfu Chen authored Jan 06, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
  898cdf03
03 Jan, 2025 1 commit
- [Model] Whisper model implementation (#11280) · e1a5c2f0
  Aurick Qiao authored Jan 03, 2025
```
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
```
  e1a5c2f0
02 Jan, 2025 1 commit
- [Misc] Minimum requirements for SageMaker compatibility (#11576) · 68d37809
  Nathan Azrak authored Jan 03, 2025
  
  68d37809
31 Dec, 2024 1 commit

[V1][VLM] V1 support for selected single-image models. (#11632) · e7c7c5e8

Roger Wang authored Dec 31, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>

e7c7c5e8

28 Dec, 2024 1 commit
- [Doc] Update mllama example based on official doc (#11567) · a6073124
  Chen Zhang authored Dec 28, 2024
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  a6073124
26 Dec, 2024 1 commit
- [Doc] Add video example to openai client for multimodal (#11521) · b85a9778
  Isotr0py authored Dec 27, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  b85a9778
24 Dec, 2024 1 commit
- [Frontend] Online Pooling API (#11457) · 9edca6bf
  Cyrus Leung authored Dec 24, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9edca6bf
19 Dec, 2024 3 commits

[Model] Refactor Qwen2-VL to use merged multimodal processor (#11258) · e24113a8

Isotr0py authored Dec 20, 2024


Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

e24113a8

[Feature] Add load generation config from model (#11164) · 5aef4980

Yanyi Liu authored Dec 19, 2024


Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

5aef4980

[VLM] Merged multimodal processor for Qwen2-Audio (#11303) · 6142ef0a
Cyrus Leung authored Dec 19, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
6142ef0a

18 Dec, 2024 1 commit
- [V1] VLM - enable processor cache by default (#11305) · fdea8ec1
  Alexander Matveev authored Dec 18, 2024
```
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
```
  fdea8ec1
17 Dec, 2024 1 commit

[Frontend] Add OpenAI API support for input_audio (#11027) · 66d4b167

kYLe authored Dec 17, 2024


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

66d4b167

16 Dec, 2024 3 commits
- [misc] Layerwise profile updates (#10242) · efbce85f
  Varun Sundar Rabindranath authored Dec 16, 2024
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  efbce85f
- [Doc] Reorder vision language examples in alphabet order (#11228) · 2ca830db
  Isotr0py authored Dec 16, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  2ca830db
- [Model] Refactor Ultravox to use merged input processor (#11198) · d927dbcd
  Isotr0py authored Dec 16, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d927dbcd