Commits · 1a11f127528e49343706b675bfb56d96319a4cbc · OpenDAS / vllm_cscc

26 Dec, 2024 1 commit
- [Doc] Add video example to openai client for multimodal (#11521) · b85a9778
  Isotr0py authored Dec 27, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  b85a9778
25 Dec, 2024 1 commit
- [Model] Add Qwen2-Audio model support · 3c23ce2d
  zhuwenwen authored Dec 25, 2024
  
  3c23ce2d
24 Dec, 2024 1 commit
- [Frontend] Online Pooling API (#11457) · 9edca6bf
  Cyrus Leung authored Dec 24, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9edca6bf
23 Dec, 2024 1 commit
- 1.解决qwen lora模型推理结果异常问题 · da39222e
  王敏 authored Dec 23, 2024
```
2.修复medusa推理时内存访问越界问题
3.benchmark_moe添加deepseek_v2模型
```
  da39222e
19 Dec, 2024 3 commits

[Model] Refactor Qwen2-VL to use merged multimodal processor (#11258) · e24113a8

Isotr0py authored Dec 20, 2024


Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

e24113a8

[Feature] Add load generation config from model (#11164) · 5aef4980

Yanyi Liu authored Dec 19, 2024


Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

5aef4980

[VLM] Merged multimodal processor for Qwen2-Audio (#11303) · 6142ef0a
Cyrus Leung authored Dec 19, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
6142ef0a

18 Dec, 2024 1 commit
- [V1] VLM - enable processor cache by default (#11305) · fdea8ec1
  Alexander Matveev authored Dec 18, 2024
```
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
```
  fdea8ec1
17 Dec, 2024 1 commit

[Frontend] Add OpenAI API support for input_audio (#11027) · 66d4b167

kYLe authored Dec 17, 2024


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

66d4b167

16 Dec, 2024 3 commits
- [misc] Layerwise profile updates (#10242) · efbce85f
  Varun Sundar Rabindranath authored Dec 16, 2024
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  efbce85f
- [Doc] Reorder vision language examples in alphabet order (#11228) · 2ca830db
  Isotr0py authored Dec 16, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  2ca830db
- [Model] Refactor Ultravox to use merged input processor (#11198) · d927dbcd
  Isotr0py authored Dec 16, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d927dbcd
15 Dec, 2024 1 commit
- [Misc] Clean up multi-modal processor (#11207) · b10609e6
  Cyrus Leung authored Dec 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b10609e6
14 Dec, 2024 1 commit
- [VLM] Fully dynamic prompt replacement in merged input processor (#11199) · 93abf23a
  Cyrus Leung authored Dec 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  93abf23a
13 Dec, 2024 4 commits
- [Doc] Reorganize online pooling APIs (#11172) · 0920ab91
  Cyrus Leung authored Dec 14, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0920ab91
- [Frontend] Separate pooling APIs in offline inference (#11129) · eeec9e33
  Cyrus Leung authored Dec 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  eeec9e33
- update offline_streaming_inference_chat_demo.py · 988eb4e6
  zhuwenwen authored Dec 13, 2024
  
  988eb4e6
- PaliGemma 2 support (#11142) · 7cd74091
  Jani Monoses authored Dec 13, 2024
  
  7cd74091
12 Dec, 2024 2 commits

[V1] Fix torch profiling for offline inference (#11125) · 4816d20a
Roger Wang authored Dec 12, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
4816d20a

[V1] VLM preprocessor hashing (#11020) · 4e116833

Alexander Matveev authored Dec 11, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

4e116833

11 Dec, 2024 3 commits
- [Misc] Split up pooling tasks (#10820) · 8f10d5e3
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f10d5e3
- 更新medusa权重转换脚本 · 0bb491f8
  王敏 authored Dec 11, 2024
  
  0bb491f8
- [feat]优化medusa代码，通过VLLM_TREE_DECODING环境变量控制是否采用tree-style解码，计算逻辑主干隔离 · 44e3ca68
  王敏 authored Dec 11, 2024
  
  44e3ca68
10 Dec, 2024 2 commits
- Add example of helm chart for vllm deployment on k8s (#9199) · fe2e10c7
  Maxime Fournioux authored Dec 10, 2024
```
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
```
  fe2e10c7
- remove sugon · f8382145
  zhuwenwen authored Dec 10, 2024
  
  f8382145
07 Dec, 2024 2 commits
- [Model] Update multi-modal processor to support Mantis(LLaVA) model (#10711) · 39e227c7
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  39e227c7
- [Doc] Explicitly state that InternVL 2.5 is supported (#10978) · 1c768fe5
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1c768fe5
06 Dec, 2024 1 commit
- Update offline_streaming_inference_chat_demo.py · 28408cdf
  xuxz authored Dec 06, 2024
  
  28408cdf
05 Dec, 2024 3 commits
- fix · 8465317a
  xuxzh1 authored Dec 05, 2024
  
  8465317a
- [Misc] Update llama 3.2 template to support system prompt with images (#10901) · 39c89e71
  Travis Johnson authored Dec 04, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  39c89e71
- change AMD GPU and dcu to hcu · 3f78216a
  zhuwenwen authored Dec 05, 2024
  
  3f78216a
04 Dec, 2024 1 commit
- update offline_inference.py · 4646fe24
  zhuwenwen authored Dec 04, 2024
  
  4646fe24
02 Dec, 2024 1 commit

[Core] Implement disagg prefill by StatelessProcessGroup (#10502) · 0590ec3f

Kuntai Du authored Dec 01, 2024



This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: YaoJiayi <120040070@link.cuhk.edu.cn>

0590ec3f

01 Dec, 2024 1 commit
- [Misc] Rename embedding classes to pooling (#10801) · d2f058e7
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d2f058e7
27 Nov, 2024 1 commit
- 1.优化medusa推理，节省cpu耗时 · 28375803
  王敏 authored Nov 27, 2024
```
2.更新medusa readme
3.解决benchmark_moe报错问题
```
  28375803
26 Nov, 2024 1 commit

[Feature] vLLM ARM Enablement for AARCH64 CPUs (#9228) · a6760f64

Sanket Kale authored Nov 26, 2024


Signed-off-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: mgoin <michael@neuralmagic.com>

a6760f64

25 Nov, 2024 2 commits

[Model]: Add support for Aria model (#10514) · b1d92053

zhou fan authored Nov 26, 2024


Signed-off-by: xffxff <1247714429@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

b1d92053

Support Cross encoder models (#10400) · 214efc2c

Maximilien de Bayser authored Nov 24, 2024


Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>

214efc2c

23 Nov, 2024 2 commits
- [Core] Fix broken log configuration (#10458) · ebda5196
  Russell Bryant authored Nov 22, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  ebda5196
- [Bugfix][Frontend] Update Llama Chat Templates to also support Non-Tool use (#10164) · 9195dbdb
  Travis Johnson authored Nov 22, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  9195dbdb