Commits · ad385667103b7a98fee5a4fc46d2a7745407333e · OpenDAS / vllm_cscc

17 Oct, 2024 2 commits

[Core] Deprecating block manager v1 and make block manager v2 default (#8704) · 81ede99c

Kuntai Du authored Oct 17, 2024

Removing the block manager v1. This is the initial piece of prefix-caching-centric design. In order to achieve prefix-caching-centric design, we need to simplify the code path so that we only use v2 block manager (which has much higher performance on prefix caching).

81ede99c

[misc] CUDA Time Layerwise Profiler (#8337) · 9d30a056

Lucas Wilkinson authored Oct 17, 2024


Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

9d30a056

16 Oct, 2024 2 commits
- [Misc] Consolidate example usage of OpenAI client for multimodal models (#9412) · 59230ef3
  Roger Wang authored Oct 16, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  59230ef3
- [Model] VLM2Vec, the first multimodal embedding model in vLLM (#9303) · 7abba39e
  Cyrus Leung authored Oct 16, 2024
  
  7abba39e
14 Oct, 2024 2 commits
- [Model] Make llama3.2 support multiple and interleaved images (#9095) · f0fe4fe8
  Xiang Xu authored Oct 14, 2024
  
  f0fe4fe8
- [Model] Molmo vLLM Integration (#9016) · dfe43a20
  Reza Salehi authored Oct 14, 2024
```
Co-authored-by: sanghol <sanghol@allenai.org>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  dfe43a20
11 Oct, 2024 2 commits
- [Model] Add GLM-4v support and meet vllm==0.6.2 (#9242) · 6cf1167c
  sixgod authored Oct 12, 2024
  
  6cf1167c
- [Model] support telechat_12B · 6d4b207a
  zhuwenwen authored Oct 11, 2024
  
  6d4b207a
09 Oct, 2024 1 commit
- 添加w8a8 rocblas非融合支持 · 35a8304d
  zhuwenwen authored Oct 09, 2024
  
  35a8304d
08 Oct, 2024 1 commit
- [Core][Frontend] Add Support for Inference Time mm_processor_kwargs (#9131) · a3691b6b
  Alex Brooks authored Oct 08, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  a3691b6b
07 Oct, 2024 2 commits
- [Model] Support NVLM-D and fix QK Norm in InternViT (#9045) · 151ef4ef
  Cyrus Leung authored Oct 07, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  151ef4ef
- [core] remove beam search from the core (#9105) · 18b296fd
  youkaichao authored Oct 06, 2024
  
  18b296fd
04 Oct, 2024 2 commits
- [Misc] Improved prefix cache example (#9077) · 05c531be
  Andy Dai authored Oct 04, 2024
  
  05c531be
- [Frontend][Feature] support tool calling for internlm/internlm2_5-7b-chat model (#8405) · 3dbb215b
  代君 authored Oct 04, 2024
  
  3dbb215b
03 Oct, 2024 1 commit
- [Bugfix] example template should not add parallel_tool_prompt if tools is none (#9007) · 19a4dd09
  Travis Johnson authored Oct 02, 2024
  
  19a4dd09
30 Sep, 2024 1 commit
- [Model] Expose InternVL2 max_dynamic_patch as a mm_processor_kwarg (#8946) · 2ae25f79
  Isotr0py authored Sep 30, 2024
  
  2ae25f79
28 Sep, 2024 1 commit
- [CI/Build] Update models tests & examples (#8874) · e1a3f5e8
  Cyrus Leung authored Sep 29, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  e1a3f5e8
27 Sep, 2024 1 commit
- [Feature] Add support for Llama 3.1 and 3.2 tool use (#8343) · 344cd2b6
  Maximilien de Bayser authored Sep 26, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  344cd2b6
25 Sep, 2024 2 commits

[Model] Add support for the multi-modal Llama 3.2 model (#8811) · 770ec602

Chen Zhang authored Sep 25, 2024


Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

770ec602

[[Misc]Upgrade bitsandbytes to the latest version 0.44.0 (#8768) · 13f9f7a3
Jee Jee Li authored Sep 25, 2024

13f9f7a3

24 Sep, 2024 2 commits

[Frontend] Batch inference for llm.chat() API (#8648) · 2529d09b

Andy authored Sep 24, 2024


Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

2529d09b

[Model] Expose Phi3v num_crops as a mm_processor_kwarg (#8658) · 8ff7ced9

Alex Brooks authored Sep 24, 2024


Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

8ff7ced9

22 Sep, 2024 2 commits

[Model][VLM] Add LLaVA-Onevision model support (#8486) · 5b595327

litianjian authored Sep 23, 2024


Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

5b595327

[Misc] Use NamedTuple in Multi-image example (#8705) · 8ca5051b
Alex Brooks authored Sep 22, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
8ca5051b

17 Sep, 2024 1 commit
- [Model] Add mistral function calling format to all models loaded with "mistral" format (#8515) · a54ed802
  Patrick von Platen authored Sep 17, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  a54ed802
13 Sep, 2024 2 commits
- [bugfix] torch profiler bug for single gpu with GPUExecutor (#8354) · ba775279
  William Lin authored Sep 12, 2024
  
  ba775279
- [Misc] Update Pixtral example (#8431) · 360ddbd3
  Roger Wang authored Sep 12, 2024
  
  360ddbd3
12 Sep, 2024 1 commit

[Model] Support multiple images for qwen-vl (#8247) · c6202dae

Alex Brooks authored Sep 12, 2024


Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

c6202dae

11 Sep, 2024 4 commits
- Pixtral (#8377) · d394787e
  Patrick von Platen authored Sep 11, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d394787e
- [Model][VLM] Add Qwen2-VL model support (#7905) · 3b7fea77
  Yang Fan authored Sep 12, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3b7fea77
- [model] Support for Llava-Next-Video model (#7559) · 6a512a00
  Yangshen⚡Deng authored Sep 11, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  6a512a00
- [Hardware][NV] Add support for ModelOpt static scaling checkpoints. (#6112) · efcf946a
  Pavani Majety authored Sep 10, 2024
  
  efcf946a
07 Sep, 2024 3 commits
- [Model][VLM] Support multi-images inputs for InternVL2 models (#8201) · e8071259
  Isotr0py authored Sep 07, 2024
  
  e8071259
- [Bugfix] Fix Hermes tool call chat template bug (#8256) · 41e95c52
  Kyle Mistele authored Sep 06, 2024
```
Co-authored-by: Kyle Mistele <kyle@constellate.ai>
```
  41e95c52
- [misc] [doc] [frontend] LLM torch profiler support (#7943) · 12dd7158
  William Lin authored Sep 06, 2024
  
  12dd7158
06 Sep, 2024 2 commits
- [Misc] Remove `SqueezeLLM` (#8220) · 23f32229
  Dipika Sikka authored Sep 06, 2024
  
  23f32229
- add template_llama_chat.jinja · b5160479
  zhuwenwen authored Sep 06, 2024
  
  b5160479
05 Sep, 2024 2 commits
- [MODEL] Qwen Multimodal Support (Qwen-VL / Qwen-VL-Chat) (#8029) · 9da25a88
  Alex Brooks authored Sep 05, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9da25a88
- [Doc] Indicate more information about supported modalities (#8181) · 288a9388
  Cyrus Leung authored Sep 05, 2024
  
  288a9388
04 Sep, 2024 1 commit
- [Neuron] Adding support for adding/ overriding neuron configuration a… (#8062) · 008cf886
  Harsha vardhan manoj Bikki authored Sep 04, 2024
```
Co-authored-by: Harsha Bikki <harbikh@amazon.com>
```
  008cf886