Commits · 6ace6fba2ca42b79a948a9b47af00487b5f73868 · OpenDAS / vllm_cscc

11 Nov, 2024 1 commit

[V1] `AsyncLLM` Implementation (#9826) · 6ace6fba

Robert Shaw authored Nov 11, 2024


Signed-off-by: Nick Hill <nickhill@us.ibm.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

6ace6fba

07 Nov, 2024 1 commit

Online video support for VLMs (#10020) · 28b2877d

litianjian authored Nov 08, 2024


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

28b2877d

06 Nov, 2024 1 commit
- [V1] Make v1 more testable (#9888) · d58268c5
  Joe Runde authored Nov 06, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  d58268c5
04 Nov, 2024 2 commits
- [Frontend] Add max_tokens prometheus metric (#9881) · ac04a97a
  tomeras91 authored Nov 05, 2024
```
Signed-off-by: Tomer Asida <tomera@ai21.com>
```
  ac04a97a
- [CI] Basic Integration Test For TPU (#9968) · 1c45f4c3
  Robert Shaw authored Nov 04, 2024
```
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
```
  1c45f4c3
01 Nov, 2024 3 commits
- [Frontend] Use a proper chat template for VLM2Vec (#9912) · ba0d8920
  Cyrus Leung authored Nov 01, 2024
  
  ba0d8920
- [Frontend] Chat-based Embeddings API (#9759) · 06386a64
  Cyrus Leung authored Nov 01, 2024
  
  06386a64
- [Bugfix][Frontend] Reject guided decoding in multistep mode (#9892) · 031a7995
  Joe Runde authored Oct 31, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  031a7995
31 Oct, 2024 1 commit
- [Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens... · abbfb613
  Guillaume Calmettes authored Oct 31, 2024
```
[Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens field for chat completion endpoint (#9837)
```
  abbfb613
29 Oct, 2024 2 commits
- [Bugfix][Frontend] Guard against bad token ids (#9634) · 67bdf8e5
  Joe Runde authored Oct 29, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  67bdf8e5
- [Frontend] re-enable multi-modality input in the new beam search implementation (#9427) · ef7865b4
  Zhong Qishuai authored Oct 29, 2024
```
Signed-off-by: Qishuai Ferdinandzhong@gmail.com
```
  ef7865b4
24 Oct, 2024 1 commit
- [Bugfix]: Make chat content text allow type content (#9358) · 33bab410
  Vinay R Damodaran authored Oct 24, 2024
```
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
```
  33bab410
23 Oct, 2024 1 commit
- [Frontend] Enable Online Multi-image Support for MLlama (#9393) · 150b7790
  Alex Brooks authored Oct 23, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  150b7790
22 Oct, 2024 2 commits
- [CI/Build] Replaced some models on tests for smaller ones (#9570) · c0292211
  Wallas Henrique authored Oct 22, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  c0292211
- [core] move parallel sampling out from vllm core (#9302) · 76a5e132
  youkaichao authored Oct 21, 2024
  
  76a5e132
20 Oct, 2024 1 commit
- [Bugfix] Pass json-schema to GuidedDecodingParams and make test stronger (#9530) · 5b59fe0f
  Chen Zhang authored Oct 19, 2024
  
  5b59fe0f
19 Oct, 2024 2 commits
- [Frontend] Support simpler image input format (#9478) · c5eea3c8
  Yue Zhang authored Oct 18, 2024
  
  c5eea3c8
- [Bugfix] Fix offline mode when using `mistral_common` (#9457) · 337ed766
  sasha0552 authored Oct 19, 2024
  
  337ed766
18 Oct, 2024 3 commits
- [MISC] Consolidate cleanup() and refactor offline_inference_with_prefix.py (#9510) · d11bf435
  Cody Yu authored Oct 18, 2024
  
  d11bf435
- [Model] Add user-configurable task for models that support both generation and embedding (#9424) · 051eaf6d
  Cyrus Leung authored Oct 19, 2024
  
  051eaf6d
- [Bugfix][Core] Use torch.cuda.memory_stats() to profile peak memory usage (#9352) · de4008e2
  Joe Runde authored Oct 17, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  de4008e2
15 Oct, 2024 2 commits
- [Bugfix] Fix vLLM UsageInfo and logprobs None AssertionError with empty token_ids (#9034) · ba309422
  Chang Su authored Oct 15, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  ba309422
- [BugFix] Fix chat API continuous usage stats (#9357) · e9d517f2
  Nick Hill authored Oct 15, 2024
  
  e9d517f2
11 Oct, 2024 1 commit
- [misc] hide best_of from engine (#9261) · cbc2ef55
  youkaichao authored Oct 10, 2024
```
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
```
  cbc2ef55
08 Oct, 2024 3 commits
- [Bugfix] fix OpenAI API server startup with --disable-frontend-multiprocessing (#8537) · 9a94ca4a
  Daniele authored Oct 08, 2024
  
  9a94ca4a
- [Frontend] Add Early Validation For Chat Template / Tool Call Parser (#9151) · 069d3bd8
  Alex Brooks authored Oct 08, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  069d3bd8
- [Frontend] API support for beam search for MQLLMEngine (#9117) · 8c746226
  Brendan Wong authored Oct 07, 2024
  
  8c746226
06 Oct, 2024 1 commit
- [Frontend] API support for beam search (#9087) · 168cab6b
  Brendan Wong authored Oct 05, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  168cab6b
04 Oct, 2024 2 commits
- Adds truncate_prompt_tokens param for embeddings creation (#8999) · 0dcc8cbe
  Flávia Béo authored Oct 04, 2024
```
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
```
  0dcc8cbe
- [Core][VLM] Test registration for OOT multimodal models (#8717) · 26aa325f
  Roger Wang authored Oct 04, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  26aa325f
01 Oct, 2024 1 commit
- [Frontend][Core] Move guided decoding params into sampling params (#8252) · 062c89e7
  Joe Runde authored Sep 30, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  062c89e7
29 Sep, 2024 1 commit
- [Frontend] Added support for HF's new `continue_final_message` parameter (#8942) · 6c9ba48f
  danieljannai21 authored Sep 29, 2024
  
  6c9ba48f
27 Sep, 2024 1 commit
- [Core] rename`PromptInputs` and `inputs` (#8876) · 3b00b9c2
  Cyrus Leung authored Sep 27, 2024
  
  3b00b9c2
26 Sep, 2024 1 commit
- [BugFix] Fix test breakages from transformers 4.45 upgrade (#8829) · 4b377d6f
  Nick Hill authored Sep 27, 2024
  
  4b377d6f
25 Sep, 2024 2 commits
- Revert "rename PromptInputs and inputs with backward compatibility (#8760) (#8810) · 4f1ba084
  Simon Mo authored Sep 25, 2024
  
  4f1ba084
- rename PromptInputs and inputs with backward compatibility (#8760) · 28e1299e
  Cyrus Leung authored Sep 26, 2024
  
  28e1299e
24 Sep, 2024 1 commit

[Frontend] Batch inference for llm.chat() API (#8648) · 2529d09b

Andy authored Sep 24, 2024


Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

2529d09b

23 Sep, 2024 1 commit
- Add output streaming support to multi-step + async while ensuring RequestOutput obj reuse (#8335) · 1a2aef3e
  Alexander Matveev authored Sep 23, 2024
  
  1a2aef3e
20 Sep, 2024 1 commit
- [Core] Support Lora lineage and base model metadata management (#6315) · 260d40b5
  Jiaxin Shan authored Sep 19, 2024
  
  260d40b5
18 Sep, 2024 1 commit

[Core][Bugfix][Perf] Introduce `MQLLMEngine` to avoid `asyncio` OH (#8157) · 7c7714d8

Alexander Matveev authored Sep 18, 2024


Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

7c7714d8