Commits · f877a7d12a0490705e6bea0987c89548d1a015ea · OpenDAS / vllm_cscc

27 Nov, 2024 2 commits
- [Frontend] don't block event loop in tokenization (preprocess) in OpenAI compatible server (#10635) · 395b1c74
  tomeras91 authored Nov 27, 2024
```
Signed-off-by: Tomer Asida <tomera@ai21.com>
```
  395b1c74
- [ci] fix slow tests (#10698) · 308cc5e2
  youkaichao authored Nov 27, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  308cc5e2
26 Nov, 2024 1 commit
- [ci] add vllm_test_utils (#10659) · 334d64d1
  youkaichao authored Nov 26, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  334d64d1
25 Nov, 2024 2 commits

[Bug]: Authorization ignored when root_path is set (#10606) · d04b13a3
Chauncey authored Nov 26, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
d04b13a3

Support Cross encoder models (#10400) · 214efc2c

Maximilien de Bayser authored Nov 24, 2024


Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>

214efc2c

23 Nov, 2024 2 commits
- [Bugfix] Internal Server Error when tool_choice is incorrect. (#10567) · 7d8ffb34
  Varun Vinayak Shenoy authored Nov 22, 2024
```
Signed-off-by: Varun Shenoy <varun.vinayak.shenoy@oracle.com>
```
  7d8ffb34
- [Bugfix][Frontend] Update Llama Chat Templates to also support Non-Tool use (#10164) · 9195dbdb
  Travis Johnson authored Nov 22, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  9195dbdb
21 Nov, 2024 1 commit
- [Bug]: When apply continue_final_message for OpenAI server, the "echo":false is ignored (#10180) · da7e702c
  Chauncey authored Nov 22, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  da7e702c
20 Nov, 2024 1 commit
- [Bugfix]: allow extra fields in requests to openai compatible server (#10463) · c68f7ede
  Guillaume Calmettes authored Nov 20, 2024
```
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  c68f7ede
16 Nov, 2024 1 commit
- [Frontend] Automatic detection of chat content format from AST (#9919) · 32e46e00
  Cyrus Leung authored Nov 16, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  32e46e00
14 Nov, 2024 1 commit
- [Frontend] Pythonic tool parser (#9859) · f67ce05d
  Mike Depinet authored Nov 13, 2024
```
Signed-off-by: Mike Depinet <mike@fixie.ai>
```
  f67ce05d
11 Nov, 2024 1 commit

[V1] `AsyncLLM` Implementation (#9826) · 6ace6fba

Robert Shaw authored Nov 11, 2024


Signed-off-by: Nick Hill <nickhill@us.ibm.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

6ace6fba

07 Nov, 2024 1 commit

Online video support for VLMs (#10020) · 28b2877d

litianjian authored Nov 08, 2024


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

28b2877d

06 Nov, 2024 1 commit
- [V1] Make v1 more testable (#9888) · d58268c5
  Joe Runde authored Nov 06, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  d58268c5
04 Nov, 2024 2 commits
- [Frontend] Add max_tokens prometheus metric (#9881) · ac04a97a
  tomeras91 authored Nov 05, 2024
```
Signed-off-by: Tomer Asida <tomera@ai21.com>
```
  ac04a97a
- [CI] Basic Integration Test For TPU (#9968) · 1c45f4c3
  Robert Shaw authored Nov 04, 2024
```
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
```
  1c45f4c3
01 Nov, 2024 3 commits
- [Frontend] Use a proper chat template for VLM2Vec (#9912) · ba0d8920
  Cyrus Leung authored Nov 01, 2024
  
  ba0d8920
- [Frontend] Chat-based Embeddings API (#9759) · 06386a64
  Cyrus Leung authored Nov 01, 2024
  
  06386a64
- [Bugfix][Frontend] Reject guided decoding in multistep mode (#9892) · 031a7995
  Joe Runde authored Oct 31, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  031a7995
31 Oct, 2024 1 commit
- [Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens... · abbfb613
  Guillaume Calmettes authored Oct 31, 2024
```
[Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens field for chat completion endpoint (#9837)
```
  abbfb613
29 Oct, 2024 2 commits
- [Bugfix][Frontend] Guard against bad token ids (#9634) · 67bdf8e5
  Joe Runde authored Oct 29, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  67bdf8e5
- [Frontend] re-enable multi-modality input in the new beam search implementation (#9427) · ef7865b4
  Zhong Qishuai authored Oct 29, 2024
```
Signed-off-by: Qishuai Ferdinandzhong@gmail.com
```
  ef7865b4
24 Oct, 2024 1 commit
- [Bugfix]: Make chat content text allow type content (#9358) · 33bab410
  Vinay R Damodaran authored Oct 24, 2024
```
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
```
  33bab410
23 Oct, 2024 1 commit
- [Frontend] Enable Online Multi-image Support for MLlama (#9393) · 150b7790
  Alex Brooks authored Oct 23, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  150b7790
22 Oct, 2024 2 commits
- [CI/Build] Replaced some models on tests for smaller ones (#9570) · c0292211
  Wallas Henrique authored Oct 22, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  c0292211
- [core] move parallel sampling out from vllm core (#9302) · 76a5e132
  youkaichao authored Oct 21, 2024
  
  76a5e132
20 Oct, 2024 1 commit
- [Bugfix] Pass json-schema to GuidedDecodingParams and make test stronger (#9530) · 5b59fe0f
  Chen Zhang authored Oct 19, 2024
  
  5b59fe0f
19 Oct, 2024 2 commits
- [Frontend] Support simpler image input format (#9478) · c5eea3c8
  Yue Zhang authored Oct 18, 2024
  
  c5eea3c8
- [Bugfix] Fix offline mode when using `mistral_common` (#9457) · 337ed766
  sasha0552 authored Oct 19, 2024
  
  337ed766
18 Oct, 2024 3 commits
- [MISC] Consolidate cleanup() and refactor offline_inference_with_prefix.py (#9510) · d11bf435
  Cody Yu authored Oct 18, 2024
  
  d11bf435
- [Model] Add user-configurable task for models that support both generation and embedding (#9424) · 051eaf6d
  Cyrus Leung authored Oct 19, 2024
  
  051eaf6d
- [Bugfix][Core] Use torch.cuda.memory_stats() to profile peak memory usage (#9352) · de4008e2
  Joe Runde authored Oct 17, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  de4008e2
15 Oct, 2024 2 commits
- [Bugfix] Fix vLLM UsageInfo and logprobs None AssertionError with empty token_ids (#9034) · ba309422
  Chang Su authored Oct 15, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  ba309422
- [BugFix] Fix chat API continuous usage stats (#9357) · e9d517f2
  Nick Hill authored Oct 15, 2024
  
  e9d517f2
11 Oct, 2024 1 commit
- [misc] hide best_of from engine (#9261) · cbc2ef55
  youkaichao authored Oct 10, 2024
```
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
```
  cbc2ef55
08 Oct, 2024 3 commits
- [Bugfix] fix OpenAI API server startup with --disable-frontend-multiprocessing (#8537) · 9a94ca4a
  Daniele authored Oct 08, 2024
  
  9a94ca4a
- [Frontend] Add Early Validation For Chat Template / Tool Call Parser (#9151) · 069d3bd8
  Alex Brooks authored Oct 08, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  069d3bd8
- [Frontend] API support for beam search for MQLLMEngine (#9117) · 8c746226
  Brendan Wong authored Oct 07, 2024
  
  8c746226
06 Oct, 2024 1 commit
- [Frontend] API support for beam search (#9087) · 168cab6b
  Brendan Wong authored Oct 05, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  168cab6b
04 Oct, 2024 1 commit
- Adds truncate_prompt_tokens param for embeddings creation (#8999) · 0dcc8cbe
  Flávia Béo authored Oct 04, 2024
```
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
```
  0dcc8cbe