Commits · 32e46e000f77499f4dd7c0bed194e33856f2df24 · OpenDAS / vllm_cscc

16 Nov, 2024 1 commit
- [Frontend] Automatic detection of chat content format from AST (#9919) · 32e46e00
  Cyrus Leung authored Nov 16, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  32e46e00
15 Nov, 2024 5 commits
- [Misc] Fix import error in tensorizer tests and cleanup some code (#10349) · b311efd0
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b311efd0
- [Misc] Consolidate pooler config overrides (#10351) · 2ac6d0e7
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  2ac6d0e7
- [Bugfix] Fix unable to load some models (#10312) · 972112d8
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  972112d8
- [Tool parsing] Improve / correct mistral tool parsing (#10333) · 11cd1ae6
  Patrick von Platen authored Nov 15, 2024
  
  11cd1ae6
- [Bugfix] use AF_INET6 for OpenAI Compatible Server with ipv6 (#9583) · 554af922
  Zijin Xiao authored Nov 15, 2024
```
Signed-off-by: xiaozijin <xiaozijin@bytedance.com>
```
  554af922
14 Nov, 2024 2 commits

[BugFix]: properly deserialize `tool_calls` iterator before processing by... · 52b48c1e

Guillaume Calmettes authored Nov 14, 2024


[BugFix]: properly deserialize `tool_calls` iterator before processing by mistral-common when MistralTokenizer is used (#9951)
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>

52b48c1e

[Frontend] Pythonic tool parser (#9859) · f67ce05d
Mike Depinet authored Nov 13, 2024
```
Signed-off-by: Mike Depinet <mike@fixie.ai>
```
f67ce05d

13 Nov, 2024 1 commit
- [1/N] Initial prototype for multi-modal processor (#10044) · 0b8bb86b
  Cyrus Leung authored Nov 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0b8bb86b
12 Nov, 2024 2 commits
- [Frontend] Add per-request number of cached token stats (#10174) · 47db6ec8
  zifeitong authored Nov 12, 2024
  
  47db6ec8
- [BugFix] Do not raise a `ValueError` when `tool_choice` is set to the... · 36c513a0
  Guillaume Calmettes authored Nov 12, 2024
```
[BugFix] Do not raise a `ValueError` when `tool_choice` is set to the supported `none` option and `tools` are not defined. (#10000)
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  36c513a0
11 Nov, 2024 2 commits

[V1] `AsyncLLM` Implementation (#9826) · 6ace6fba

Robert Shaw authored Nov 11, 2024


Signed-off-by: Nick Hill <nickhill@us.ibm.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

6ace6fba

[Doc] Fix typo error in vllm/entrypoints/openai/cli_args.py (#10196) · ad9a78bf
yansh97 authored Nov 11, 2024

ad9a78bf

09 Nov, 2024 3 commits
- [Frontend][Core] Override HF `config.json` via CLI (#5836) · b09895a6
  Krishna Mandal authored Nov 09, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b09895a6
- [Frontend] add `add_request_id` middleware (#9594) · d88bff1b
  cjackal authored Nov 09, 2024
```
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  d88bff1b
- bugfix: fix the bug that stream generate not work (#2756) · 9e372664
  Zhao Yingzhuo authored Nov 09, 2024
  
  9e372664
07 Nov, 2024 4 commits
- Online video support for VLMs (#10020) · 28b2877d
  litianjian authored Nov 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  28b2877d
- [Frontend] Tool calling parser for Granite 3.0 models (#9027) · ae62fd17
  Maximilien de Bayser authored Nov 07, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  ae62fd17
- [Frontend] Fix multiple values for keyword argument error (#10075) (#10076) · 0dfba97b
  Lei Yang authored Nov 07, 2024
```
Signed-off-by: Lei <ylxx@live.com>
```
  0dfba97b
- [Frontend] Adjust try/except blocks in API impl (#10056) · 29862b88
  Nick Hill authored Nov 07, 2024
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  29862b88
06 Nov, 2024 4 commits
- [V1] Make v1 more testable (#9888) · d58268c5
  Joe Runde authored Nov 06, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  d58268c5
- [Model] Add Idefics3 support (#9767) · a5bba7d2
  Jee Jee Li authored Nov 06, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: B-201 <Joy25810@foxmail.com>
```
  a5bba7d2
- [CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
  Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  21063c11
- [Bugfix] Remove CustomChatCompletionContentPartParam multimodal input type (#10054) · 9d59b755
  zifeitong authored Nov 05, 2024
```
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
```
  9d59b755
05 Nov, 2024 1 commit
- [Frontend] Fix tcp port reservation for api server (#10012) · 5952d811
  Russell Bryant authored Nov 05, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  5952d811
04 Nov, 2024 3 commits
- [Bugfix] Fix `MQLLMEngine` hanging (#9973) · 04cef2c6
  Robert Shaw authored Nov 04, 2024
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  04cef2c6
- [Frontend] Multi-Modality Support for Loading Local Image Files (#9915) · ac6b8f19
  Chauncey authored Nov 04, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  ac6b8f19
- [Model] Add support for H2OVL-Mississippi models (#9747) · 54597724
  shanshan wang authored Nov 03, 2024
```
Signed-off-by: Shanshan Wang <shanshan.wang@h2o.ai>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  54597724
01 Nov, 2024 2 commits
- [Frontend] Use a proper chat template for VLM2Vec (#9912) · ba0d8920
  Cyrus Leung authored Nov 01, 2024
  
  ba0d8920
- [Frontend] Chat-based Embeddings API (#9759) · 06386a64
  Cyrus Leung authored Nov 01, 2024
  
  06386a64
31 Oct, 2024 3 commits
- [Misc] Remove deprecated arg for cuda graph capture (#9864) · 3ea2dc2e
  Roger Wang authored Oct 31, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  3ea2dc2e
- Revert "[Bugfix] Use host argument to bind to interface (#9798)" (#9852) · 890ca360
  Kevin H. Luu authored Oct 30, 2024
  
  890ca360
- [Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens... · abbfb613
  Guillaume Calmettes authored Oct 31, 2024
```
[Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens field for chat completion endpoint (#9837)
```
  abbfb613
30 Oct, 2024 2 commits
- [Bugfix][core] replace heartbeat with pid check (#9818) · 3b3f1e74
  Joe Runde authored Oct 30, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  3b3f1e74
- [Model] Support math-shepherd-mistral-7b-prm model (#9697) · 81f09cfd
  Went-Liang authored Oct 31, 2024
```
Signed-off-by: Went-Liang <wenteng_liang@163.com>
```
  81f09cfd
29 Oct, 2024 3 commits
- [Model] tool calling support for ibm-granite/granite-20b-functioncalling (#8339) · 882a1ad0
  Will Eaton authored Oct 29, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Maximilien de Bayser <maxdebayser@gmail.com>
```
  882a1ad0
- [Bugfix] Use host argument to bind to interface (#9798) · 0f433871
  Sven Seeberg authored Oct 29, 2024
  
  0f433871
- [Frontend] re-enable multi-modality input in the new beam search implementation (#9427) · ef7865b4
  Zhong Qishuai authored Oct 29, 2024
```
Signed-off-by: Qishuai Ferdinandzhong@gmail.com
```
  ef7865b4
26 Oct, 2024 1 commit
- [Bugfix] Steaming continuous_usage_stats default to False (#9709) · 067e77f9
  Sam Stoelinga authored Oct 25, 2024
```
Signed-off-by: Sam Stoelinga <sammiestoel@gmail.com>
```
  067e77f9
24 Oct, 2024 1 commit
- [Log][Bugfix] Fix default value check for `image_url.detail` (#9663) · e26d37a1
  Michael Goin authored Oct 24, 2024
  
  e26d37a1