Commits · afd0da2186c1d58fb48e138df0a2f548612b5d7d · OpenDAS / vllm_cscc · GitLab

29 Jan, 2025 1 commit

[Frontend] Support reasoning content for deepseek r1 (#12473) · a7e3eba6

Ce Gao authored Jan 29, 2025


Signed-off-by: Ce Gao <cegao@tensorchord.ai>
Co-authored-by: Rafael Vasquez <rafvasq21@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>

a7e3eba6

05 Jan, 2025 1 commit
- [Frontend] Improve `StreamingResponse` Exception Handling (#11752) · 33fc1e2e
  Robert Shaw authored Jan 05, 2025
  
  33fc1e2e
01 Jan, 2025 1 commit
- [Bugfix][Refactor] Unify model management in frontend (#11660) · 4db72e57
  Joe Runde authored Dec 31, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  4db72e57
19 Dec, 2024 1 commit

[Feature] Add load generation config from model (#11164) · 5aef4980

Yanyi Liu authored Dec 19, 2024


Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

5aef4980

17 Dec, 2024 1 commit
- [Bugfix] Fix request cancellation without polling (#11190) · 2d1b9baa
  Joe Runde authored Dec 17, 2024
  
  2d1b9baa
14 Dec, 2024 1 commit
- [Frontend] Add `logits_processors` as an extra completion argument (#11150) · 9c3dadd1
  Brad Hilton authored Dec 14, 2024
```
Signed-off-by: Brad Hilton <brad.hilton.nw@gmail.com>
```
  9c3dadd1
12 Dec, 2024 2 commits
- [Misc][LoRA] Ensure Lora Adapter requests return adapter name (#11094) · 85362f02
  Jiaxin Shan authored Dec 12, 2024
```
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  85362f02
- [Bugfix] Multiple fixes to tool streaming with hermes and mistral (#10979) · 7439a8b5
  Clayton authored Dec 11, 2024
```
Signed-off-by: cedonley <clayton@donley.io>
```
  7439a8b5
10 Dec, 2024 1 commit
- [Frontend] Use request id from header (#10968) · 980ad394
  Joe Runde authored Dec 09, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  980ad394
21 Nov, 2024 1 commit
- [Bug]: When apply continue_final_message for OpenAI server, the "echo":false is ignored (#10180) · da7e702c
  Chauncey authored Nov 22, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  da7e702c
16 Nov, 2024 1 commit
- [Frontend] Automatic detection of chat content format from AST (#9919) · 32e46e00
  Cyrus Leung authored Nov 16, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  32e46e00
15 Nov, 2024 1 commit
- [Tool parsing] Improve / correct mistral tool parsing (#10333) · 11cd1ae6
  Patrick von Platen authored Nov 15, 2024
  
  11cd1ae6
14 Nov, 2024 2 commits

[BugFix]: properly deserialize `tool_calls` iterator before processing by... · 52b48c1e

Guillaume Calmettes authored Nov 14, 2024


[BugFix]: properly deserialize `tool_calls` iterator before processing by mistral-common when MistralTokenizer is used (#9951)
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>

52b48c1e

[Frontend] Pythonic tool parser (#9859) · f67ce05d
Mike Depinet authored Nov 13, 2024
```
Signed-off-by: Mike Depinet <mike@fixie.ai>
```
f67ce05d

13 Nov, 2024 1 commit
- [1/N] Initial prototype for multi-modal processor (#10044) · 0b8bb86b
  Cyrus Leung authored Nov 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0b8bb86b
12 Nov, 2024 1 commit
- [Frontend] Add per-request number of cached token stats (#10174) · 47db6ec8
  zifeitong authored Nov 12, 2024
  
  47db6ec8
01 Nov, 2024 1 commit
- [Frontend] Chat-based Embeddings API (#9759) · 06386a64
  Cyrus Leung authored Nov 01, 2024
  
  06386a64
29 Oct, 2024 1 commit
- [Frontend] re-enable multi-modality input in the new beam search implementation (#9427) · ef7865b4
  Zhong Qishuai authored Oct 29, 2024
```
Signed-off-by: Qishuai Ferdinandzhong@gmail.com
```
  ef7865b4
24 Oct, 2024 1 commit
- [Bugfix]: Make chat content text allow type content (#9358) · 33bab410
  Vinay R Damodaran authored Oct 24, 2024
```
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
```
  33bab410
22 Oct, 2024 1 commit
- [Frontend] Support custom request_id from request (#9550) · 434984e6
  Yuhong Guo authored Oct 23, 2024
```
Co-authored-by: Yuhong Guo <yuhong.gyh@antgroup.com>
```
  434984e6
17 Oct, 2024 1 commit
- [Misc] Print stack trace using `logger.exception` (#9461) · 390be746
  Cyrus Leung authored Oct 17, 2024
  
  390be746
15 Oct, 2024 2 commits
- [Bugfix] Fix vLLM UsageInfo and logprobs None AssertionError with empty token_ids (#9034) · ba309422
  Chang Su authored Oct 15, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  ba309422
- [BugFix] Fix chat API continuous usage stats (#9357) · e9d517f2
  Nick Hill authored Oct 15, 2024
  
  e9d517f2
14 Oct, 2024 1 commit
- [Frontend] merge beam search implementations (#9296) · 4d31cd42
  Brendan Wong authored Oct 14, 2024
  
  4d31cd42
12 Oct, 2024 1 commit
- [BugFix] Fix tool call finish reason in streaming case (#9209) · ec10cb85
  Maximilien de Bayser authored Oct 11, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  ec10cb85
08 Oct, 2024 1 commit
- [Frontend] API support for beam search for MQLLMEngine (#9117) · 8c746226
  Brendan Wong authored Oct 07, 2024
  
  8c746226
06 Oct, 2024 2 commits
- [Bugfix] fix tool_parser error handling when serve a model not support it (#8709) · fdf59d30
  Yanyi Liu authored Oct 06, 2024
  
  fdf59d30
- [Frontend] API support for beam search (#9087) · 168cab6b
  Brendan Wong authored Oct 05, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  168cab6b
04 Oct, 2024 1 commit
- [Frontend][Feature] support tool calling for internlm/internlm2_5-7b-chat model (#8405) · 3dbb215b
  代君 authored Oct 04, 2024
  
  3dbb215b
01 Oct, 2024 2 commits
- [Core] [Frontend] Priority scheduling for embeddings and in the OpenAI-API (#8965) · 35bd2151
  Sebastian Schoennenbeck authored Oct 01, 2024
  
  35bd2151
- [Frontend][Core] Move guided decoding params into sampling params (#8252) · 062c89e7
  Joe Runde authored Sep 30, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  062c89e7
29 Sep, 2024 1 commit
- [Frontend] Added support for HF's new `continue_final_message` parameter (#8942) · 6c9ba48f
  danieljannai21 authored Sep 29, 2024
  
  6c9ba48f
27 Sep, 2024 1 commit
- [Feature] Add support for Llama 3.1 and 3.2 tool use (#8343) · 344cd2b6
  Maximilien de Bayser authored Sep 26, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  344cd2b6
26 Sep, 2024 2 commits
- [BugFix] Fix test breakages from transformers 4.45 upgrade (#8829) · 4b377d6f
  Nick Hill authored Sep 27, 2024
  
  4b377d6f
- [Bugfix] Include encoder prompts len to non-stream api usage response (#8861) · 93d364da
  Pernekhan Utemuratov authored Sep 26, 2024
  
  93d364da
25 Sep, 2024 2 commits

[Model] Add support for the multi-modal Llama 3.2 model (#8811) · 770ec602

Chen Zhang authored Sep 25, 2024


Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

770ec602

[Frontend] OpenAI server: propagate usage accounting to FastAPI middleware layer (#8672) · 1ac3de09
Adam Tilghman authored Sep 25, 2024

1ac3de09

20 Sep, 2024 1 commit
- [Core] Support Lora lineage and base model metadata management (#6315) · 260d40b5
  Jiaxin Shan authored Sep 19, 2024
  
  260d40b5
18 Sep, 2024 1 commit

[Core][Bugfix][Perf] Introduce `MQLLMEngine` to avoid `asyncio` OH (#8157) · 7c7714d8

Alexander Matveev authored Sep 18, 2024


Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

7c7714d8

17 Sep, 2024 1 commit
- [Model] Add mistral function calling format to all models loaded with "mistral" format (#8515) · a54ed802
  Patrick von Platen authored Sep 17, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  a54ed802