Commits · 7c7714d856eee6fa94aade729b67f00584f72a4c · OpenDAS / vllm_cscc

18 Sep, 2024 1 commit

[Core][Bugfix][Perf] Introduce `MQLLMEngine` to avoid `asyncio` OH (#8157) · 7c7714d8

Alexander Matveev authored Sep 18, 2024


Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

7c7714d8

17 Sep, 2024 1 commit
- [Model] Add mistral function calling format to all models loaded with "mistral" format (#8515) · a54ed802
  Patrick von Platen authored Sep 17, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  a54ed802
12 Sep, 2024 2 commits
- [Core] Add engine option to return only deltas or final output (#7381) · 551ce010
  Nick Hill authored Sep 12, 2024
  
  551ce010
- [Bugfix]: Fix the logic for deciding if tool parsing is used (#8366) · 5a60699c
  tomeras91 authored Sep 12, 2024
  
  5a60699c
10 Sep, 2024 1 commit
- [Frontend] Clean up type annotations for mistral tokenizer (#8314) · 8c054b7a
  Cyrus Leung authored Sep 11, 2024
  
  8c054b7a
09 Sep, 2024 1 commit
- [Bugfix] Streamed tool calls now more strictly follow OpenAI's format; ensures... · 08287ef6
  Kyle Mistele authored Sep 09, 2024
```
[Bugfix] Streamed tool calls now more strictly follow OpenAI's format; ensures Vercel AI SDK compatibility (#8272)
```
  08287ef6
04 Sep, 2024 2 commits
- [Feature] OpenAI-Compatible Tools API + Streaming for Hermes & Mistral models (#5649) · e02ce498
  Kyle Mistele authored Sep 04, 2024
```
Co-authored-by: constellate <constellate@1-ai-appserver-staging.codereach.com>
Co-authored-by: Kyle Mistele <kyle@constellate.ai>
```
  e02ce498
- [Frontend] Multimodal support in offline chat (#8098) · 855c262a
  Cyrus Leung authored Sep 04, 2024
  
  855c262a
31 Aug, 2024 1 commit
- [Frontend][VLM] Add support for multiple multi-modal items (#8049) · 5231f089
  Roger Wang authored Aug 31, 2024
  
  5231f089
27 Aug, 2024 1 commit
- [Model] Add Mistral Tokenization to improve robustness and chat encoding (#7739) · 6fc4e6e0
  Patrick von Platen authored Aug 27, 2024
  
  6fc4e6e0
21 Aug, 2024 1 commit
- [mypy] Enable following imports for entrypoints (#7248) · baaedfdb
  Cyrus Leung authored Aug 21, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Fei <dfdfcai4@gmail.com>
```
  baaedfdb
16 Aug, 2024 1 commit
- [Feature]: Add OpenAI server prompt_logprobs support #6508 (#7453) · f878c8fe
  Grant Pinkert authored Aug 16, 2024
  
  f878c8fe
07 Aug, 2024 2 commits
- [Frontend] Gracefully handle missing chat template and fix CI failure (#7238) · 66d617e3
  Cyrus Leung authored Aug 07, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  66d617e3
- [BugFix] Overhaul async request cancellation (#7111) · 9a3f49ae
  Nick Hill authored Aug 06, 2024
  
  9a3f49ae
03 Aug, 2024 2 commits

[Frontend] Factor out chat message parsing (#7055) · 8c025fa7
Cyrus Leung authored Aug 03, 2024

8c025fa7

[ Frontend ] Multiprocessing for OpenAI Server with `zeromq` (#6883) · ed812a73

Robert Shaw authored Aug 02, 2024


Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

ed812a73

01 Aug, 2024 1 commit
- [Bugfix] Set SamplingParams.max_tokens for OpenAI requests if not provided by user (#6954) · 3c10591e
  zifeitong authored Jul 31, 2024
  
  3c10591e
29 Jul, 2024 1 commit
- [Frontend] New `allowed_token_ids` decoding request parameter (#6753) · 9f69d824
  Nick Hill authored Jul 29, 2024
  
  9f69d824
25 Jul, 2024 1 commit
- [Frontend] Represent tokens with identifiable strings (#6626) · 5689e256
  Evan Z. Liu authored Jul 24, 2024
  
  5689e256
23 Jul, 2024 1 commit
- [Frontend] Add Usage data in each chunk for chat_serving. #6540 (#6652) · 58f53034
  Yehoshua Cohen authored Jul 23, 2024
  
  58f53034
22 Jul, 2024 1 commit
- [Frontend] Refactor prompt processing (#4028) · 739b61a3
  Cyrus Leung authored Jul 23, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  739b61a3
21 Jul, 2024 1 commit
- [Frontend] Move chat utils (#6602) · d7f4178d
  Cyrus Leung authored Jul 21, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d7f4178d
18 Jul, 2024 1 commit
- [BugFix][Frontend] Use LoRA tokenizer in OpenAI APIs (#6227) · e2fbaee7
  Nick Hill authored Jul 18, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  e2fbaee7
16 Jul, 2024 1 commit
- [Frontend] Support for chat completions input in the tokenize endpoint (#5923) · 7a3d2a5b
  sasha0552 authored Jul 16, 2024
  
  7a3d2a5b
09 Jul, 2024 1 commit

[CORE] Adding support for insertion of soft-tuned prompts (#4645) · 4d6ada94

Swapnil Parekh authored Jul 09, 2024


Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

4d6ada94

03 Jul, 2024 3 commits

[vlm] Remove vision language config. (#6089) · d9e98f42

xwjiang2010 authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

d9e98f42

[VLM][Frontend] Proper Image Prompt Formatting from OpenAI API (#6091) · 3a86b54f
Roger Wang authored Jul 02, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
3a86b54f

[Core] Dynamic image size support for VLMs (#5276) · 9831aec4

Cyrus Leung authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

9831aec4

02 Jul, 2024 2 commits
- [VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c
  xwjiang2010 authored Jul 02, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  98d6682c
- [Frontend] Add template related params to request (#5709) · 2c37540a
  danieljannai21 authored Jul 02, 2024
  
  2c37540a
18 Jun, 2024 1 commit

[Misc] Add OpenTelemetry support (#4687) · 7879f24d

Ronen Schaffer authored Jun 18, 2024

This PR adds basic support for OpenTelemetry distributed tracing.
It includes changes to enable tracing functionality and improve monitoring capabilities.

I've also added a markdown with print-screens to guide users how to use this feature. You can find it here

7879f24d

15 Jun, 2024 1 commit
- [mypy] Enable type checking for test directory (#5017) · 0e9164b4
  Cyrus Leung authored Jun 15, 2024
  
  0e9164b4
11 Jun, 2024 1 commit
- [Bugfix][Frontend] Cleanup "fix chat logprobs" (#5026) · 640052b0
  Cyrus Leung authored Jun 11, 2024
  
  640052b0
10 Jun, 2024 1 commit
- [Feature][Frontend]: Continued `stream_options` implementation also in CompletionRequest (#5319) · 774d1035
  Itay Etelis authored Jun 10, 2024
  
  774d1035
07 Jun, 2024 2 commits
- [Frontend] Add OpenAI Vision API Support (#5237) · 7a9cb294
  Roger Wang authored Jun 07, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7a9cb294
- [Feature][Frontend]: Add support for `stream_options` in `ChatCompletionRequest` (#5135) · baa15a9e
  Itay Etelis authored Jun 07, 2024
  
  baa15a9e
05 Jun, 2024 1 commit
- [Frontend] OpenAI API server: Add `add_special_tokens` to... · f0a50054
  tomeras91 authored Jun 05, 2024
```
[Frontend] OpenAI API server: Add `add_special_tokens` to ChatCompletionRequest (default False) (#5278)
```
  f0a50054
03 Jun, 2024 1 commit
- [FRONTEND] OpenAI `tools` support named functions (#5032) · f775a07e
  Breno Faria authored Jun 04, 2024
  
  f775a07e
30 May, 2024 1 commit
- [BUGFIX] [FRONTEND] Correct chat logprobs (#5029) · 87d41c84
  Breno Faria authored May 30, 2024
```
Co-authored-by: Breno Faria <breno.faria@intrafind.com>
```
  87d41c84
29 May, 2024 1 commit
- [Bugfix] logprobs is not compatible with the OpenAI spec #4795 (#5031) · 7c3604fb
  Itay Etelis authored May 30, 2024
  
  7c3604fb