Commits · 5bfd1bbc9831fed39632f071f16bb62373ec1249 · OpenDAS / vllm_cscc

22 Jun, 2024 1 commit
- [Misc] Remove #4789 workaround left in vllm/entrypoints/openai/run_batch.py (#5756) · ff9ddbce
  zifeitong authored Jun 21, 2024
  
  ff9ddbce
20 Jun, 2024 1 commit
- [Frontend] Add FlexibleArgumentParser to support both underscore and dash in names (#5718) · 8065a7e2
  Michael Goin authored Jun 20, 2024
  
  8065a7e2
18 Jun, 2024 1 commit

[Misc] Add OpenTelemetry support (#4687) · 7879f24d

Ronen Schaffer authored Jun 18, 2024

This PR adds basic support for OpenTelemetry distributed tracing.
It includes changes to enable tracing functionality and improve monitoring capabilities.

I've also added a markdown with print-screens to guide users how to use this feature. You can find it here

7879f24d

17 Jun, 2024 1 commit
- [Fix] Use utf-8 encoding in entrypoints/openai/run_batch.py (#5606) · 26e1188e
  zifeitong authored Jun 17, 2024
  
  26e1188e
15 Jun, 2024 2 commits
- [Fix] Correct OpenAI batch response format (#5554) · 3ce2c050
  zifeitong authored Jun 15, 2024
  
  3ce2c050
- [mypy] Enable type checking for test directory (#5017) · 0e9164b4
  Cyrus Leung authored Jun 15, 2024
  
  0e9164b4
13 Jun, 2024 1 commit
- [Misc] Add vLLM version getter to utils (#5098) · 03dccc88
  Cyrus Leung authored Jun 14, 2024
  
  03dccc88
11 Jun, 2024 2 commits
- [Bugfix][Frontend] Cleanup "fix chat logprobs" (#5026) · 640052b0
  Cyrus Leung authored Jun 11, 2024
  
  640052b0
- [Bugfix] OpenAI entrypoint limits logprobs while ignoring server defined --max-logprobs (#5312) · 351d5e7b
  maor-ps authored Jun 11, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  351d5e7b
10 Jun, 2024 2 commits
- [Feature][Frontend]: Continued `stream_options` implementation also in CompletionRequest (#5319) · 774d1035
  Itay Etelis authored Jun 10, 2024
  
  774d1035
- [Frontend][Misc] Enforce Pixel Values as Input Type for VLMs in API Server (#5374) · 68bc8170
  Roger Wang authored Jun 10, 2024
  
  68bc8170
07 Jun, 2024 2 commits
- [Frontend] Add OpenAI Vision API Support (#5237) · 7a9cb294
  Roger Wang authored Jun 07, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7a9cb294
- [Feature][Frontend]: Add support for `stream_options` in `ChatCompletionRequest` (#5135) · baa15a9e
  Itay Etelis authored Jun 07, 2024
  
  baa15a9e
05 Jun, 2024 1 commit
- [Frontend] OpenAI API server: Add `add_special_tokens` to... · f0a50054
  tomeras91 authored Jun 05, 2024
```
[Frontend] OpenAI API server: Add `add_special_tokens` to ChatCompletionRequest (default False) (#5278)
```
  f0a50054
04 Jun, 2024 1 commit
- [Bugfix] Support `prompt_logprobs==0` (#5217) · 06b2550c
  Toshiki Kataoka authored Jun 04, 2024
  
  06b2550c
03 Jun, 2024 1 commit
- [FRONTEND] OpenAI `tools` support named functions (#5032) · f775a07e
  Breno Faria authored Jun 04, 2024
  
  f775a07e
02 Jun, 2024 1 commit
- [Frontend][OpenAI] Support for returning max_model_len on /v1/models response (#4643) · f790ad3c
  Avinash Raj authored Jun 02, 2024
  
  f790ad3c
01 Jun, 2024 1 commit
- [Bugfix] Fix call to init_logger in openai server (#4765) · 37464a0f
  Nadav Shmayovits authored Jun 01, 2024
  
  37464a0f
30 May, 2024 1 commit
- [BUGFIX] [FRONTEND] Correct chat logprobs (#5029) · 87d41c84
  Breno Faria authored May 30, 2024
```
Co-authored-by: Breno Faria <breno.faria@intrafind.com>
```
  87d41c84
29 May, 2024 1 commit
- [Bugfix] logprobs is not compatible with the OpenAI spec #4795 (#5031) · 7c3604fb
  Itay Etelis authored May 30, 2024
  
  7c3604fb
28 May, 2024 1 commit
- [Core] Consolidate prompt arguments to LLM engines (#4328) · 5ae5ed1e
  Cyrus Leung authored May 29, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  5ae5ed1e
25 May, 2024 1 commit

[Kernel][Backend][Model] Blocksparse flash attention kernel and Phi-3-Small model (#4799) · 8e192ff9

Eric Xihui Lin authored May 25, 2024


Co-authored-by: beagleski <yunanzhang@microsoft.com>
Co-authored-by: bapatra <bapatra@microsoft.com>
Co-authored-by: Barun Patra <codedecde@users.noreply.github.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

8e192ff9

17 May, 2024 1 commit
- [Frontend] OpenAI API server: Do not add bos token by default when encoding (#4688) · 0150a106
  bofeng huang authored May 17, 2024
  
  0150a106
16 May, 2024 2 commits
- [Bugfix] Bypass authorization API token for preflight requests (#4862) · 9216b9cc
  Pierre Dulac authored May 16, 2024
  
  9216b9cc
- [Frontend] Separate OpenAI Batch Runner usage from API Server (#4851) · 5e0391c0
  Alex Wu authored May 16, 2024
  
  5e0391c0
15 May, 2024 2 commits
- [Frontend] Support OpenAI batch file format (#4794) · 52f8107c
  Alex Wu authored May 15, 2024
```
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  52f8107c
- [Frontend] Re-enable custom roles in Chat Completions API (#4758) · fc0d9dfc
  Cyrus Leung authored May 16, 2024
  
  fc0d9dfc
11 May, 2024 1 commit
- [Model][Misc] Add e5-mistral-7b-instruct and Embedding API (#3734) · e254497b
  Chang Su authored May 11, 2024
  
  e254497b
09 May, 2024 1 commit
- [Frontend] Move async logic outside of constructor (#4674) · f12b20de
  Cyrus Leung authored May 09, 2024
  
  f12b20de
06 May, 2024 1 commit
- [Bugfix] Fix `asyncio.Task` not being subscriptable (#4623) · 323f27b9
  Cyrus Leung authored May 07, 2024
  
  323f27b9
04 May, 2024 1 commit
- [Bugfix] Fix inappropriate content of model_name tag in Prometheus metrics (#3937) · 43029870
  DearPlanet authored May 05, 2024
  
  43029870
03 May, 2024 3 commits
- Fix/async chat serving (#2727) · f8e7adda
  Sebastian Schoennenbeck authored May 03, 2024
  
  f8e7adda
- [Bugfix] Allow "None" or "" to be passed to CLI for string args that default to None (#4586) · 7e65477e
  Michael Goin authored May 03, 2024
  
  7e65477e
- [BugFix] Prevent the task of `_force_log` from being garbage collected (#4567) · 808632d3
  Yang, Bo authored May 02, 2024
  
  808632d3
02 May, 2024 1 commit
- [Misc] centralize all usage of environment variables (#4548) · 5b8a7c1c
  youkaichao authored May 02, 2024
  
  5b8a7c1c
01 May, 2024 3 commits
- [Bugfix] Add validation for seed (#4529) · c47ba4aa
  sasha0552 authored May 01, 2024
  
  c47ba4aa
- [Bugfix] Fix 307 Redirect for `/metrics` (#4523) · 4dc8026d
  Robert Shaw authored May 01, 2024
  
  4dc8026d
- Allow user to define whitespace pattern for outlines (#4305) · c3845d82
  Robert Caulk authored May 01, 2024
  
  c3845d82
30 Apr, 2024 1 commit
- [Frontend] Support complex message content for chat completions endpoint (#3467) · a4941404
  Florian Greinacher authored May 01, 2024
```
Co-authored-by: Lily Liu <lilyliupku@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  a4941404
27 Apr, 2024 1 commit
- [Bugfix][Core] Fix get decoding config from ray (#4335) · 7134303c
  Roy authored Apr 27, 2024
  
  7134303c