Commits · 739b61a348afa5da297a80ff15f4e39d6e524b53 · OpenDAS / vllm_cscc

22 Jul, 2024 1 commit
- [Frontend] Refactor prompt processing (#4028) · 739b61a3
  Cyrus Leung authored Jul 23, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  739b61a3
21 Jul, 2024 1 commit
- [Frontend] Move chat utils (#6602) · d7f4178d
  Cyrus Leung authored Jul 21, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d7f4178d
19 Jul, 2024 2 commits
- [Bugfix][Frontend] remove duplicate init logger (#6581) · 51f8aa90
  Daniele authored Jul 19, 2024
  
  51f8aa90
- [Bugfix][Frontend] Fix missing `/metrics` endpoint (#6463) · 6366efc6
  Cyrus Leung authored Jul 19, 2024
  
  6366efc6
18 Jul, 2024 2 commits
- [BugFix][Frontend] Use LoRA tokenizer in OpenAI APIs (#6227) · e2fbaee7
  Nick Hill authored Jul 18, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  e2fbaee7
- [core][model] yet another cpu offload implementation (#6496) · 1c27d25f
  youkaichao authored Jul 17, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  1c27d25f
16 Jul, 2024 2 commits
- [Frontend] Support for chat completions input in the tokenize endpoint (#5923) · 7a3d2a5b
  sasha0552 authored Jul 16, 2024
  
  7a3d2a5b
- [Bugfix][CI/Build] Test prompt adapters in openai entrypoint tests (#6419) · d92b3c5c
  Joe authored Jul 15, 2024
  
  d92b3c5c
15 Jul, 2024 1 commit
- [BugFix] BatchResponseData body should be optional (#6345) · b47008b4
  zifeitong authored Jul 14, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  b47008b4
14 Jul, 2024 1 commit
- [Feature] vLLM CLI (#5090) · dbfe254e
  Ethan Xu authored Jul 14, 2024
```
Co-authored-by: simon-mo <simon.mo@hey.com>
```
  dbfe254e
09 Jul, 2024 1 commit

[CORE] Adding support for insertion of soft-tuned prompts (#4645) · 4d6ada94

Swapnil Parekh authored Jul 09, 2024


Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

4d6ada94

08 Jul, 2024 1 commit
- do not exclude `object` field in CompletionStreamResponse (#6196) · 16620f43
  kczimm authored Jul 07, 2024
  
  16620f43
07 Jul, 2024 1 commit
- [misc][frontend] log all available endpoints (#6195) · 3b08fe2b
  youkaichao authored Jul 07, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  3b08fe2b
05 Jul, 2024 1 commit
- [Frontend] Continuous usage stats in OpenAI completion API (#5742) · f1e15da6
  jvlunteren authored Jul 05, 2024
  
  f1e15da6
03 Jul, 2024 3 commits

[vlm] Remove vision language config. (#6089) · d9e98f42

xwjiang2010 authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

d9e98f42

[VLM][Frontend] Proper Image Prompt Formatting from OpenAI API (#6091) · 3a86b54f
Roger Wang authored Jul 02, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
3a86b54f

[Core] Dynamic image size support for VLMs (#5276) · 9831aec4

Cyrus Leung authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

9831aec4

02 Jul, 2024 3 commits
- [VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c
  xwjiang2010 authored Jul 02, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  98d6682c
- [Frontend] Add template related params to request (#5709) · 2c37540a
  danieljannai21 authored Jul 02, 2024
  
  2c37540a
- [Core] Optimize block_manager_v2 vs block_manager_v1 (to make V2 default) (#5602) · 3476ed08
  Alexander Matveev authored Jul 01, 2024
  
  3476ed08
30 Jun, 2024 1 commit
- [Frontend]: Support base64 embedding (#5935) · c6c240aa
  llmpros authored Jun 30, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  c6c240aa
28 Jun, 2024 1 commit
- [Bugfix] Fix Engine Failing After Invalid Request - AsyncEngineDeadError (#5963) · 6a62cb82
  Robert Shaw authored Jun 28, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  6a62cb82
26 Jun, 2024 1 commit
- [Frontend] Add tokenize/detokenize endpoints (#5054) · c54269d9
  sasha0552 authored Jun 26, 2024
  
  c54269d9
22 Jun, 2024 1 commit
- [Misc] Remove #4789 workaround left in vllm/entrypoints/openai/run_batch.py (#5756) · ff9ddbce
  zifeitong authored Jun 21, 2024
  
  ff9ddbce
20 Jun, 2024 1 commit
- [Frontend] Add FlexibleArgumentParser to support both underscore and dash in names (#5718) · 8065a7e2
  Michael Goin authored Jun 20, 2024
  
  8065a7e2
18 Jun, 2024 1 commit

[Misc] Add OpenTelemetry support (#4687) · 7879f24d

Ronen Schaffer authored Jun 18, 2024

This PR adds basic support for OpenTelemetry distributed tracing.
It includes changes to enable tracing functionality and improve monitoring capabilities.

I've also added a markdown with print-screens to guide users how to use this feature. You can find it here

7879f24d

17 Jun, 2024 1 commit
- [Fix] Use utf-8 encoding in entrypoints/openai/run_batch.py (#5606) · 26e1188e
  zifeitong authored Jun 17, 2024
  
  26e1188e
15 Jun, 2024 2 commits
- [Fix] Correct OpenAI batch response format (#5554) · 3ce2c050
  zifeitong authored Jun 15, 2024
  
  3ce2c050
- [mypy] Enable type checking for test directory (#5017) · 0e9164b4
  Cyrus Leung authored Jun 15, 2024
  
  0e9164b4
13 Jun, 2024 1 commit
- [Misc] Add vLLM version getter to utils (#5098) · 03dccc88
  Cyrus Leung authored Jun 14, 2024
  
  03dccc88
12 Jun, 2024 1 commit
- [Frontend] Add "input speed" to tqdm postfix alongside output speed (#5425) · 7d19de2e
  Michael Goin authored Jun 12, 2024
  
  7d19de2e
11 Jun, 2024 2 commits
- [Bugfix][Frontend] Cleanup "fix chat logprobs" (#5026) · 640052b0
  Cyrus Leung authored Jun 11, 2024
  
  640052b0
- [Bugfix] OpenAI entrypoint limits logprobs while ignoring server defined --max-logprobs (#5312) · 351d5e7b
  maor-ps authored Jun 11, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  351d5e7b
10 Jun, 2024 2 commits
- [Feature][Frontend]: Continued `stream_options` implementation also in CompletionRequest (#5319) · 774d1035
  Itay Etelis authored Jun 10, 2024
  
  774d1035
- [Frontend][Misc] Enforce Pixel Values as Input Type for VLMs in API Server (#5374) · 68bc8170
  Roger Wang authored Jun 10, 2024
  
  68bc8170
07 Jun, 2024 2 commits
- [Frontend] Add OpenAI Vision API Support (#5237) · 7a9cb294
  Roger Wang authored Jun 07, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7a9cb294
- [Feature][Frontend]: Add support for `stream_options` in `ChatCompletionRequest` (#5135) · baa15a9e
  Itay Etelis authored Jun 07, 2024
  
  baa15a9e
06 Jun, 2024 1 commit
- [Frontend] enable passing multiple LoRA adapters at once to generate() (#5300) · 828da0d4
  Matthew Goldey authored Jun 06, 2024
  
  828da0d4
05 Jun, 2024 2 commits
- [BugFix] Apply get_cached_tokenizer to the tokenizer setter of LLM (#5207) · eb8fcd26
  DriverSong authored Jun 06, 2024
```
Co-authored-by: qiujiawei9 <qiujiawei9@jd.com>
```
  eb8fcd26
- [Frontend] OpenAI API server: Add `add_special_tokens` to... · f0a50054
  tomeras91 authored Jun 05, 2024
```
[Frontend] OpenAI API server: Add `add_special_tokens` to ChatCompletionRequest (default False) (#5278)
```
  f0a50054