Commits · 344cd2b6f4c22bf278cff96066001d216ec1fe82 · OpenDAS / vllm_cscc

27 Sep, 2024 1 commit
- [Feature] Add support for Llama 3.1 and 3.2 tool use (#8343) · 344cd2b6
  Maximilien de Bayser authored Sep 26, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  344cd2b6
26 Sep, 2024 3 commits
- [BugFix] Fix test breakages from transformers 4.45 upgrade (#8829) · 4b377d6f
  Nick Hill authored Sep 27, 2024
  
  4b377d6f
- fix validation: Only set tool_choice `auto` if at least one tool is provided (#8568) · ee2da3e9
  Chirag Jain authored Sep 27, 2024
  
  ee2da3e9
- [Bugfix] Include encoder prompts len to non-stream api usage response (#8861) · 93d364da
  Pernekhan Utemuratov authored Sep 26, 2024
  
  93d364da
25 Sep, 2024 5 commits
- [Model] Add support for the multi-modal Llama 3.2 model (#8811) · 770ec602
  Chen Zhang authored Sep 25, 2024
```
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  770ec602
- Revert "rename PromptInputs and inputs with backward compatibility (#8760) (#8810) · 4f1ba084
  Simon Mo authored Sep 25, 2024
  
  4f1ba084
- rename PromptInputs and inputs with backward compatibility (#8760) · 28e1299e
  Cyrus Leung authored Sep 26, 2024
  
  28e1299e
- [Frontend] OpenAI server: propagate usage accounting to FastAPI middleware layer (#8672) · 1ac3de09
  Adam Tilghman authored Sep 25, 2024
  
  1ac3de09
- [Core] Adding Priority Scheduling (#5958) · 6da1ab6b
  Archit Patke authored Sep 24, 2024
  
  6da1ab6b
24 Sep, 2024 3 commits
- [Frontend] Batch inference for llm.chat() API (#8648) · 2529d09b
  Andy authored Sep 24, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  2529d09b
- Revert "[Core] Rename `PromptInputs` to `PromptType`, and `inputs` to `prompt`" (#8750) · 3185fb0c
  Simon Mo authored Sep 23, 2024
  
  3185fb0c
- re-implement beam search on top of vllm core (#8726) · 0250dd68
  youkaichao authored Sep 23, 2024
```
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
```
  0250dd68
23 Sep, 2024 1 commit
- [Core][Frontend] Support Passing Multimodal Processor Kwargs (#8657) · 9b8c8ba1
  Alex Brooks authored Sep 23, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  9b8c8ba1
21 Sep, 2024 1 commit
- [Core] Rename `PromptInputs` and `inputs`(#8673) · 0057894e
  Cyrus Leung authored Sep 21, 2024
  
  0057894e
20 Sep, 2024 2 commits
- [Bugfix] Config got an unexpected keyword argument 'engine' (#8556) · 2874bac6
  Pastel！ authored Sep 21, 2024
  
  2874bac6
- [Core] Support Lora lineage and base model metadata management (#6315) · 260d40b5
  Jiaxin Shan authored Sep 19, 2024
  
  260d40b5
18 Sep, 2024 3 commits
- [BugFix] Nonzero exit code if MQLLMEngine startup fails (#8572) · d9cd78eb
  Nick Hill authored Sep 18, 2024
  
  d9cd78eb
- [Core][Bugfix][Perf] Introduce `MQLLMEngine` to avoid `asyncio` OH (#8157) · 7c7714d8
  Alexander Matveev authored Sep 18, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  7c7714d8
- [Misc] Add argument to disable FastAPI docs (#8554) · e3515729
  Jiaxin Shan authored Sep 18, 2024
  
  e3515729
17 Sep, 2024 2 commits
- [Model] Add mistral function calling format to all models loaded with "mistral" format (#8515) · a54ed802
  Patrick von Platen authored Sep 17, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  a54ed802
- [Encoder decoder] Add cuda graph support during decoding for encoder-decoder models (#7631) · 1009e93c
  sroy745 authored Sep 17, 2024
  
  1009e93c
16 Sep, 2024 3 commits
- [Bugfix] Bind api server port before starting engine (#8491) · 47f5e03b
  Kevin Lin authored Sep 16, 2024
  
  47f5e03b
- [BugFix] Fix clean shutdown issues (#8492) · acd5511b
  Nick Hill authored Sep 16, 2024
  
  acd5511b
- [Frontend] Expose revision arg in OpenAI server (#8501) · 837c1968
  lewtun authored Sep 16, 2024
  
  837c1968
12 Sep, 2024 4 commits
- [Core] Add engine option to return only deltas or final output (#7381) · 551ce010
  Nick Hill authored Sep 12, 2024
  
  551ce010
- [BugFix] Fix Duplicate Assignment in Hermes2ProToolParser (#8423) · 1f0c75af
  Luis Vega authored Sep 12, 2024
  
  1f0c75af
- [Bugfix]: Fix the logic for deciding if tool parsing is used (#8366) · 5a60699c
  tomeras91 authored Sep 12, 2024
  
  5a60699c
- [misc] remove engine_use_ray (#8126) · f842a7af
  youkaichao authored Sep 11, 2024
  
  f842a7af
11 Sep, 2024 4 commits
- Pixtral (#8377) · d394787e
  Patrick von Platen authored Sep 11, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d394787e
- [Bugfix] Add missing attributes in mistral tokenizer (#8364) · 7015417f
  Cyrus Leung authored Sep 12, 2024
  
  7015417f
- [Model][VLM] Add Qwen2-VL model support (#7905) · 3b7fea77
  Yang Fan authored Sep 12, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3b7fea77
- [Frontend] Create ErrorResponse instead of raising exceptions in run_batch (#8347) · cea95dfb
  Pooya Davoodi authored Sep 10, 2024
  
  cea95dfb
10 Sep, 2024 1 commit
- [Frontend] Clean up type annotations for mistral tokenizer (#8314) · 8c054b7a
  Cyrus Leung authored Sep 11, 2024
  
  8c054b7a
09 Sep, 2024 2 commits
- [Frontend] Add progress reporting to run_batch.py (#8060) · 58fcc854
  Adam Lugowski authored Sep 09, 2024
```
Co-authored-by: Adam Lugowski <adam.lugowski@parasail.io>
```
  58fcc854
- [Bugfix] Streamed tool calls now more strictly follow OpenAI's format; ensures... · 08287ef6
  Kyle Mistele authored Sep 09, 2024
```
[Bugfix] Streamed tool calls now more strictly follow OpenAI's format; ensures Vercel AI SDK compatibility (#8272)
```
  08287ef6
07 Sep, 2024 1 commit
- [misc] [doc] [frontend] LLM torch profiler support (#7943) · 12dd7158
  William Lin authored Sep 06, 2024
  
  12dd7158
06 Sep, 2024 2 commits
- [Misc] Remove `SqueezeLLM` (#8220) · 23f32229
  Dipika Sikka authored Sep 06, 2024
  
  23f32229
- [Core] Support load and unload LoRA in api server (#6566) · db3bf7c9
  Jiaxin Shan authored Sep 05, 2024
```
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  db3bf7c9
05 Sep, 2024 1 commit
- [MODEL] Qwen Multimodal Support (Qwen-VL / Qwen-VL-Chat) (#8029) · 9da25a88
  Alex Brooks authored Sep 05, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9da25a88
04 Sep, 2024 1 commit

[Feature] OpenAI-Compatible Tools API + Streaming for Hermes & Mistral models (#5649) · e02ce498

Kyle Mistele authored Sep 04, 2024


Co-authored-by: constellate <constellate@1-ai-appserver-staging.codereach.com>
Co-authored-by: Kyle Mistele <kyle@constellate.ai>

e02ce498