Commits · ac2f3f7fee93cf9cd97c0078e362feab7b6c8299 · OpenDAS / vllm_cscc

10 Jan, 2025 1 commit
- [Bugfix] Validate lora adapters to avoid crashing server (#11727) · ac2f3f7f
  Joe Runde authored Jan 10, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ac2f3f7f
09 Jan, 2025 3 commits
- [Bugfix] fix beam search input errors and latency benchmark script (#11875) · 1d967acb
  Ye (Charlotte) Qi authored Jan 09, 2025
```
Signed-off-by: Ye Qi <yeq@meta.com>
Co-authored-by: yeq <yeq@devgpu004.lla3.facebook.com>
```
  1d967acb
- [Misc] Move `print_*_once` from utils to logger (#11298) · d848800e
  Cyrus Leung authored Jan 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Co-authored-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
```
  d848800e
- treat do_lower_case in the same way as the sentence-transformers library (#11815) · 1fe554ba
  Maximilien de Bayser authored Jan 09, 2025
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  1fe554ba
08 Jan, 2025 1 commit
- [Hardware][Apple] Native support for macOS Apple Silicon (#11696) · cfd3219f
  Wallas Henrique authored Jan 08, 2025
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  cfd3219f
06 Jan, 2025 1 commit
- [Misc] Fix typo for valid_tool_parses (#11753) · f8fcca10
  Rui Qiao authored Jan 05, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  f8fcca10
05 Jan, 2025 1 commit
- [Frontend] Improve `StreamingResponse` Exception Handling (#11752) · 33fc1e2e
  Robert Shaw authored Jan 05, 2025
  
  33fc1e2e
03 Jan, 2025 2 commits
- [V1] Chore: cruft removal (#11724) · ad0d567e
  Robert Shaw authored Jan 03, 2025
  
  ad0d567e
- [V1] Simplify Shutdown (#11659) · 80c751e7
  Robert Shaw authored Jan 03, 2025
  
  80c751e7
02 Jan, 2025 1 commit
- [Misc] Minimum requirements for SageMaker compatibility (#11576) · 68d37809
  Nathan Azrak authored Jan 03, 2025
  
  68d37809
01 Jan, 2025 1 commit
- [Bugfix][Refactor] Unify model management in frontend (#11660) · 4db72e57
  Joe Runde authored Dec 31, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  4db72e57
30 Dec, 2024 1 commit
- [V1] [6/N] API Server: Better Shutdown (#11586) · 5886aa49
  Robert Shaw authored Dec 30, 2024
  
  5886aa49
28 Dec, 2024 1 commit
- [V1] [4/N] API Server: ZMQ/MP Utilities (#11541) · df04dffa
  Robert Shaw authored Dec 27, 2024
  
  df04dffa
27 Dec, 2024 1 commit
- [Misc] Abstract the logic for reading and writing media content (#11527) · 7af553ea
  Cyrus Leung authored Dec 27, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7af553ea
26 Dec, 2024 2 commits
- [2/N] API Server: Avoid ulimit footgun (#11530) · 55fb97f7
  Robert Shaw authored Dec 26, 2024
  
  55fb97f7
- [1/N] API Server (Remove Proxy) (#11529) · 720b10fd
  Robert Shaw authored Dec 26, 2024
  
  720b10fd
24 Dec, 2024 2 commits
- [Frontend] Online Pooling API (#11457) · 9edca6bf
  Cyrus Leung authored Dec 24, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9edca6bf
- [Misc]Suppress irrelevant exception stack trace information when CUDA… (#11438) · 4f074fbf
  dpxa authored Dec 24, 2024
```
Co-authored-by: shiquan <shiquan>
```
  4f074fbf
23 Dec, 2024 1 commit
- [Docs] Convert rST to MyST (Markdown) (#11145) · 32aa2059
  Rafael Vasquez authored Dec 23, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  32aa2059
21 Dec, 2024 1 commit
- [V1] Make AsyncLLMEngine v1-v0 opaque (#11383) · 584f0ae4
  Ricky Xu authored Dec 20, 2024
```
Signed-off-by: Ricky Xu <xuchen727@hotmail.com>
```
  584f0ae4
20 Dec, 2024 1 commit
- [Bugfix] Don't log OpenAI field aliases as ignored (#11378) · d573aead
  Michael Goin authored Dec 20, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  d573aead
19 Dec, 2024 2 commits

[Feature] Add load generation config from model (#11164) · 5aef4980

Yanyi Liu authored Dec 19, 2024


Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

5aef4980

[Model] IBM Granite 3.1 (#11307) · 17ca9642
Travis Johnson authored Dec 18, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
17ca9642

17 Dec, 2024 3 commits
- [Bugfix] Fix request cancellation without polling (#11190) · 2d1b9baa
  Joe Runde authored Dec 17, 2024
  
  2d1b9baa
- [Frontend] Add OpenAI API support for input_audio (#11027) · 66d4b167
  kYLe authored Dec 17, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  66d4b167
- [CI] Add test case with JSON schema using references + use xgrammar by default... · 0064f697
  Michael Goin authored Dec 16, 2024
```
[CI] Add test case with JSON schema using references + use xgrammar by default with OpenAI parse (#10935)
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  0064f697
16 Dec, 2024 2 commits
- [Model] Refactor Ultravox to use merged input processor (#11198) · d927dbcd
  Isotr0py authored Dec 16, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d927dbcd
- [Bugfix] Fix the default value for temperature in ChatCompletionRequest (#11219) · 17138af7
  yansh97 authored Dec 16, 2024
  
  17138af7
14 Dec, 2024 2 commits
- [Frontend] Add `logits_processors` as an extra completion argument (#11150) · 9c3dadd1
  Brad Hilton authored Dec 14, 2024
```
Signed-off-by: Brad Hilton <brad.hilton.nw@gmail.com>
```
  9c3dadd1
- [Core] V1: Use multiprocessing by default (#11074) · 4863e5fb
  Russell Bryant authored Dec 13, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  4863e5fb
13 Dec, 2024 3 commits
- [Doc] Reorganize online pooling APIs (#11172) · 0920ab91
  Cyrus Leung authored Dec 14, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0920ab91
- [Bugfix] using len(tokenizer) instead of tokenizer.vocab_size in... · 5b0ed839
  zhangjf authored Dec 13, 2024
```
[Bugfix] using len(tokenizer) instead of tokenizer.vocab_size in AllowedTokenIdsLogitsProcessor (#11156)
```
  5b0ed839
- [Frontend] Separate pooling APIs in offline inference (#11129) · eeec9e33
  Cyrus Leung authored Dec 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  eeec9e33
12 Dec, 2024 3 commits
- [Misc][LoRA] Ensure Lora Adapter requests return adapter name (#11094) · 85362f02
  Jiaxin Shan authored Dec 12, 2024
```
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  85362f02
- [Core] cleanup zmq ipc sockets on exit (#11115) · ccede2b2
  Russell Bryant authored Dec 11, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  ccede2b2
- [Bugfix] Multiple fixes to tool streaming with hermes and mistral (#10979) · 7439a8b5
  Clayton authored Dec 11, 2024
```
Signed-off-by: cedonley <clayton@donley.io>
```
  7439a8b5
11 Dec, 2024 4 commits
- [Doc] Update docs to refer to pooling models (#11093) · cad5c0a6
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cad5c0a6
- [Misc] Split up pooling tasks (#10820) · 8f10d5e3
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f10d5e3
- [Bugfix]: Clamp `-inf` logprob values in prompt_logprobs (#11073) · 40766ca1
  Rafael Vasquez authored Dec 11, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  40766ca1
- Fix streaming for granite tool call when <|tool_call|> is present (#11069) · e39400a4
  Maximilien de Bayser authored Dec 11, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  e39400a4