Commits · 85362f028c0324d8d00b0438f29c3d9f64737b9a · OpenDAS / vllm_cscc · GitLab

12 Dec, 2024 10 commits
- [Misc][LoRA] Ensure Lora Adapter requests return adapter name (#11094) · 85362f02
  Jiaxin Shan authored Dec 12, 2024
```
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  85362f02
- [core][distributed] initialization from StatelessProcessGroup (#10986) · 62de37a3
  youkaichao authored Dec 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  62de37a3
- [Hardware][Intel-Gaudi] Enable LoRA support for Intel Gaudi (HPU) (#10565) · 81958242
  Sanju C Sudhakaran authored Dec 12, 2024
```
Signed-off-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
```
  81958242
- [V1] Use more persistent buffers to optimize input preparation overheads (#11111) · f092153f
  Woosuk Kwon authored Dec 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f092153f
- [Model] Add support for embedding model GritLM (#10816) · 1da8f0e1
  Pooya Davoodi authored Dec 11, 2024
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  1da8f0e1
- [Core] cleanup zmq ipc sockets on exit (#11115) · ccede2b2
  Russell Bryant authored Dec 11, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  ccede2b2
- Update link to LlamaStack remote vLLM guide in serving_with_llamastack.rst (#11112) · 24a36d6d
  Yuan Tang authored Dec 11, 2024
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  24a36d6d
- [Docs] Add media kit (#11121) · 8fb26dac
  Simon Mo authored Dec 11, 2024
  
  8fb26dac
- [Bugfix] Multiple fixes to tool streaming with hermes and mistral (#10979) · 7439a8b5
  Clayton authored Dec 11, 2024
```
Signed-off-by: cedonley <clayton@donley.io>
```
  7439a8b5
- [V1] VLM preprocessor hashing (#11020) · 4e116833
  Alexander Matveev authored Dec 11, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  4e116833
11 Dec, 2024 20 commits
- [V1][Core] Remove should_shutdown to simplify core process termination (#11113) · 452a723b
  Tyler Michael Smith authored Dec 11, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  452a723b
- [CI/Build] Split up VLM tests (#11083) · d1e21a97
  Cyrus Leung authored Dec 12, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d1e21a97
- [core] Bump ray to use _overlap_gpu_communication in compiled graph tests (#10410) · 72ff3a96
  Rui Qiao authored Dec 11, 2024
```
Signed-off-by: Rui Qiao <ubuntu@ip-172-31-15-128.us-west-2.compute.internal>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Rui Qiao <ubuntu@ip-172-31-15-128.us-west-2.compute.internal>
```
  72ff3a96
- [torch.compile] remove graph logging in ci (#11110) · 66aaa772
  youkaichao authored Dec 11, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  66aaa772
- [V1] Use input_ids as input for text-only models (#11032) · d643c2ab
  Woosuk Kwon authored Dec 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d643c2ab
- [torch.compile] use depyf to dump torch.compile internals (#10972) · 91642db9
  youkaichao authored Dec 11, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  91642db9
- [Doc] Installed version of llmcompressor for int8/fp8 quantization (#11103) · fd222206
  bingps authored Dec 11, 2024
```
Signed-off-by: Guangda Liu <bingps@users.noreply.github.com>
Co-authored-by: Guangda Liu <bingps@users.noreply.github.com>
```
  fd222206
- [CI/Build] Enable prefix caching test for AMD (#11098) · b2f77545
  hissu-hyvarinen authored Dec 11, 2024
```
Signed-off-by: Hissu Hyvarinen <hissu.hyvarinen@amd.com>
```
  b2f77545
- [Doc] Update docs to refer to pooling models (#11093) · cad5c0a6
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cad5c0a6
- [Misc] Split up pooling tasks (#10820) · 8f10d5e3
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f10d5e3
- [Bugfix]: Clamp `-inf` logprob values in prompt_logprobs (#11073) · 40766ca1
  Rafael Vasquez authored Dec 11, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  40766ca1
- [Bugfix] Fix Idefics3 fails during multi-image inference (#11080) · 2e32f5d2
  B-201 authored Dec 11, 2024
```
Signed-off-by: B-201 <Joy25810@foxmail.com>
```
  2e32f5d2
- [Core] v1: Use atexit to handle engine core client shutdown (#11076) · 61b1d2f6
  Russell Bryant authored Dec 11, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  61b1d2f6
- [ci/build] Fix entrypoints test and pin outlines version (#11088) · 9974fca0
  Kevin H. Luu authored Dec 11, 2024
  
  9974fca0
- [ci/build] Fix AMD CI dependencies (#11087) · 3fb4b4f1
  Kevin H. Luu authored Dec 11, 2024
  
  3fb4b4f1
- [CI/Build] Check transformers v4.47 (#10991) · 2e33fe41
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  2e33fe41
- Fix streaming for granite tool call when <|tool_call|> is present (#11069) · e39400a4
  Maximilien de Bayser authored Dec 11, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  e39400a4
- [Model] PP support for Mamba-like models (#10992) · ffa48c91
  Mor Zusman authored Dec 11, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  ffa48c91
- [Misc] LoRA + Chunked Prefill (#9057) · d5c5154f
  Aurick Qiao authored Dec 10, 2024
  
  d5c5154f
- [Bugfix] Fix Mamba multistep (#11071) · 9a939737
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  9a939737
10 Dec, 2024 10 commits
- [V1][Bugfix] Always set enable_chunked_prefill = True for V1 (#11061) · 134810b3
  Woosuk Kwon authored Dec 10, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  134810b3
- [torch.compile] add a flag to track batchsize statistics (#11059) · 75f89dc4
  youkaichao authored Dec 10, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  75f89dc4
- [Core] Update to outlines >= 0.1.8 (#10576) · e7391949
  Russell Bryant authored Dec 10, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  e7391949
- [BUG] Remove token param #10921 (#11022) · 250ee65d
  Flávia Béo authored Dec 10, 2024
```
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
```
  250ee65d
- [Bugfix] Backport request id validation to v0 (#11036) · 9b9cef31
  Joe Runde authored Dec 10, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  9b9cef31
- [Misc][LoRA] Add PEFTHelper for LoRA (#11003) · d05f8867
  Jee Jee Li authored Dec 10, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  d05f8867
- [Bugfix] Handle <|tool_call|> token in granite tool parser (#11039) · beb16b2c
  Travis Johnson authored Dec 10, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  beb16b2c
- Add example of helm chart for vllm deployment on k8s (#9199) · fe2e10c7
  Maxime Fournioux authored Dec 10, 2024
```
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
```
  fe2e10c7
- [Bugfix] cuda error running llama 3.2 (#11047) · 82c73fd5
  Gene Der Su authored Dec 09, 2024
  
  82c73fd5
- Update README.md (#11034) · bfd61043
  Diego Marinho authored Dec 10, 2024
  
  bfd61043