Commits · c31d4a57a6b639900a7c70b6e844db0116c2f9f6 · OpenDAS / vllm_cscc · GitLab

13 Dec, 2024 12 commits
- [Core] support LoRA and prompt adapter in content-based hashing for Block... · c31d4a57
  Sungjae Lee authored Dec 14, 2024
```
[Core] support LoRA and prompt adapter in content-based hashing for Block Manager v2 prefix caching (#8240)
```
  c31d4a57
- [Refactor]A simple device-related refactor (#11163) · d1fa714c
  Chenguang Li authored Dec 13, 2024
```
Signed-off-by: noemotiovon <noemotiovon@gmail.com>
Co-authored-by: noemotiovon <noemotiovon@gmail.com>
```
  d1fa714c
- [V1][VLM] Fix edge case bug for InternVL2 (#11165) · 969da7d7
  Roger Wang authored Dec 13, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  969da7d7
- [Frontend] Separate pooling APIs in offline inference (#11129) · eeec9e33
  Cyrus Leung authored Dec 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  eeec9e33
- PaliGemma 2 support (#11142) · 7cd74091
  Jani Monoses authored Dec 13, 2024
  
  7cd74091
- [core] clean up cudagraph batchsize padding logic (#10996) · be39e3cd
  youkaichao authored Dec 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  be39e3cd
- [Bugfix][V1] Fix 'NoneType' object has no attribute 'hash_value' (#11157) · 34f1a806
  Cody Yu authored Dec 12, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  34f1a806
- [ROCm][AMD] Disable auto enabling chunked prefill on ROCm (#11146) · 00c1bde5
  Gregory Shtrasberg authored Dec 13, 2024
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  00c1bde5
- [Bugfix] Update starcoder2 to remap k/v scale names for kv_cache quantization (#11148) · 3989a798
  Dipika Sikka authored Dec 13, 2024
  
  3989a798
- [Bugfix] Use runner_type instead of task in GritLM (#11144) · 1efce686
  Pooya Davoodi authored Dec 12, 2024
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  1efce686
- [torch.compile] Dynamic fp8 + rms_norm fusion (#10906) · 30870b4f
  Luka Govedič authored Dec 12, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  30870b4f
- [Misc][V1] Fix type in v1 prefix caching (#11151) · 78ed8f57
  Cody Yu authored Dec 12, 2024
  
  78ed8f57
12 Dec, 2024 13 commits
- [Bugfix] Fix value unpack error of simple connector for KVCache transfer. (#11058) · db6c264a
  shangmingc authored Dec 13, 2024
```
Signed-off-by: ShangmingCai <csmthu@gmail.com>
```
  db6c264a
- Fix logging of the vLLM Config (#11143) · 9f3974a3
  Jeremy Arnold authored Dec 12, 2024
  
  9f3974a3
- [Misc] Validate grammar and fail early (#11119) · 2c97eca1
  Cody Yu authored Dec 12, 2024
  
  2c97eca1
- [Bugfix] Quick fix to make Pixtral-HF load correctly again after 39e227c7 . (#11024) · 5d712571
  Jeff Cook authored Dec 12, 2024
  
  5d712571
- [V1] Fix torch profiling for offline inference (#11125) · 4816d20a
  Roger Wang authored Dec 12, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  4816d20a
- [Misc][LoRA] Ensure Lora Adapter requests return adapter name (#11094) · 85362f02
  Jiaxin Shan authored Dec 12, 2024
```
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  85362f02
- [core][distributed] initialization from StatelessProcessGroup (#10986) · 62de37a3
  youkaichao authored Dec 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  62de37a3
- [Hardware][Intel-Gaudi] Enable LoRA support for Intel Gaudi (HPU) (#10565) · 81958242
  Sanju C Sudhakaran authored Dec 12, 2024
```
Signed-off-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
```
  81958242
- [V1] Use more persistent buffers to optimize input preparation overheads (#11111) · f092153f
  Woosuk Kwon authored Dec 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f092153f
- [Model] Add support for embedding model GritLM (#10816) · 1da8f0e1
  Pooya Davoodi authored Dec 11, 2024
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  1da8f0e1
- [Core] cleanup zmq ipc sockets on exit (#11115) · ccede2b2
  Russell Bryant authored Dec 11, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  ccede2b2
- [Bugfix] Multiple fixes to tool streaming with hermes and mistral (#10979) · 7439a8b5
  Clayton authored Dec 11, 2024
```
Signed-off-by: cedonley <clayton@donley.io>
```
  7439a8b5
- [V1] VLM preprocessor hashing (#11020) · 4e116833
  Alexander Matveev authored Dec 11, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  4e116833
11 Dec, 2024 15 commits
- [V1][Core] Remove should_shutdown to simplify core process termination (#11113) · 452a723b
  Tyler Michael Smith authored Dec 11, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  452a723b
- [core] Bump ray to use _overlap_gpu_communication in compiled graph tests (#10410) · 72ff3a96
  Rui Qiao authored Dec 11, 2024
```
Signed-off-by: Rui Qiao <ubuntu@ip-172-31-15-128.us-west-2.compute.internal>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Rui Qiao <ubuntu@ip-172-31-15-128.us-west-2.compute.internal>
```
  72ff3a96
- [torch.compile] remove graph logging in ci (#11110) · 66aaa772
  youkaichao authored Dec 11, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  66aaa772
- [V1] Use input_ids as input for text-only models (#11032) · d643c2ab
  Woosuk Kwon authored Dec 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d643c2ab
- [torch.compile] use depyf to dump torch.compile internals (#10972) · 91642db9
  youkaichao authored Dec 11, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  91642db9
- [Doc] Update docs to refer to pooling models (#11093) · cad5c0a6
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cad5c0a6
- [Misc] Split up pooling tasks (#10820) · 8f10d5e3
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f10d5e3
- [Bugfix]: Clamp `-inf` logprob values in prompt_logprobs (#11073) · 40766ca1
  Rafael Vasquez authored Dec 11, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  40766ca1
- [Bugfix] Fix Idefics3 fails during multi-image inference (#11080) · 2e32f5d2
  B-201 authored Dec 11, 2024
```
Signed-off-by: B-201 <Joy25810@foxmail.com>
```
  2e32f5d2
- [Core] v1: Use atexit to handle engine core client shutdown (#11076) · 61b1d2f6
  Russell Bryant authored Dec 11, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  61b1d2f6
- [ci/build] Fix entrypoints test and pin outlines version (#11088) · 9974fca0
  Kevin H. Luu authored Dec 11, 2024
  
  9974fca0
- Fix streaming for granite tool call when <|tool_call|> is present (#11069) · e39400a4
  Maximilien de Bayser authored Dec 11, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  e39400a4
- [Model] PP support for Mamba-like models (#10992) · ffa48c91
  Mor Zusman authored Dec 11, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  ffa48c91
- [Misc] LoRA + Chunked Prefill (#9057) · d5c5154f
  Aurick Qiao authored Dec 10, 2024
  
  d5c5154f
- [Bugfix] Fix Mamba multistep (#11071) · 9a939737
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  9a939737