Commits · 996aa70f00818933866d8cfdbbf8f131a6a63664 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "6781a21e33e461720f533a25c33b40705ea78353"

18 Dec, 2024 5 commits
- [Bugfix] Fix broken phi3-v mm_processor_kwargs tests (#11263) · 996aa70f
  Isotr0py authored Dec 19, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  996aa70f
- [Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd
  Dipika Sikka authored Dec 18, 2024
```
Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  60508ffd
- [Bugfix] Fix guided decoding with tokenizer mode mistral (#11046) · 8b79f9e1
  Wallas Henrique authored Dec 18, 2024
  
  8b79f9e1
- [Bugfix] Restore support for larger block sizes (#11259) · 866fa455
  Konrad Zawora authored Dec 18, 2024
```
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
```
  866fa455
- [V1] Prefix caching for vision language models (#11187) · bf8717eb
  Cody Yu authored Dec 17, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  bf8717eb
17 Dec, 2024 8 commits
- [Bugfix] Fix request cancellation without polling (#11190) · 2d1b9baa
  Joe Runde authored Dec 17, 2024
  
  2d1b9baa
- [Misc] Allow passing logits_soft_cap for xformers backend (#11252) · f9ecbb18
  Isotr0py authored Dec 17, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  f9ecbb18
- [V1][Core] Use weakref.finalize instead of atexit (#11242) · 2bfdbf2a
  Tyler Michael Smith authored Dec 17, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  2bfdbf2a
- [Platform] platform agnostic for EngineArgs initialization (#11225) · e88db68c
  wangxiyuan authored Dec 17, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  e88db68c
- [V1][VLM] Proper memory profiling for image language models (#11210) · 59c9b6eb
  Roger Wang authored Dec 16, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  59c9b6eb
- [Frontend] Add OpenAI API support for input_audio (#11027) · 66d4b167
  kYLe authored Dec 17, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  66d4b167
- [CI] Add test case with JSON schema using references + use xgrammar by default... · 0064f697
  Michael Goin authored Dec 16, 2024
```
[CI] Add test case with JSON schema using references + use xgrammar by default with OpenAI parse (#10935)
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  0064f697
- [torch.compile] fast inductor (#11108) · 88a412ed
  youkaichao authored Dec 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  88a412ed
16 Dec, 2024 6 commits
- [core] overhaul memory profiling and fix backward compatibility (#10511) · 551603fe
  youkaichao authored Dec 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  551603fe
- [misc] Layerwise profile updates (#10242) · efbce85f
  Varun Sundar Rabindranath authored Dec 16, 2024
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  efbce85f
- [Model] Refactor Ultravox to use merged input processor (#11198) · d927dbcd
  Isotr0py authored Dec 16, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d927dbcd
- [Model] Support Cohere2ForCausalLM (Cohere R7B) (#11203) · bddbbcb1
  Jani Monoses authored Dec 16, 2024
  
  bddbbcb1
- [Bugfix] Fix the default value for temperature in ChatCompletionRequest (#11219) · 17138af7
  yansh97 authored Dec 16, 2024
  
  17138af7
- [Bugfix] Fix block size validation (#10938) · 69ba344d
  chenqianfzh authored Dec 15, 2024
  
  69ba344d
15 Dec, 2024 6 commits
- [V1][Minor] Cache np arange to reduce input preparation overhead (#11214) · 25ebed2f
  Woosuk Kwon authored Dec 15, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  25ebed2f
- [Core] Support disaggregated prefill with Mooncake Transfer Engine (#10884) · d263bd9d
  shangmingc authored Dec 16, 2024
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  d263bd9d
- [Bugfix] Fix error handling of unsupported sliding window (#11213) · 96d673e0
  Cyrus Leung authored Dec 16, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  96d673e0
- [Misc] Clean up multi-modal processor (#11207) · b10609e6
  Cyrus Leung authored Dec 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b10609e6
- [torch.compile] allow tracking forward time (#11081) · a1c02058
  youkaichao authored Dec 14, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  a1c02058
- [[Misc]Upgrade bitsandbytes to the latest version 0.45.0 (#11201) · 15859f23
  Jee Jee Li authored Dec 15, 2024
  
  15859f23
14 Dec, 2024 10 commits
- [Performance][Core] Optimize the performance of evictor v1 and v2 by applying... · 88693683
  Sungjae Lee authored Dec 15, 2024
```
[Performance][Core] Optimize the performance of evictor v1 and v2 by applying a priority queue and lazy deletion (#7209)
```
  88693683
- Enable mypy checking on V1 code (#11105) · 6d917d0e
  Mark McLoughlin authored Dec 14, 2024
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  6d917d0e
- [VLM] Fully dynamic prompt replacement in merged input processor (#11199) · 93abf23a
  Cyrus Leung authored Dec 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  93abf23a
- [Frontend] Add `logits_processors` as an extra completion argument (#11150) · 9c3dadd1
  Brad Hilton authored Dec 14, 2024
```
Signed-off-by: Brad Hilton <brad.hilton.nw@gmail.com>
```
  9c3dadd1
- [Misc] Minor improvements to the readability of PunicaWrapperBase (#11200) · 3cb57698
  Jee Jee Li authored Dec 15, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  3cb57698
- [V1][Bugfix] Fix V1 TP trust-remote-code (#11182) · ea7bd68d
  Tyler Michael Smith authored Dec 14, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  ea7bd68d
- [Core] Update outlines and increase its threadpool size (#11140) · 48259264
  Russell Bryant authored Dec 14, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  48259264
- [Bugfix][V1] Re-compute an entire block when fully cache hit (#11186) · 9855aea2
  Cody Yu authored Dec 13, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  9855aea2
- [V1][Bugfix] Fix EngineCoreProc profile (#11185) · 4b5b8a6a
  Tyler Michael Smith authored Dec 13, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  4b5b8a6a
- [Core] V1: Use multiprocessing by default (#11074) · 4863e5fb
  Russell Bryant authored Dec 13, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  4863e5fb
13 Dec, 2024 5 commits
- [Distributed] Allow the placement group more time to wait for resources to be ready (#11138) · 0d8451c3
  Jiaxin Shan authored Dec 13, 2024
```
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
```
  0d8451c3
- [Bugfix][Hardware][CPU] Enable Gemma2 with SDPA on CPU backend (#11169) · 0a56bcc0
  Jani Monoses authored Dec 13, 2024
  
  0a56bcc0
- [Doc] Reorganize online pooling APIs (#11172) · 0920ab91
  Cyrus Leung authored Dec 14, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0920ab91
- [Bugfix] using len(tokenizer) instead of tokenizer.vocab_size in... · 5b0ed839
  zhangjf authored Dec 13, 2024
```
[Bugfix] using len(tokenizer) instead of tokenizer.vocab_size in AllowedTokenIdsLogitsProcessor (#11156)
```
  5b0ed839
- [Core] support LoRA and prompt adapter in content-based hashing for Block... · c31d4a57
  Sungjae Lee authored Dec 14, 2024
```
[Core] support LoRA and prompt adapter in content-based hashing for Block Manager v2 prefix caching (#8240)
```
  c31d4a57