Commits · b880ffb87e0bcde5e3693203b480df49e46d67bc · OpenDAS / vllm_cscc · GitLab

20 Dec, 2024 2 commits
- [Misc] Add tqdm progress bar during graph capture (#11349) · b880ffb8
  Michael Goin authored Dec 19, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  b880ffb8
- [Bugfix][Hardware][POWERPC] Fix auto dtype failure in case of POWER10 (#11331) · 48edab80
  Akash kaothalkar authored Dec 20, 2024
```
Signed-off-by: Akash Kaothalkar <0052v2@linux.vnet.ibm.com>
```
  48edab80
19 Dec, 2024 14 commits
- [Misc] Remove unused vllm/block.py (#11336) · e461c262
  yangzhibin authored Dec 20, 2024
  
  e461c262
- [Bugfix] Fix broken CPU compressed-tensors test (#11338) · 276738ce
  Isotr0py authored Dec 20, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  276738ce
- [Misc] Clean up and consolidate LRUCache (#11339) · cdf22afd
  Cyrus Leung authored Dec 20, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cdf22afd
- [Model] Refactor Qwen2-VL to use merged multimodal processor (#11258) · e24113a8
  Isotr0py authored Dec 20, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e24113a8
- [V1] Fix multimodal profiling for `Molmo` (#11325) · 7379b3d4
  Roger Wang authored Dec 19, 2024
```
Signed-off-by: ywang96 <ywang@example.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  7379b3d4
- [Model] Add JambaForSequenceClassification model (#10860) · 6c7f8815
  Yehoshua Cohen authored Dec 19, 2024
```
Signed-off-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6c7f8815
- [Bugfix] Cleanup Pixtral HF code (#11333) · a0f7d53b
  Cyrus Leung authored Dec 19, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a0f7d53b
- [Feature] Add load generation config from model (#11164) · 5aef4980
  Yanyi Liu authored Dec 19, 2024
```
Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  5aef4980
- [Misc] Optimize ray worker initialization time (#11275) · f26c4aee
  Rui Qiao authored Dec 18, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  f26c4aee
- [VLM] Merged multimodal processor for Qwen2-Audio (#11303) · 6142ef0a
  Cyrus Leung authored Dec 19, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6142ef0a
- [V1] Simplify prefix caching logic by removing `num_evictable_computed_blocks` (#11310) · c6b0a7d3
  Chen Zhang authored Dec 18, 2024
  
  c6b0a7d3
- [CI] Expand test_guided_generate to test all backends (#11313) · a30482f0
  Michael Goin authored Dec 18, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  a30482f0
- [Model] IBM Granite 3.1 (#11307) · 17ca9642
  Travis Johnson authored Dec 18, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  17ca9642
- [Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (#11311) · 5a9da2e6
  Tyler Michael Smith authored Dec 18, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  5a9da2e6
18 Dec, 2024 6 commits
- [V1] VLM - enable processor cache by default (#11305) · fdea8ec1
  Alexander Matveev authored Dec 18, 2024
```
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
```
  fdea8ec1
- [Bugfix] Fix broken phi3-v mm_processor_kwargs tests (#11263) · 996aa70f
  Isotr0py authored Dec 19, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  996aa70f
- [Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd
  Dipika Sikka authored Dec 18, 2024
```
Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  60508ffd
- [Bugfix] Fix guided decoding with tokenizer mode mistral (#11046) · 8b79f9e1
  Wallas Henrique authored Dec 18, 2024
  
  8b79f9e1
- [Bugfix] Restore support for larger block sizes (#11259) · 866fa455
  Konrad Zawora authored Dec 18, 2024
```
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
```
  866fa455
- [V1] Prefix caching for vision language models (#11187) · bf8717eb
  Cody Yu authored Dec 17, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  bf8717eb
17 Dec, 2024 8 commits
- [Bugfix] Fix request cancellation without polling (#11190) · 2d1b9baa
  Joe Runde authored Dec 17, 2024
  
  2d1b9baa
- [Misc] Allow passing logits_soft_cap for xformers backend (#11252) · f9ecbb18
  Isotr0py authored Dec 17, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  f9ecbb18
- [V1][Core] Use weakref.finalize instead of atexit (#11242) · 2bfdbf2a
  Tyler Michael Smith authored Dec 17, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  2bfdbf2a
- [Platform] platform agnostic for EngineArgs initialization (#11225) · e88db68c
  wangxiyuan authored Dec 17, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  e88db68c
- [V1][VLM] Proper memory profiling for image language models (#11210) · 59c9b6eb
  Roger Wang authored Dec 16, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  59c9b6eb
- [Frontend] Add OpenAI API support for input_audio (#11027) · 66d4b167
  kYLe authored Dec 17, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  66d4b167
- [CI] Add test case with JSON schema using references + use xgrammar by default... · 0064f697
  Michael Goin authored Dec 16, 2024
```
[CI] Add test case with JSON schema using references + use xgrammar by default with OpenAI parse (#10935)
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  0064f697
- [torch.compile] fast inductor (#11108) · 88a412ed
  youkaichao authored Dec 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  88a412ed
16 Dec, 2024 6 commits
- [core] overhaul memory profiling and fix backward compatibility (#10511) · 551603fe
  youkaichao authored Dec 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  551603fe
- [misc] Layerwise profile updates (#10242) · efbce85f
  Varun Sundar Rabindranath authored Dec 16, 2024
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  efbce85f
- [Model] Refactor Ultravox to use merged input processor (#11198) · d927dbcd
  Isotr0py authored Dec 16, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d927dbcd
- [Model] Support Cohere2ForCausalLM (Cohere R7B) (#11203) · bddbbcb1
  Jani Monoses authored Dec 16, 2024
  
  bddbbcb1
- [Bugfix] Fix the default value for temperature in ChatCompletionRequest (#11219) · 17138af7
  yansh97 authored Dec 16, 2024
  
  17138af7
- [Bugfix] Fix block size validation (#10938) · 69ba344d
  chenqianfzh authored Dec 15, 2024
  
  69ba344d
15 Dec, 2024 4 commits
- [V1][Minor] Cache np arange to reduce input preparation overhead (#11214) · 25ebed2f
  Woosuk Kwon authored Dec 15, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  25ebed2f
- [Core] Support disaggregated prefill with Mooncake Transfer Engine (#10884) · d263bd9d
  shangmingc authored Dec 16, 2024
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  d263bd9d
- [Bugfix] Fix error handling of unsupported sliding window (#11213) · 96d673e0
  Cyrus Leung authored Dec 16, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  96d673e0
- [Misc] Clean up multi-modal processor (#11207) · b10609e6
  Cyrus Leung authored Dec 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b10609e6