Commits · a985f7af9f7b249974b283a9d999575ac30fac3d · OpenDAS / vllm_cscc

"docs/features/quantization/bnb.md" did not exist on "3892e58ad7c0f325c9257cb3468aa1cde9e1ed6f"

19 Dec, 2024 17 commits
- [CI] Adding CPU docker pipeline (#11261) · a985f7af
  Yuan authored Dec 20, 2024
```
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
Co-authored-by: Kevin H. Luu <kevin@anyscale.com>
```
  a985f7af
- [Misc] Remove unused vllm/block.py (#11336) · e461c262
  yangzhibin authored Dec 20, 2024
  
  e461c262
- [Bugfix] Fix broken CPU compressed-tensors test (#11338) · 276738ce
  Isotr0py authored Dec 20, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  276738ce
- [Misc] Clean up and consolidate LRUCache (#11339) · cdf22afd
  Cyrus Leung authored Dec 20, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cdf22afd
- [Model] Refactor Qwen2-VL to use merged multimodal processor (#11258) · e24113a8
  Isotr0py authored Dec 20, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e24113a8
- [V1] Fix multimodal profiling for `Molmo` (#11325) · 7379b3d4
  Roger Wang authored Dec 19, 2024
```
Signed-off-by: ywang96 <ywang@example.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  7379b3d4
- [Model] Add JambaForSequenceClassification model (#10860) · 6c7f8815
  Yehoshua Cohen authored Dec 19, 2024
```
Signed-off-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6c7f8815
- [Bugfix] Cleanup Pixtral HF code (#11333) · a0f7d53b
  Cyrus Leung authored Dec 19, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a0f7d53b
- [Feature] Add load generation config from model (#11164) · 5aef4980
  Yanyi Liu authored Dec 19, 2024
```
Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  5aef4980
- [misc] benchmark_throughput : Add LoRA (#11267) · 98356735
  Varun Sundar Rabindranath authored Dec 19, 2024
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  98356735
- [Misc] Optimize ray worker initialization time (#11275) · f26c4aee
  Rui Qiao authored Dec 18, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  f26c4aee
- [Kernel] Refactor Cutlass c3x (#10049) · 8936316d
  Varun Sundar Rabindranath authored Dec 19, 2024
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  8936316d
- [VLM] Merged multimodal processor for Qwen2-Audio (#11303) · 6142ef0a
  Cyrus Leung authored Dec 19, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6142ef0a
- [V1] Simplify prefix caching logic by removing `num_evictable_computed_blocks` (#11310) · c6b0a7d3
  Chen Zhang authored Dec 18, 2024
  
  c6b0a7d3
- [CI] Expand test_guided_generate to test all backends (#11313) · a30482f0
  Michael Goin authored Dec 18, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  a30482f0
- [Model] IBM Granite 3.1 (#11307) · 17ca9642
  Travis Johnson authored Dec 18, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  17ca9642
- [Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (#11311) · 5a9da2e6
  Tyler Michael Smith authored Dec 18, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  5a9da2e6
18 Dec, 2024 11 commits
- [V1] VLM - enable processor cache by default (#11305) · fdea8ec1
  Alexander Matveev authored Dec 18, 2024
```
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
```
  fdea8ec1
- [Bugfix] fix minicpmv test (#11304) · ca5f54a9
  Joe Runde authored Dec 18, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  ca5f54a9
- [FIX] update openai version (#11287) · f954fe0e
  Kunshang Ji authored Dec 19, 2024
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  f954fe0e
- [CI][Misc] Remove Github Action Release Workflow (#11274) · 362cff1e
  Simon Mo authored Dec 18, 2024
  
  362cff1e
- [Bugfix] Fix broken phi3-v mm_processor_kwargs tests (#11263) · 996aa70f
  Isotr0py authored Dec 19, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  996aa70f
- [Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd
  Dipika Sikka authored Dec 18, 2024
```
Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  60508ffd
- [MISC][XPU]update ipex link for CI fix (#11278) · f04e407e
  Yan Ma authored Dec 18, 2024
  
  f04e407e
- [Bugfix] Fix guided decoding with tokenizer mode mistral (#11046) · 8b79f9e1
  Wallas Henrique authored Dec 18, 2024
  
  8b79f9e1
- [Bugfix] Restore support for larger block sizes (#11259) · 866fa455
  Konrad Zawora authored Dec 18, 2024
```
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
```
  866fa455
- [V1] Prefix caching for vision language models (#11187) · bf8717eb
  Cody Yu authored Dec 17, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  bf8717eb
- [Bugfix] Set temperature=0.7 in test_guided_choice_chat (#11264) · c77eb8a3
  Michael Goin authored Dec 17, 2024
  
  c77eb8a3
17 Dec, 2024 10 commits
- [Bugfix] Fix request cancellation without polling (#11190) · 2d1b9baa
  Joe Runde authored Dec 17, 2024
  
  2d1b9baa
- [Misc] Allow passing logits_soft_cap for xformers backend (#11252) · f9ecbb18
  Isotr0py authored Dec 17, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  f9ecbb18
- [Misc] Kernel Benchmark for `RMSNorm` (#11241) · 02222a02
  Roger Wang authored Dec 16, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Xiaoyu Zhang <BBuf@users.noreply.github.com>
```
  02222a02
- [V1][Core] Use weakref.finalize instead of atexit (#11242) · 2bfdbf2a
  Tyler Michael Smith authored Dec 17, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  2bfdbf2a
- [Platform] platform agnostic for EngineArgs initialization (#11225) · e88db68c
  wangxiyuan authored Dec 17, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  e88db68c
- [V1][VLM] Proper memory profiling for image language models (#11210) · 59c9b6eb
  Roger Wang authored Dec 16, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  59c9b6eb
- [Frontend] Add OpenAI API support for input_audio (#11027) · 66d4b167
  kYLe authored Dec 17, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  66d4b167
- [CI] Add test case with JSON schema using references + use xgrammar by default... · 0064f697
  Michael Goin authored Dec 16, 2024
```
[CI] Add test case with JSON schema using references + use xgrammar by default with OpenAI parse (#10935)
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  0064f697
- fix gh200 tests on main (#11246) · 35bae114
  youkaichao authored Dec 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  35bae114
- [torch.compile] fast inductor (#11108) · 88a412ed
  youkaichao authored Dec 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  88a412ed
16 Dec, 2024 2 commits

[ci][tests] add gh200 tests (#11244) · c301616e
youkaichao authored Dec 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
c301616e

[Docs] hint to enable use of GPU performance counters in profiling tools for... · 35ffa682

bk-TurbaAI authored Dec 16, 2024


[Docs] hint to enable use of GPU performance counters in profiling tools for multi-node distributed serving (#11235)
Co-authored-by: Michael Goin <michael@neuralmagic.com>

35ffa682