Commits · 334d64d1e816cc7c9fa2f67e22d24638e63c8e15 · OpenDAS / vllm_cscc

26 Nov, 2024 3 commits
- [ci] add vllm_test_utils (#10659) · 334d64d1
  youkaichao authored Nov 26, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  334d64d1
- custom allreduce + torch.compile (#10121) · 9a88f897
  Sage Moore authored Nov 26, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  9a88f897
- [v1] EngineArgs for better config handling for v1 (#10382) · 519e8e41
  Ricky Xu authored Nov 25, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  519e8e41
25 Nov, 2024 9 commits
- [Model] Add OLMo November 2024 model (#10503) · 9db713a1
  Shane A authored Nov 25, 2024
  
  9db713a1
- [Model]: Add support for Aria model (#10514) · b1d92053
  zhou fan authored Nov 26, 2024
```
Signed-off-by: xffxff <1247714429@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  b1d92053
- [Bugfix] Fix chunked prefill with model dtype float32 on Turing Devices (#9850) · c27df94e
  Wallas Henrique authored Nov 25, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  c27df94e
- [Bug]: Authorization ignored when root_path is set (#10606) · d04b13a3
  Chauncey authored Nov 26, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  d04b13a3
- [Model] Support `is_causal` HF config field for Qwen2 model (#10621) · ed46f143
  Cyrus Leung authored Nov 25, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ed46f143
- [misc] move functions to config.py (#10624) · 05d1f8c9
  youkaichao authored Nov 25, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  05d1f8c9
- [misc] add torch.compile compatibility check (#10618) · 25d806e9
  youkaichao authored Nov 24, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  25d806e9
- [torch.compile] support encoder based models (#10613) · 571841b7
  youkaichao authored Nov 24, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  571841b7
- Support Cross encoder models (#10400) · 214efc2c
  Maximilien de Bayser authored Nov 24, 2024
```
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
```
  214efc2c
24 Nov, 2024 1 commit

[Bugfix] Fix LoRA weight sharding (#10450) · 1700c543

Jee Jee Li authored Nov 24, 2024


Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

1700c543

23 Nov, 2024 6 commits
- [2/N] handling placeholders in merged multi-modal processor (#10485) · c8acd805
  Cyrus Leung authored Nov 23, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c8acd805
- Prefix Cache Aware Scheduling [1/n] (#10128) · 4634a89d
  Ricky Xu authored Nov 22, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  4634a89d
- [Bugfix] Internal Server Error when tool_choice is incorrect. (#10567) · 7d8ffb34
  Varun Vinayak Shenoy authored Nov 22, 2024
```
Signed-off-by: Varun Shenoy <varun.vinayak.shenoy@oracle.com>
```
  7d8ffb34
- [core] gemma2 full context length support (#10584) · 4aba6e3d
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4aba6e3d
- [Misc] Add pynccl wrappers for all_gather and reduce_scatter (#9432) · 978b3974
  Tyler Michael Smith authored Nov 22, 2024
  
  978b3974
- [Bugfix][Frontend] Update Llama Chat Templates to also support Non-Tool use (#10164) · 9195dbdb
  Travis Johnson authored Nov 22, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  9195dbdb
22 Nov, 2024 4 commits
- [v1] Refactor KVCacheManager for more hash input than token ids (#10507) · 97814fbf
  Ricky Xu authored Nov 22, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  97814fbf
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
- [bugfix] fix full graph tests (#10581) · db100c5c
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  db100c5c
- [9/N] torch.compile LLM usage (#10552) · 33e0a254
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  33e0a254
21 Nov, 2024 7 commits
- [8/N] enable cli flag without a space (#10529) · 7560ae5c
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7560ae5c
- [Kernel] Register punica ops directly (#10522) · 2385b60d
  Jee Jee Li authored Nov 22, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  2385b60d
- [Bug]: When apply continue_final_message for OpenAI server, the "echo":false is ignored (#10180) · da7e702c
  Chauncey authored Nov 22, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  da7e702c
- [Model] Expose `dynamic_image_size` as mm_processor_kwargs for InternVL2 models (#10518) · d5ec121f
  Isotr0py authored Nov 21, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  d5ec121f
- [torch.compile] Inductor code caching fix (#10273) · 8b0fe06c
  Luka Govedič authored Nov 21, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
```
  8b0fe06c
- [Core] Add Sliding Window Support with Flashinfer (#10462) · 6c1208d0
  Pavani Majety authored Nov 20, 2024
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  6c1208d0
- [torch.compile] limit inductor threads and lazy import quant (#10482) · 388ee3de
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  388ee3de
20 Nov, 2024 4 commits
- [Bugfix]: allow extra fields in requests to openai compatible server (#10463) · c68f7ede
  Guillaume Calmettes authored Nov 20, 2024
```
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  c68f7ede
- [7/N] torch.compile, reduce compilation time (#10460) · 0cd3d971
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  0cd3d971
- [Hardware][CPU] Support chunked-prefill and prefix-caching on CPU (#10355) · 63f1fde2
  Li, Jiang authored Nov 20, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  63f1fde2
- [Bugfix] Marlin 2:4 temp fix for large M dim (>256) (#10464) · d200972e
  Lucas Wilkinson authored Nov 19, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  d200972e
19 Nov, 2024 3 commits
- [Model][Quantization] HQQ support through Marlin kernel expansion (#9766) · b00b33d7
  ElizaWszola authored Nov 19, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
  b00b33d7
- [6/N] torch.compile rollout to users (#10437) · 803f37ea
  youkaichao authored Nov 19, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  803f37ea
- [Platform][Refactor] Extract func `get_default_attn_backend` to `Platform` (#10358) · 8c1fb507
  Mengqing Cao authored Nov 19, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  8c1fb507
18 Nov, 2024 3 commits
- [Kernel] Initial Machete W4A8 support + Refactors (#9855) · 96d999fb
  Lucas Wilkinson authored Nov 18, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  96d999fb
- [Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) · 6b2d25ef
  Yan Ma authored Nov 19, 2024
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  6b2d25ef
- [VLM] Report multi_modal_placeholders in output (#10407) · c7dec926
  lkchen authored Nov 18, 2024
```
Signed-off-by: Linkun Chen <lkchen+anyscale@github.com>
```
  c7dec926