Commits · cf73f0c95e09836efff876d5bfd9b9c6cc1ba06e · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "470dc4152e50a8875c72eb30f207ec7495c21b5d"

25 Nov, 2024 12 commits
- [Model] Enable optional prefix when loading embedding models (#10639) · cf73f0c9
  Cyrus Leung authored Nov 26, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cf73f0c9
- [Model]: Add support for Aria model (#10514) · b1d92053
  zhou fan authored Nov 26, 2024
```
Signed-off-by: xffxff <1247714429@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  b1d92053
- [Bugfix] Fix chunked prefill with model dtype float32 on Turing Devices (#9850) · c27df94e
  Wallas Henrique authored Nov 25, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  c27df94e
- [Bug]: Authorization ignored when root_path is set (#10606) · d04b13a3
  Chauncey authored Nov 26, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  d04b13a3
- [Model] Support `is_causal` HF config field for Qwen2 model (#10621) · ed46f143
  Cyrus Leung authored Nov 25, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ed46f143
- [misc] move functions to config.py (#10624) · 05d1f8c9
  youkaichao authored Nov 25, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  05d1f8c9
- [misc] add torch.compile compatibility check (#10618) · 25d806e9
  youkaichao authored Nov 24, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  25d806e9
- [torch.compile] add warning for unsupported models (#10622) · 65813781
  youkaichao authored Nov 24, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  65813781
- [torch.compile] force inductor threads (#10620) · 7c2134be
  Jee Jee Li authored Nov 25, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  7c2134be
- [torch.compile] support encoder based models (#10613) · 571841b7
  youkaichao authored Nov 24, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  571841b7
- [Refactor][MISC] del redundant code in ParallelConfig.postinit (#10614) · 7ea3cd7c
  Mengqing Cao authored Nov 25, 2024
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  7ea3cd7c
- Support Cross encoder models (#10400) · 214efc2c
  Maximilien de Bayser authored Nov 24, 2024
```
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
```
  214efc2c
24 Nov, 2024 3 commits
- [model][utils] add extract_layer_index utility function (#10599) · c0557478
  youkaichao authored Nov 23, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c0557478
- [Bugfix] Fix LoRA weight sharding (#10450) · 1700c543
  Jee Jee Li authored Nov 24, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  1700c543
- [bugfix] Fix example/tensorize_vllm_model tests (#10595) · 17d8fc18
  Jee Jee Li authored Nov 24, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  17d8fc18
23 Nov, 2024 16 commits
- [Bugfix] Avoid import AttentionMetadata explicitly in Mllama (#10593) · 04668ebe
  Isotr0py authored Nov 24, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  04668ebe
- [Platforms] Refactor openvino code (#10573) · 86a44fb8
  JiHuazhong authored Nov 23, 2024
```
Signed-off-by: statelesshz <hzji210@gmail.com>
```
  86a44fb8
- [Bugfix] `multi_modal_kwargs` broadcast for CPU tensor parallel (#10541) · 4cfe5d2b
  Isotr0py authored Nov 23, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  4cfe5d2b
- [2/N] handling placeholders in merged multi-modal processor (#10485) · c8acd805
  Cyrus Leung authored Nov 23, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c8acd805
- Prefix Cache Aware Scheduling [1/n] (#10128) · 4634a89d
  Ricky Xu authored Nov 22, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  4634a89d
- [AMD] Add support for GGUF quantization on ROCm (#10254) · 7c25fe45
  kliuae authored Nov 23, 2024
  
  7c25fe45
- Update default max_num_batch_tokens for chunked prefill to 2048 (#10544) · 02a43f82
  Michael Goin authored Nov 23, 2024
  
  02a43f82
- [Model] Fix Baichuan BNB online quantization (#10572) · cfea9c04
  Chen Wu authored Nov 23, 2024
```
Signed-off-by: Chen Wu <cntryroa@gmail.com>
```
  cfea9c04
- [Bugfix] Internal Server Error when tool_choice is incorrect. (#10567) · 7d8ffb34
  Varun Vinayak Shenoy authored Nov 22, 2024
```
Signed-off-by: Varun Shenoy <varun.vinayak.shenoy@oracle.com>
```
  7d8ffb34
- [core] gemma2 full context length support (#10584) · 4aba6e3d
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4aba6e3d
- [Misc] Add pynccl wrappers for all_gather and reduce_scatter (#9432) · 978b3974
  Tyler Michael Smith authored Nov 22, 2024
  
  978b3974
- [Core] Fix broken log configuration (#10458) · ebda5196
  Russell Bryant authored Nov 22, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  ebda5196
- [bugfix] fix cpu tests (#10585) · d559979c
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  d559979c
- [V1] EngineCore supports profiling (#10564) · d345f409
  Zhonghua Deng authored Nov 23, 2024
```
Signed-off-by: Abatom <abzhonghua@gmail.com>
```
  d345f409
- [Core] remove temporary local variables in LLMEngine.__init__ (#10577) · 28598f39
  Russell Bryant authored Nov 22, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  28598f39
- support bitsandbytes quantization with qwen model (#10549) · 948c8595
  zixuanzhang226 authored Nov 22, 2024
```
Signed-off-by: Ubuntu <zixuanzhang@bytedance.com>
```
  948c8595
22 Nov, 2024 7 commits
- [v1] Refactor KVCacheManager for more hash input than token ids (#10507) · 97814fbf
  Ricky Xu authored Nov 22, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  97814fbf
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
- Remove token-adding chat embedding params (#10551) · 11fcf0e0
  Noam Gat authored Nov 22, 2024
```
Signed-off-by: Noam Gat <noamgat@gmail.com>
```
  11fcf0e0
- [Bugfix] Fix Phi-3 BNB quantization with tensor parallel (#9948) · b6374e09
  Isotr0py authored Nov 22, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b6374e09
- [platforms] absorb worker cls difference into platforms folder (#10555) · a111d015
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  a111d015
- [Minor] Fix line-too-long (#10563) · 446c7806
  Woosuk Kwon authored Nov 21, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  446c7806
- [9/N] torch.compile LLM usage (#10552) · 33e0a254
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  33e0a254
21 Nov, 2024 2 commits
- [misc] improve error message (#10553) · cf656f5a
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  cf656f5a
- [V1] Fix Compilation config & Enable CUDA graph by default (#10528) · f9310cbd
  Woosuk Kwon authored Nov 21, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f9310cbd