Commits · 1bf905ddaa969e6458fe0d15a1db80318f39fade · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "ebe4d1db3a42096cebcc2b2d289143bc0ef02d3d"

27 Nov, 2024 3 commits
- [Bugfix][SpecDecode] apply sampling parameters to target probabilities for... · 1bf905dd
  jeongin601 authored Nov 27, 2024
```
[Bugfix][SpecDecode] apply sampling parameters to target probabilities for consistency in rejection sampling. (#10198)
Signed-off-by: jeongin601 <0200angela@gmail.com>
Signed-off-by: jeong_in.bae <jeong_in.bae@navercorp.com>
```
  1bf905dd
- [V1] Update interface for idefics3 (#10680) · 0a4d9685
  Roger Wang authored Nov 26, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  0a4d9685
- Remove hard-dependencies of Speculative decode to CUDA workers (#10587) · 0a71900b
  Chendi.Xue authored Nov 26, 2024
```
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
```
  0a71900b
26 Nov, 2024 11 commits
- [V1] Refactor model executable interface for multimodal models (#10570) · 2f0a0a17
  Roger Wang authored Nov 26, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  2f0a0a17
- [Bugfix] Check bnb_4bit_quant_storage for bitsandbytes (#10642) · 7576cd38
  Michael Goin authored Nov 26, 2024
  
  7576cd38
- [Bugfix] Fix using `-O[0,3]` with LLM entrypoint (#10677) · 9a99273b
  Michael Goin authored Nov 26, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  9a99273b
- [Hardware][NVIDIA] Add non-NVML CUDA mode for Jetson (#9735) · f5792c7c
  Conroy Cheers authored Nov 27, 2024
```
Signed-off-by: Conroy Cheers <conroy@corncheese.org>
```
  f5792c7c
- [Bugfix] Fix for Spec model TP + Chunked Prefill (#10232) · db66e018
  Murali Andoorveedu authored Nov 26, 2024
```
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
Signed-off-by: Sourashis Roy <sroy@roblox.com>
Co-authored-by: Sourashis Roy <sroy@roblox.com>
```
  db66e018
- [V1] Enable profile for LLMEngine (#10665) · 1f6584ee
  Kunshang Ji authored Nov 26, 2024
  
  1f6584ee
- [Misc] Remove outdated init protocols (#10655) · 94063534
  Cyrus Leung authored Nov 26, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  94063534
- custom allreduce + torch.compile (#10121) · 9a88f897
  Sage Moore authored Nov 26, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  9a88f897
- [v1] EngineArgs for better config handling for v1 (#10382) · 519e8e41
  Ricky Xu authored Nov 25, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  519e8e41
- [bugfix] fix aria model and add torch.compile (#10645) · 45ac4ff2
  youkaichao authored Nov 25, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  45ac4ff2
- [misc] do not read HOST_IP (#10644) · 6e9ff050
  youkaichao authored Nov 25, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6e9ff050
25 Nov, 2024 13 commits
- [Model] Add OLMo November 2024 model (#10503) · 9db713a1
  Shane A authored Nov 25, 2024
  
  9db713a1
- [Model] Enable optional prefix when loading embedding models (#10639) · cf73f0c9
  Cyrus Leung authored Nov 26, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cf73f0c9
- [Model]: Add support for Aria model (#10514) · b1d92053
  zhou fan authored Nov 26, 2024
```
Signed-off-by: xffxff <1247714429@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  b1d92053
- [Bugfix] Fix chunked prefill with model dtype float32 on Turing Devices (#9850) · c27df94e
  Wallas Henrique authored Nov 25, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  c27df94e
- [Bug]: Authorization ignored when root_path is set (#10606) · d04b13a3
  Chauncey authored Nov 26, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  d04b13a3
- [Model] Support `is_causal` HF config field for Qwen2 model (#10621) · ed46f143
  Cyrus Leung authored Nov 25, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ed46f143
- [misc] move functions to config.py (#10624) · 05d1f8c9
  youkaichao authored Nov 25, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  05d1f8c9
- [misc] add torch.compile compatibility check (#10618) · 25d806e9
  youkaichao authored Nov 24, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  25d806e9
- [torch.compile] add warning for unsupported models (#10622) · 65813781
  youkaichao authored Nov 24, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  65813781
- [torch.compile] force inductor threads (#10620) · 7c2134be
  Jee Jee Li authored Nov 25, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  7c2134be
- [torch.compile] support encoder based models (#10613) · 571841b7
  youkaichao authored Nov 24, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  571841b7
- [Refactor][MISC] del redundant code in ParallelConfig.postinit (#10614) · 7ea3cd7c
  Mengqing Cao authored Nov 25, 2024
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  7ea3cd7c
- Support Cross encoder models (#10400) · 214efc2c
  Maximilien de Bayser authored Nov 24, 2024
```
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
```
  214efc2c
24 Nov, 2024 3 commits
- [model][utils] add extract_layer_index utility function (#10599) · c0557478
  youkaichao authored Nov 23, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c0557478
- [Bugfix] Fix LoRA weight sharding (#10450) · 1700c543
  Jee Jee Li authored Nov 24, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  1700c543
- [bugfix] Fix example/tensorize_vllm_model tests (#10595) · 17d8fc18
  Jee Jee Li authored Nov 24, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  17d8fc18
23 Nov, 2024 10 commits
- [Bugfix] Avoid import AttentionMetadata explicitly in Mllama (#10593) · 04668ebe
  Isotr0py authored Nov 24, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  04668ebe
- [Platforms] Refactor openvino code (#10573) · 86a44fb8
  JiHuazhong authored Nov 23, 2024
```
Signed-off-by: statelesshz <hzji210@gmail.com>
```
  86a44fb8
- [Bugfix] `multi_modal_kwargs` broadcast for CPU tensor parallel (#10541) · 4cfe5d2b
  Isotr0py authored Nov 23, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  4cfe5d2b
- [2/N] handling placeholders in merged multi-modal processor (#10485) · c8acd805
  Cyrus Leung authored Nov 23, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c8acd805
- Prefix Cache Aware Scheduling [1/n] (#10128) · 4634a89d
  Ricky Xu authored Nov 22, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  4634a89d
- [AMD] Add support for GGUF quantization on ROCm (#10254) · 7c25fe45
  kliuae authored Nov 23, 2024
  
  7c25fe45
- Update default max_num_batch_tokens for chunked prefill to 2048 (#10544) · 02a43f82
  Michael Goin authored Nov 23, 2024
  
  02a43f82
- [Model] Fix Baichuan BNB online quantization (#10572) · cfea9c04
  Chen Wu authored Nov 23, 2024
```
Signed-off-by: Chen Wu <cntryroa@gmail.com>
```
  cfea9c04
- [Bugfix] Internal Server Error when tool_choice is incorrect. (#10567) · 7d8ffb34
  Varun Vinayak Shenoy authored Nov 22, 2024
```
Signed-off-by: Varun Shenoy <varun.vinayak.shenoy@oracle.com>
```
  7d8ffb34
- [core] gemma2 full context length support (#10584) · 4aba6e3d
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4aba6e3d