Commits · 278be671a355ea89843141928a426a303bfd8036 · OpenDAS / vllm_cscc · GitLab

28 Nov, 2024 1 commit
- [Model] support bitsandbytes quantization with minicpm3 model (#10682) · 70dc14fb
  zixuanzhang226 authored Nov 27, 2024
```
Signed-off-by: Ubuntu <zixuanzhang@bytedance.com>
```
  70dc14fb
27 Nov, 2024 13 commits
- [Frontend] don't block event loop in tokenization (preprocess) in OpenAI compatible server (#10635) · 395b1c74
  tomeras91 authored Nov 27, 2024
```
Signed-off-by: Tomer Asida <tomera@ai21.com>
```
  395b1c74
- [Bugfix] Ignore `lm_head` when loading embedding models (#10719) · 9b4b1503
  Cyrus Leung authored Nov 28, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9b4b1503
- [Bugfix][Mamba] Fix Multistep on Mamba-like models (#10705) · 197b4484
  Mor Zusman authored Nov 27, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  197b4484
- [Bugfix] Fix GGUF inference with FP16 unquantized checkpoint (#10675) · b98c62ba
  Isotr0py authored Nov 28, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b98c62ba
- [torch.compile] fix shape specialization (#10722) · c411def2
  youkaichao authored Nov 27, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c411def2
- [V1] Update interface for mistral-format Pixtral (#10703) · 9e0a147d
  Roger Wang authored Nov 27, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  9e0a147d
- [Model] Support telechat2 (#10311) · 1209261e
  shunxing12345 authored Nov 27, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: xiangw2 <xiangw2@chinatelecom.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  1209261e
- [Misc]Further reduce BNB static variable (#10597) · 15cc2a9f
  Jee Jee Li authored Nov 27, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  15cc2a9f
- [Hardware][Gaudi]add get_name method for HPUAttentionBackend (#10667) · e85250b1
  Kunshang Ji authored Nov 27, 2024
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  e85250b1
- [bugfix] fix the default value of llm_int8_threshold in BitsAndBytesConfig (#10657) · cfb3bf25
  yansh97 authored Nov 27, 2024
  
  cfb3bf25
- [Bugfix][SpecDecode] apply sampling parameters to target probabilities for... · 1bf905dd
  jeongin601 authored Nov 27, 2024
```
[Bugfix][SpecDecode] apply sampling parameters to target probabilities for consistency in rejection sampling. (#10198)
Signed-off-by: jeongin601 <0200angela@gmail.com>
Signed-off-by: jeong_in.bae <jeong_in.bae@navercorp.com>
```
  1bf905dd
- [V1] Update interface for idefics3 (#10680) · 0a4d9685
  Roger Wang authored Nov 26, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  0a4d9685
- Remove hard-dependencies of Speculative decode to CUDA workers (#10587) · 0a71900b
  Chendi.Xue authored Nov 26, 2024
```
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
```
  0a71900b
26 Nov, 2024 11 commits
- [V1] Refactor model executable interface for multimodal models (#10570) · 2f0a0a17
  Roger Wang authored Nov 26, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  2f0a0a17
- [Bugfix] Check bnb_4bit_quant_storage for bitsandbytes (#10642) · 7576cd38
  Michael Goin authored Nov 26, 2024
  
  7576cd38
- [Bugfix] Fix using `-O[0,3]` with LLM entrypoint (#10677) · 9a99273b
  Michael Goin authored Nov 26, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  9a99273b
- [Hardware][NVIDIA] Add non-NVML CUDA mode for Jetson (#9735) · f5792c7c
  Conroy Cheers authored Nov 27, 2024
```
Signed-off-by: Conroy Cheers <conroy@corncheese.org>
```
  f5792c7c
- [Bugfix] Fix for Spec model TP + Chunked Prefill (#10232) · db66e018
  Murali Andoorveedu authored Nov 26, 2024
```
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
Signed-off-by: Sourashis Roy <sroy@roblox.com>
Co-authored-by: Sourashis Roy <sroy@roblox.com>
```
  db66e018
- [V1] Enable profile for LLMEngine (#10665) · 1f6584ee
  Kunshang Ji authored Nov 26, 2024
  
  1f6584ee
- [Misc] Remove outdated init protocols (#10655) · 94063534
  Cyrus Leung authored Nov 26, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  94063534
- custom allreduce + torch.compile (#10121) · 9a88f897
  Sage Moore authored Nov 26, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  9a88f897
- [v1] EngineArgs for better config handling for v1 (#10382) · 519e8e41
  Ricky Xu authored Nov 25, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  519e8e41
- [bugfix] fix aria model and add torch.compile (#10645) · 45ac4ff2
  youkaichao authored Nov 25, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  45ac4ff2
- [misc] do not read HOST_IP (#10644) · 6e9ff050
  youkaichao authored Nov 25, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6e9ff050
25 Nov, 2024 13 commits
- [Model] Add OLMo November 2024 model (#10503) · 9db713a1
  Shane A authored Nov 25, 2024
  
  9db713a1
- [Model] Enable optional prefix when loading embedding models (#10639) · cf73f0c9
  Cyrus Leung authored Nov 26, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cf73f0c9
- [Model]: Add support for Aria model (#10514) · b1d92053
  zhou fan authored Nov 26, 2024
```
Signed-off-by: xffxff <1247714429@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  b1d92053
- [Bugfix] Fix chunked prefill with model dtype float32 on Turing Devices (#9850) · c27df94e
  Wallas Henrique authored Nov 25, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  c27df94e
- [Bug]: Authorization ignored when root_path is set (#10606) · d04b13a3
  Chauncey authored Nov 26, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  d04b13a3
- [Model] Support `is_causal` HF config field for Qwen2 model (#10621) · ed46f143
  Cyrus Leung authored Nov 25, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ed46f143
- [misc] move functions to config.py (#10624) · 05d1f8c9
  youkaichao authored Nov 25, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  05d1f8c9
- [misc] add torch.compile compatibility check (#10618) · 25d806e9
  youkaichao authored Nov 24, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  25d806e9
- [torch.compile] add warning for unsupported models (#10622) · 65813781
  youkaichao authored Nov 24, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  65813781
- [torch.compile] force inductor threads (#10620) · 7c2134be
  Jee Jee Li authored Nov 25, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  7c2134be
- [torch.compile] support encoder based models (#10613) · 571841b7
  youkaichao authored Nov 24, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  571841b7
- [Refactor][MISC] del redundant code in ParallelConfig.postinit (#10614) · 7ea3cd7c
  Mengqing Cao authored Nov 25, 2024
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  7ea3cd7c
- Support Cross encoder models (#10400) · 214efc2c
  Maximilien de Bayser authored Nov 24, 2024
```
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
```
  214efc2c
24 Nov, 2024 2 commits
- [model][utils] add extract_layer_index utility function (#10599) · c0557478
  youkaichao authored Nov 23, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c0557478
- [Bugfix] Fix LoRA weight sharding (#10450) · 1700c543
  Jee Jee Li authored Nov 24, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  1700c543