Commits · b02fd288b28f0bfa2d7ac8958fe0d71ec22ffc1b · OpenDAS / vllm_cscc

29 Jan, 2025 4 commits

[Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models. (#11787) · b02fd288
Pavani Majety authored Jan 29, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
b02fd288

[Model] Refactoring of MiniCPM-V and add MiniCPM-o-2.6 support for vLLM (#12069) · d93bf4da

Alphi authored Jan 29, 2025


Signed-off-by: hzh <hezhihui_thu@163.com>
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Signed-off-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Oleg Mosalov <oleg@krai.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>
Signed-off-by: Chenguang Li <757486878@qq.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Shanshan Shen <467638484@qq.com>
Signed-off-by: elijah <f1renze.142857@gmail.com>
Signed-off-by: Yikun <yikunkero@gmail.com>
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Co-authored-by: shaochangxu <85155497+shaochangxu@users.noreply.github.com>
Co-authored-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: sixgod <evethwillbeok@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Rafael Vasquez <rafvasq21@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Akshat Tripathi <Akshat.tripathi6568@gmail.com>
Co-authored-by: Oleg Mosalov <oleg@krai.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Avshalom Manevich <12231371+avshalomman@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Yangcheng Li <liyangcheng.lyc@alibaba-inc.com>
Co-authored-by: Siyuan Li <94890248+liaoyanqing666@users.noreply.github.com>
Co-authored-by: Concurrensee <yida.wu@amd.com>
Co-authored-by: Chenguang Li <757486878@qq.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Alex Brooks <alex.brooks@ibm.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
Co-authored-by: elijah <30852919+e1ijah1@users.noreply.github.com>
Co-authored-by: Yikun Jiang <yikunkero@gmail.com>
Co-authored-by: Steve Luo <36296769+SunflowerAries@users.noreply.github.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: TJian <tunjian1996@gmail.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: maang-h <55082429+maang-h@users.noreply.github.com>
Co-authored-by: Elfie Guo <164945471+elfiegg@users.noreply.github.com>
Co-authored-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

d93bf4da

[Bugfix] handle alignment of arguments in convert_sparse_cross_attention_mask_to_dense (#12347) · 036ca94c

Travis Johnson authored Jan 29, 2025


Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Wallas Santos <wallashss@ibm.com>

036ca94c

Bugfix for whisper quantization due to fake k_proj bias (#12524) · bd02164c
Michael Goin authored Jan 28, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
bd02164c

28 Jan, 2025 2 commits
- [VLM] Merged multi-modal processor and V1 support for Qwen-VL (#12504) · 8f58a513
  Cyrus Leung authored Jan 29, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f58a513
- Update `pre-commit` hooks (#12475) · 823ab796
  Harry Mellor authored Jan 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  823ab796
27 Jan, 2025 2 commits
- [Bugfix] Fix gpt2 GGUF inference (#12467) · ce69f7f7
  Isotr0py authored Jan 27, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  ce69f7f7
- [Bugfix] Fix Granite 3.0 MoE model loading (#12446) · 5204ff5c
  Cyrus Leung authored Jan 27, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  5204ff5c
25 Jan, 2025 1 commit
- [Bugfix] Fix BLIP-2 processing (#12412) · fb30ee92
  Cyrus Leung authored Jan 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  fb30ee92
23 Jan, 2025 2 commits
- [FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) · e97f802b
  Gregory Shtrasberg authored Jan 23, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
```
  e97f802b
- [Bugfix] Fix k_proj's bias for whisper self attention (#12342) · c5b4b11d
  Isotr0py authored Jan 23, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c5b4b11d
22 Jan, 2025 6 commits
- [Doc] Add docs for prompt replacement (#12318) · 6609cdf0
  Cyrus Leung authored Jan 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6609cdf0
- [Bugfix][VLM] Fix mixed-modality inference backward compatibility for V0 (#12313) · 16366ee8
  Roger Wang authored Jan 22, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  16366ee8
- [Model][Bugfix]: correct Aria model output (#12309) · 528dbcac
  zhou fan authored Jan 22, 2025
```
Signed-off-by: xffxff <1247714429@qq.com>
```
  528dbcac
- [VLM] Avoid unnecessary tokenization (#12310) · cd7b6f08
  Cyrus Leung authored Jan 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cd7b6f08
- [ci/lint] Add back default arg for pre-commit (#12279) · 64ea24d0
  Kevin H. Luu authored Jan 21, 2025
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  64ea24d0
- [VLM] Simplify post-processing of replacement info (#12269) · df76e5af
  Cyrus Leung authored Jan 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  df76e5af
21 Jan, 2025 4 commits
- [Misc] Set default backend to SDPA for get_vit_attn_backend (#12235) · fa9ee081
  wangxiyuan authored Jan 22, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  fa9ee081
- [Misc] Remove redundant TypeVar from base model (#12248) · f2e9f2a3
  Cyrus Leung authored Jan 21, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f2e9f2a3
- [Misc]Add BNB quantization for PaliGemmaForConditionalGeneration (#12237) · 1f1542af
  Jee Jee Li authored Jan 21, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1f1542af
- [Misc] Rename `MultiModalInputsV2 -> MultiModalInputs` (#12244) · 96912550
  Cyrus Leung authored Jan 21, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  96912550
20 Jan, 2025 2 commits
- [Model] Upgrade Aria to transformers 4.48 (#12203) · b37d8279
  Cyrus Leung authored Jan 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b37d8279
- [Model] Add Qwen2 PRM model support (#12202) · 83609791
  Isotr0py authored Jan 20, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  83609791
19 Jan, 2025 4 commits

[Model] Support for fairseq2 Llama (#11442) · bbe5f9de

Martin Gleize authored Jan 19, 2025


Signed-off-by: Martin Gleize <mgleize@meta.com>
Co-authored-by: mgleize user <mgleize@a100-st-p4de24xlarge-4.fair-a100.hpcaas>

bbe5f9de

[V1] Add V1 support of Qwen2-VL (#12128) · 81763c58

Roger Wang authored Jan 19, 2025


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: imkero <kerorek@outlook.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

81763c58

[Misc] Add BNB support to GLM4-V model (#12184) · edaae198
Isotr0py authored Jan 19, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
edaae198
[Bugfix] Fix multi-modal processors for transformers 4.48 (#12187) · 630eb5b5
Cyrus Leung authored Jan 19, 2025

630eb5b5

18 Jan, 2025 1 commit
- [Model] Port deepseek-vl2 processor, remove dependency (#12169) · 02798eca
  Isotr0py authored Jan 18, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  02798eca
16 Jan, 2025 4 commits
- [Model] Add support for deepseek-vl2-tiny model (#12068) · 62b06ba2
  Isotr0py authored Jan 17, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  62b06ba2
- [Bugfix] Fix max image feature size for Llava-one-vision (#12104) · 874f7c29
  Roger Wang authored Jan 16, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  874f7c29
- [Bugfix] Remove hardcoded `head_size=256` for Deepseek v2 and v3 (#12067) · dd7c9ad8
  Isotr0py authored Jan 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  dd7c9ad8
- Various cosmetic/comment fixes (#12089) · 9aa1519f
  Michael Goin authored Jan 16, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  9aa1519f
15 Jan, 2025 3 commits
- [Misc][Quark] Upstream Quark format to VLLM (#10765) · de0526f6
  kewang-xlnx authored Jan 16, 2025
```
Signed-off-by: kewang-xlnx <kewang@xilinx.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  de0526f6
- [Model]: Support internlm3 (#12037) · 97eb97b5
  RunningLeon authored Jan 15, 2025
  
  97eb97b5
- [Kernel] Support MulAndSilu (#11624) · 42f5e7c5
  Jee Jee Li authored Jan 15, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  42f5e7c5
14 Jan, 2025 1 commit
- [Misc] Merge bitsandbytes_stacked_params_mapping and packed_modules_mapping (#11924) · a3a3ee4e
  Jee Jee Li authored Jan 15, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a3a3ee4e
13 Jan, 2025 4 commits
- [Misc] Fix Deepseek V2 fp8 kv-scale remapping (#11947) · cf6bbcb4
  Concurrensee authored Jan 13, 2025
```
Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>
```
  cf6bbcb4
- [CI][Spec Decode] fix: broken test for EAGLE model (#11972) · 80ea3af1
  Sungjae Lee authored Jan 13, 2025
```
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
```
  80ea3af1
- [Bug] Fix usage of `.transpose()` and `.view()` consecutively. (#11979) · 9dd02d85
  Siyuan Li authored Jan 13, 2025
  
  9dd02d85
- [Model] Support GGUF models newly added in `transformers` 4.46.0 (#9685) · d14e98d9
  Isotr0py authored Jan 13, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d14e98d9