Commits · afd0da2186c1d58fb48e138df0a2f548612b5d7d · OpenDAS / vllm_cscc

01 Feb, 2025 1 commit

[Attention] Deepseek v3 MLA support with FP8 compute (#12601) · baeded25

Lucas Wilkinson authored Feb 01, 2025



This PR implements the Deepseek V3 support by performing matrix absorption the fp8 weights 

---------
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Alexander Matveev <59768536+alexm-neuralmagic@users.noreply.github.com>

baeded25

31 Jan, 2025 1 commit

[Attention] MLA decode optimizations (#12528) · cabaf4ef

Lucas Wilkinson authored Jan 31, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Alexander Matveev <59768536+alexm-neuralmagic@users.noreply.github.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>

cabaf4ef

28 Jan, 2025 1 commit
- Update `pre-commit` hooks (#12475) · 823ab796
  Harry Mellor authored Jan 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  823ab796
27 Jan, 2025 1 commit

[FlashInfer] Upgrade to 0.2.0 (#11194) · 2bc3fbba

Bowen Wang authored Jan 28, 2025


Signed-off-by: Bowen Wang <abmfy@icloud.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

2bc3fbba

22 Jan, 2025 1 commit
- [Misc] Improve the readability of BNB error messages (#12320) · 84bee4bd
  Jee Jee Li authored Jan 23, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  84bee4bd
19 Jan, 2025 2 commits
- [Model] Support for fairseq2 Llama (#11442) · bbe5f9de
  Martin Gleize authored Jan 19, 2025
```
Signed-off-by: Martin Gleize <mgleize@meta.com>
Co-authored-by: mgleize user <mgleize@a100-st-p4de24xlarge-4.fair-a100.hpcaas>
```
  bbe5f9de
- [Misc] Add BNB support to GLM4-V model (#12184) · edaae198
  Isotr0py authored Jan 19, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  edaae198
14 Jan, 2025 1 commit
- [Misc] Merge bitsandbytes_stacked_params_mapping and packed_modules_mapping (#11924) · a3a3ee4e
  Jee Jee Li authored Jan 15, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a3a3ee4e
08 Jan, 2025 1 commit
- [Doc] Move examples into categories (#11840) · aba8d6ee
  Harry Mellor authored Jan 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  aba8d6ee
27 Dec, 2024 2 commits
- [Misc] Improve BNB loader to handle mixture of sharded and merged weights with same suffix (#11566) · dde1fa18
  Isotr0py authored Dec 28, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  dde1fa18
- [Misc]Add BNB quantization for MolmoForCausalLM (#11551) · 0240402c
  Jee Jee Li authored Dec 28, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  0240402c
26 Dec, 2024 1 commit
- [Misc] Add placeholder module (#11501) · eec906d8
  Cyrus Leung authored Dec 26, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  eec906d8
20 Dec, 2024 1 commit
- [Core] Loading model from S3 using RunAI Model Streamer as optional loader (#10192) · 995f5623
  omer-dayan authored Dec 20, 2024
```
Signed-off-by: OmerD <omer@run.ai>
```
  995f5623
07 Dec, 2024 1 commit
- [Model] Composite weight loading for multimodal Qwen2 (#10944) · bf0e382e
  Cyrus Leung authored Dec 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  bf0e382e
05 Dec, 2024 1 commit
- [Bugfix] Fix BNB loader target_modules (#10720) · 1f958a7d
  Jee Jee Li authored Dec 05, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1f958a7d
02 Dec, 2024 1 commit
- [Model] Add TP and BNB quantization support to LlavaMultiModalProjector (#10834) · 4c05edb3
  Isotr0py authored Dec 03, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  4c05edb3
01 Dec, 2024 1 commit
- [Model] Replace embedding models with pooling adapter (#10769) · 13370712
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  13370712
27 Nov, 2024 1 commit
- [Misc]Further reduce BNB static variable (#10597) · 15cc2a9f
  Jee Jee Li authored Nov 27, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  15cc2a9f
25 Nov, 2024 1 commit
- [misc] move functions to config.py (#10624) · 05d1f8c9
  youkaichao authored Nov 25, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  05d1f8c9
22 Nov, 2024 1 commit
- [Bugfix] Fix Phi-3 BNB quantization with tensor parallel (#9948) · b6374e09
  Isotr0py authored Nov 22, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b6374e09
19 Nov, 2024 1 commit
- [Doc] fix link for page that was renamed (#10455) · fd9f1249
  Russell Bryant authored Nov 19, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  fd9f1249
18 Nov, 2024 2 commits
- [Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) · 6b2d25ef
  Yan Ma authored Nov 19, 2024
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  6b2d25ef
- [Misc] Add uninitialized params tracking for `AutoWeightsLoader` (#10327) · c4e46433
  Isotr0py authored Nov 18, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c4e46433
17 Nov, 2024 1 commit
- [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) · 4fd93750
  youkaichao authored Nov 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4fd93750
15 Nov, 2024 1 commit
- [core][misc] keep compatibility for old-style classes (#10356) · 3a763ba0
  youkaichao authored Nov 15, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3a763ba0
14 Nov, 2024 1 commit
- [misc] error early for old-style class (#10304) · 504ac53d
  youkaichao authored Nov 13, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  504ac53d
13 Nov, 2024 1 commit
- [Bugfix] bitsandbytes models fail to run pipeline parallel (#10200) · ac49b59d
  HoangCongDuc authored Nov 14, 2024
```
Signed-off-by: Hoang Cong Duc <hoangcongducltt@gmail.com>
```
  ac49b59d
09 Nov, 2024 1 commit
- [5/N] pass the whole config to model (#9983) · 1a95f10e
  youkaichao authored Nov 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1a95f10e
06 Nov, 2024 1 commit
- [CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
  Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  21063c11
05 Nov, 2024 1 commit
- [Misc] Modify BNB parameter name (#9997) · b9c64c0c
  Jee Jee Li authored Nov 06, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  b9c64c0c
04 Nov, 2024 3 commits
- [Misc]Reduce BNB static variable (#9987) · fb2716d6
  Jee Jee Li authored Nov 05, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  fb2716d6
- [4/N] make quant config first-class citizen (#9978) · 8d72bb20
  youkaichao authored Nov 04, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  8d72bb20
- [Bugfix] Fix MiniCPMV and Mllama BNB bug (#9917) · c49f0407
  Jee Jee Li authored Nov 04, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c49f0407
02 Nov, 2024 2 commits
- [bugfix] fix tsts (#9959) · 3bb4befe
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3bb4befe
- [3/N] model runner pass the whole config to model (#9958) · cea808f3
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  cea808f3
30 Oct, 2024 1 commit
- [Model] Support math-shepherd-mistral-7b-prm model (#9697) · 81f09cfd
  Went-Liang authored Oct 31, 2024
```
Signed-off-by: Went-Liang <wenteng_liang@163.com>
```
  81f09cfd
29 Oct, 2024 2 commits
- [Bugfix] Fix prefix strings for quantized VLMs (#9772) · bc73e982
  Michael Goin authored Oct 29, 2024
  
  bc73e982
- [Model] Add BNB quantization support for Mllama (#9720) · 09500f7d
  Isotr0py authored Oct 29, 2024
  
  09500f7d
09 Oct, 2024 2 commits
- support bitsandbytes quantization with more models (#9148) · 2f4117c3
  chenqianfzh authored Oct 08, 2024
  
  2f4117c3
- 添加w8a8 rocblas非融合支持 · 35a8304d
  zhuwenwen authored Oct 09, 2024
  
  35a8304d