Commits · e97f802b2d74861af77997691a7d1c36498f6dca · OpenDAS / vllm_cscc · GitLab

23 Jan, 2025 1 commit

[FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) · e97f802b

Gregory Shtrasberg authored Jan 23, 2025


Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>

e97f802b

16 Jan, 2025 1 commit
- Various cosmetic/comment fixes (#12089) · 9aa1519f
  Michael Goin authored Jan 16, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  9aa1519f
15 Jan, 2025 1 commit

[Misc][Quark] Upstream Quark format to VLLM (#10765) · de0526f6

kewang-xlnx authored Jan 16, 2025


Signed-off-by: kewang-xlnx <kewang@xilinx.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

de0526f6

14 Jan, 2025 1 commit
- [Misc] Merge bitsandbytes_stacked_params_mapping and packed_modules_mapping (#11924) · a3a3ee4e
  Jee Jee Li authored Jan 15, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a3a3ee4e
02 Jan, 2025 1 commit
- [Bugfix] Change kv scaling factor by param json on nvidia gpu (#11688) · 187e3299
  bjmsong authored Jan 03, 2025
```
Signed-off-by: bjmsong <bjmsong@126.com>
Co-authored-by: bjmsong <bjmsong@126.com>
```
  187e3299
07 Dec, 2024 1 commit
- [Doc] Explicitly state that PP isn't compatible with speculative decoding yet (#10975) · c889d588
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c889d588
22 Nov, 2024 1 commit
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
18 Nov, 2024 1 commit
- [Misc] Add uninitialized params tracking for `AutoWeightsLoader` (#10327) · c4e46433
  Isotr0py authored Nov 18, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c4e46433
17 Nov, 2024 1 commit
- [V1] Refactor model executable interface for all text-only language models (#10374) · 643ecf7b
  Roger Wang authored Nov 16, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  643ecf7b
11 Nov, 2024 1 commit
- [6/N] pass whole config to inner model (#10205) · f89d18ff
  youkaichao authored Nov 10, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  f89d18ff
09 Nov, 2024 1 commit
- [5/N] pass the whole config to model (#9983) · 1a95f10e
  youkaichao authored Nov 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1a95f10e
06 Nov, 2024 2 commits
- [V1] Make v1 more testable (#9888) · d58268c5
  Joe Runde authored Nov 06, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  d58268c5
- [CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
  Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  21063c11
28 Oct, 2024 1 commit
- [Hardware][ROCM] using current_platform.is_rocm (#9642) · 4e2d95e3
  wangshuai09 authored Oct 28, 2024
```
Signed-off-by: wangshuai09 <391746016@qq.com>
```
  4e2d95e3
24 Oct, 2024 1 commit
- [torch.compile] Adding torch compile annotations to some models (#9641) · d27cfbf7
  Yongzao authored Oct 25, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  d27cfbf7
04 Oct, 2024 1 commit

[Models] Add remaining model PP support (#7168) · 0f6d7a9a

Murali Andoorveedu authored Oct 03, 2024

Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Signed-off-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

0f6d7a9a

18 Sep, 2024 1 commit
- [Model] Support Solar Model (#8386) · e18749ff
  Geun, Lim authored Sep 19, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  e18749ff
17 Sep, 2024 1 commit
- [Bugfix] Fix TP > 1 for new granite (#8544) · 98f97133
  Joe Runde authored Sep 17, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  98f97133
02 Sep, 2024 1 commit
- [Model] Add Granite model (#7436) · f8d60145
  Shawn Tan authored Sep 01, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  f8d60145
30 Aug, 2024 1 commit
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
13 Aug, 2024 1 commit
- [Bugfix] Fix weight loading for Chameleon when TP>1 (#7410) · 7025b11d
  Cyrus Leung authored Aug 13, 2024
  
  7025b11d
05 Aug, 2024 1 commit
- [Core] Support loading GGUF model (#5191) · 360bd67c
  Isotr0py authored Aug 06, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  360bd67c
01 Aug, 2024 1 commit
- [Bugfix][Model] Skip loading lm_head weights if using tie_word_embeddings (#6758) · 630dd9e0
  Travis Johnson authored Jul 31, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  630dd9e0
31 Jul, 2024 1 commit

[Bugfix] Clean up MiniCPM-V (#6939) · 2f4e108f

Alphi authored Jul 31, 2024


Co-authored-by: hezhihui <hzh7269@modelbest.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

2f4e108f

25 Jul, 2024 1 commit
- [Model] Adding support for MiniCPM-V (#4087) · 9e169a4c
  Alphi authored Jul 25, 2024
  
  9e169a4c
23 Jul, 2024 1 commit
- [Misc] Support FP8 kv cache scales from compressed-tensors (#6528) · 9e0b558a
  Michael Goin authored Jul 23, 2024
  
  9e0b558a
19 Jul, 2024 1 commit
- [ Misc ] non-uniform quantization via `compressed-tensors` for `Llama` (#6515) · dbe55885
  Robert Shaw authored Jul 18, 2024
  
  dbe55885
18 Jul, 2024 1 commit
- [Model] Support Mistral-Nemo (#6548) · 15c6a079
  Michael Goin authored Jul 18, 2024
  
  15c6a079
17 Jul, 2024 1 commit
- [Distributed][PP] only create embedding & lm head when necessary (#6455) · 1d094fd7
  Wushi Dong authored Jul 16, 2024
```
original title: [Distributed][Model] Rank-based Component Creation for Pipeline Parallelism Memory Optimization
```
  1d094fd7
16 Jul, 2024 1 commit
- [Kernel][Attention] Separate `Attention.kv_scale` into `k_scale` and `v_scale` (#6081) · 978aed53
  Michael Goin authored Jul 16, 2024
  
  978aed53
15 Jul, 2024 1 commit
- [core][distributed] simplify code to support pipeline parallel (#6406) · 69672f11
  youkaichao authored Jul 14, 2024
  
  69672f11
02 Jul, 2024 2 commits
- [CORE] Quantized lm-head Framework (#4442) · ee93f4f9
  Qubitium-ModelCloud authored Jul 03, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: ZX <zx@lbx.dev>
```
  ee93f4f9
- [Core] Pipeline Parallel Support (#4412) · c5832d2a
  Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  c5832d2a
27 Jun, 2024 2 commits
- [Model][Bugfix] Implicit model flags and reenable Phi-3-Vision (#5896) · 98cf2ed6
  Cyrus Leung authored Jun 28, 2024
  
  98cf2ed6
- [Model] Add base class for LoRA-supported models (#5018) · 96354d6a
  Cyrus Leung authored Jun 27, 2024
  
  96354d6a
01 Jun, 2024 1 commit
- [Feature][Kernel] Support bitsandbytes quantization and QLoRA (#4776) · b9c0605a
  chenqianfzh authored Jun 01, 2024
  
  b9c0605a
27 May, 2024 1 commit

[Bugfix / Core] Prefix Caching Guards (merged with main) (#4846) · 1102bef2

Zhuohan Li authored May 27, 2024


Co-authored-by: rsnm2 <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>

1102bef2

23 May, 2024 1 commit

[Kernel] Initial Activation Quantization Support (#4525) · a1242324

Dipika Sikka authored May 23, 2024


Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

a1242324

22 May, 2024 2 commits
- [Minor] Fix small typo in llama.py: QKVParallelLinear -> QuantizationConfig (#4991) · a36de682
  Philipp Moritz authored May 22, 2024
  
  a36de682
- [Misc] Load FP8 kv-cache scaling factors from checkpoints (#4893) · a3a73ab0
  Cody Yu authored May 22, 2024
```
The 2nd PR for #4532.

This PR supports loading FP8 kv-cache scaling factors from a FP8 checkpoint (with .kv_scale parameter).
```
  a3a73ab0