Commits · ad385667103b7a98fee5a4fc46d2a7745407333e · OpenDAS / vllm_cscc

11 Oct, 2024 1 commit
- [Misc] Update config loading for Qwen2-VL and remove Granite · a7e72c36
  zhuwenwen authored Oct 11, 2024
  
  a7e72c36
04 Oct, 2024 1 commit

[Models] Add remaining model PP support (#7168) · 0f6d7a9a

Murali Andoorveedu authored Oct 03, 2024

Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Signed-off-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

0f6d7a9a

03 Oct, 2024 1 commit
- [Model] Adding Granite MoE. (#8206) · 19f0d257
  Shawn Tan authored Oct 02, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  19f0d257
26 Sep, 2024 1 commit
- [Misc] Update config loading for Qwen2-VL and remove Granite (#8837) · 4bb98f21
  Roger Wang authored Sep 26, 2024
  
  4bb98f21
17 Sep, 2024 1 commit
- [Bugfix] Fix TP > 1 for new granite (#8544) · 98f97133
  Joe Runde authored Sep 17, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  98f97133
02 Sep, 2024 1 commit
- [Model] Add Granite model (#7436) · f8d60145
  Shawn Tan authored Sep 01, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  f8d60145
30 Aug, 2024 1 commit
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
19 Aug, 2024 2 commits
- Add fa pad conditions and automatic switching strategy · b40f2ffc
  zhuwenwen authored Aug 19, 2024
  
  b40f2ffc
- Add fa pad conditions and automatic switching strategy · fa973559
  zhuwenwen authored Aug 19, 2024
  
  fa973559
17 Aug, 2024 2 commits
- update fp16 model layout conversion conditions · 4d821524
  zhuwenwen authored Aug 17, 2024
  
  4d821524
- update fp16 model layout conversion conditions · a528f350
  zhuwenwen authored Aug 17, 2024
  
  a528f350
13 Aug, 2024 1 commit
- [Bugfix] Fix weight loading for Chameleon when TP>1 (#7410) · 7025b11d
  Cyrus Leung authored Aug 13, 2024
  
  7025b11d
09 Aug, 2024 1 commit
- 增添lmslim gptq支持 · dcaabcf7
  gaoqiong authored Aug 09, 2024
  
  dcaabcf7
05 Aug, 2024 1 commit
- [Core] Support loading GGUF model (#5191) · 360bd67c
  Isotr0py authored Aug 06, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  360bd67c
04 Aug, 2024 1 commit
- add llama model awq support · 5f5ddc3d
  gaoqiong authored Aug 04, 2024
  
  5f5ddc3d
01 Aug, 2024 2 commits
- [Bugfix][Model] Skip loading lm_head weights if using tie_word_embeddings (#6758) · 630dd9e0
  Travis Johnson authored Jul 31, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  630dd9e0
- support fa pad · ffbef65c
  zhuwenwen authored Aug 01, 2024
  
  ffbef65c
31 Jul, 2024 1 commit

[Bugfix] Clean up MiniCPM-V (#6939) · 2f4e108f

Alphi authored Jul 31, 2024


Co-authored-by: hezhihui <hzh7269@modelbest.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

2f4e108f

25 Jul, 2024 1 commit
- [Model] Adding support for MiniCPM-V (#4087) · 9e169a4c
  Alphi authored Jul 25, 2024
  
  9e169a4c
24 Jul, 2024 2 commits
- support transpose lm_head weight to nn layout · 99426767
  zhuwenwen authored Jul 24, 2024
  
  99426767
- add gemm pad and fa pad for 7b model · 0b5e4e11
  zhuwenwen authored Jul 24, 2024
  
  0b5e4e11
23 Jul, 2024 1 commit
- [Misc] Support FP8 kv cache scales from compressed-tensors (#6528) · 9e0b558a
  Michael Goin authored Jul 23, 2024
  
  9e0b558a
20 Jul, 2024 3 commits
- support nn layout · 1e0cb1f4
  zhuwenwen authored Jul 20, 2024
  
  1e0cb1f4
- 去掉调试信息 · 9653385f
  gaoqiong authored Jul 20, 2024
  
  9653385f
- 修改nn支持方式 · 835bd9fc
  gaoqiong authored Jul 20, 2024
  
  835bd9fc
19 Jul, 2024 1 commit
- [ Misc ] non-uniform quantization via `compressed-tensors` for `Llama` (#6515) · dbe55885
  Robert Shaw authored Jul 18, 2024
  
  dbe55885
18 Jul, 2024 1 commit
- [Model] Support Mistral-Nemo (#6548) · 15c6a079
  Michael Goin authored Jul 18, 2024
  
  15c6a079
17 Jul, 2024 1 commit
- [Distributed][PP] only create embedding & lm head when necessary (#6455) · 1d094fd7
  Wushi Dong authored Jul 16, 2024
```
original title: [Distributed][Model] Rank-based Component Creation for Pipeline Parallelism Memory Optimization
```
  1d094fd7
16 Jul, 2024 1 commit
- [Kernel][Attention] Separate `Attention.kv_scale` into `k_scale` and `v_scale` (#6081) · 978aed53
  Michael Goin authored Jul 16, 2024
  
  978aed53
15 Jul, 2024 1 commit
- [core][distributed] simplify code to support pipeline parallel (#6406) · 69672f11
  youkaichao authored Jul 14, 2024
  
  69672f11
08 Jul, 2024 1 commit
- add 7b pad dim · 5cdabd7b
  zhuwenwen authored Jul 08, 2024
  
  5cdabd7b
06 Jul, 2024 1 commit
- add fa pad · 371b1251
  zhuwenwen authored Jul 06, 2024
  
  371b1251
02 Jul, 2024 2 commits
- [CORE] Quantized lm-head Framework (#4442) · ee93f4f9
  Qubitium-ModelCloud authored Jul 03, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: ZX <zx@lbx.dev>
```
  ee93f4f9
- [Core] Pipeline Parallel Support (#4412) · c5832d2a
  Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  c5832d2a
27 Jun, 2024 2 commits
- [Model][Bugfix] Implicit model flags and reenable Phi-3-Vision (#5896) · 98cf2ed6
  Cyrus Leung authored Jun 28, 2024
  
  98cf2ed6
- [Model] Add base class for LoRA-supported models (#5018) · 96354d6a
  Cyrus Leung authored Jun 27, 2024
  
  96354d6a
01 Jun, 2024 1 commit
- [Feature][Kernel] Support bitsandbytes quantization and QLoRA (#4776) · b9c0605a
  chenqianfzh authored Jun 01, 2024
  
  b9c0605a
27 May, 2024 1 commit

[Bugfix / Core] Prefix Caching Guards (merged with main) (#4846) · 1102bef2

Zhuohan Li authored May 27, 2024


Co-authored-by: rsnm2 <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>

1102bef2

23 May, 2024 1 commit

[Kernel] Initial Activation Quantization Support (#4525) · a1242324

Dipika Sikka authored May 23, 2024


Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

a1242324

22 May, 2024 1 commit
- [Minor] Fix small typo in llama.py: QKVParallelLinear -> QuantizationConfig (#4991) · a36de682
  Philipp Moritz authored May 22, 2024
  
  a36de682