Commits · 3256ffec4f6ccf64a181cef03695219eba4b7ddc · OpenDAS / AutoAWQ

20 Sep, 2023 1 commit
- Support kv_heads · 3256ffec
  Casper Hansen authored Sep 20, 2023
  
  3256ffec
19 Sep, 2023 1 commit
- Use GEMM v2 kernel for context processing · f3a71d1d
  Casper Hansen authored Sep 19, 2023
  
  f3a71d1d
13 Sep, 2023 1 commit
- Set batch size for attention shapes · d3625d1c
  Casper Hansen authored Sep 13, 2023
  
  d3625d1c
12 Sep, 2023 7 commits
- Falcon 40B attention shapes [WIP] · 28d52d81
  Casper Hansen authored Sep 12, 2023
  
  28d52d81
- xq view key · bc4f93c2
  Casper Hansen authored Sep 12, 2023
  
  bc4f93c2
- Fix Falcon inputs ids · ba4da393
  Casper Hansen authored Sep 12, 2023
  
  ba4da393
- Create custom attention shape for Falcon 7B · ac770875
  Casper Hansen authored Sep 12, 2023
  
  ac770875
- xk_reshape key · 7f8f9f16
  Casper Hansen authored Sep 12, 2023
  
  7f8f9f16
- Refactor view/reshaping into a predefined dict · a8c9afd5
  Casper Hansen authored Sep 12, 2023
  
  a8c9afd5
- Create Falcon block and model for fusing · 4517b3f2
  Casper Hansen authored Sep 12, 2023
  
  4517b3f2
11 Sep, 2023 6 commits
- Update MPTBlock, fuse with MPTModel · dd41a223
  Casper Hansen authored Sep 11, 2023
  
  dd41a223
- Fix attention, support alibi · 7631add1
  Casper Hansen authored Sep 11, 2023
  
  7631add1
- Create MPTModel class · 90f54dbb
  Casper Hansen authored Sep 11, 2023
  
  90f54dbb
- Initialize with device · e80663bb
  Casper Hansen authored Sep 11, 2023
  
  e80663bb
- Fuse MPT block · 950851b3
  Casper Hansen authored Sep 11, 2023
  
  950851b3
- Switch to torch SDPA · d7badefc
  Casper Hansen authored Sep 11, 2023
  
  d7badefc
09 Sep, 2023 4 commits
- Fix alibi, rotary arg, neox, arg · 73c5e2bf
  Casper Hansen authored Sep 09, 2023
  
  73c5e2bf
- Fuse MPT · f3695d60
  Casper Hansen authored Sep 09, 2023
  
  f3695d60
- Implement ALiBi. · adc5304b
  Casper Hansen authored Sep 09, 2023
  
  adc5304b
- Create torch SDPA implementation · ebbbc3a5
  Casper Hansen authored Sep 09, 2023
  
  ebbbc3a5
08 Sep, 2023 7 commits
- GEMM + GEMV compatibility · 890b6aa7
  Casper Hansen authored Sep 08, 2023
  
  890b6aa7
- Use apply_rotary_emb · 64e6b3e1
  Casper Hansen authored Sep 08, 2023
  
  64e6b3e1
- Update kernel, remove unused code · a11c313a
  Casper Hansen authored Sep 08, 2023
  
  a11c313a
- Attention works · fbfa9d82
  Casper Hansen authored Sep 08, 2023
  
  fbfa9d82
- Fix MLP module · 75710806
  Casper Hansen authored Sep 08, 2023
  
  75710806
- Implement GEMM/GEMV in quantize function and fused modules · 5db43a7f
  Casper Hansen authored Sep 08, 2023
  
  5db43a7f
- Refactor modules, create separate GEMM and GEMV · fe314160
  Casper Hansen authored Sep 08, 2023
  
  fe314160
06 Sep, 2023 3 commits
- Correct input sizes · ab7d68e7
  Casper Hansen authored Sep 06, 2023
  
  ab7d68e7
- Update shapes for cuda kernel · e71181bd
  Casper Hansen authored Sep 06, 2023
  
  e71181bd
- Use hf_rotary per default · b7290868
  Casper Hansen authored Sep 06, 2023
  
  b7290868
05 Sep, 2023 4 commits
- Remove CustomQuantAttention class · 42655843
  Casper Hansen authored Sep 05, 2023
  
  42655843
- Revert "Update neox kernel" · bbe1d46a
  Casper Hansen authored Sep 05, 2023
```
This reverts commit 4fe9974a.
```
  bbe1d46a
- Update neox kernel · 4fe9974a
  Casper Hansen authored Sep 05, 2023
  
  4fe9974a
- Attempt new module · 8bb8fe20
  Casper Hansen authored Sep 05, 2023
  
  8bb8fe20
02 Sep, 2023 4 commits
- Refactor MPT Quant MLP · 1df0136e
  Casper Hansen authored Sep 02, 2023
  
  1df0136e
- Refactor Llama Quant MLP · ded3ea71
  Casper Hansen authored Sep 02, 2023
  
  ded3ea71
- Refactor Llama Quant RMSNorm · 620966e8
  Casper Hansen authored Sep 02, 2023
  
  620966e8
- Refactor Llama quant attention · 2082197d
  Casper Hansen authored Sep 02, 2023
  
  2082197d
27 Aug, 2023 1 commit
- Replace print with logging, Remove uncommented code · b491c2d6
  Casper authored Aug 27, 2023
  
  b491c2d6
24 Aug, 2023 1 commit
- Integrate fused modules into AWQ model loading · 63346c34
  Casper Hansen authored Aug 24, 2023
  
  63346c34