Commits · 40e6952ab1608abd2fcd204c0b911ea5af54665d · OpenDAS / AutoAWQ

05 Oct, 2023 1 commit
- Remove past_key_value (save 2GB VRAM) · 40e6952a
  Casper Hansen authored Oct 05, 2023
  
  40e6952a
02 Oct, 2023 3 commits
- Mistral fused modules · d5bb4ec8
  Casper Hansen authored Oct 02, 2023
  
  d5bb4ec8
- Fix Falcon n_kv_heads parameter · 80eea43e
  Casper Hansen authored Oct 02, 2023
  
  80eea43e
- Fix unexpected keyword · d016c513
  Casper Hansen authored Oct 02, 2023
  
  d016c513
27 Sep, 2023 5 commits
- Fix apply_rotary_emb() to have both tensors on the same device · 841a2313
  s4rduk4r authored Sep 27, 2023
  
  841a2313
- Clean code · cba9a28c
  Casper Hansen authored Sep 27, 2023
  
  cba9a28c
- Remove unused imports · 707315a6
  Casper Hansen authored Sep 27, 2023
  
  707315a6
- Move cache initialization back to init · 8c80b3e0
  Casper Hansen authored Sep 27, 2023
  
  8c80b3e0
- Roll cache efficiently and correctly · c2cd6535
  Casper Hansen authored Sep 27, 2023
  
  c2cd6535
26 Sep, 2023 2 commits
- Fix edge case · 0392a823
  Casper Hansen authored Sep 26, 2023
  
  0392a823
- Automatically reset/increase cache · 202c6d6a
  Casper Hansen authored Sep 26, 2023
  
  202c6d6a
24 Sep, 2023 1 commit
- Fix typing classes · 4abfefc9
  Vik Paruchuri authored Sep 23, 2023
  
  4abfefc9
21 Sep, 2023 4 commits
- Remove imports · 9060b980
  Casper Hansen authored Sep 21, 2023
  
  9060b980
- Remove fused modules (later) · 37db5759
  Casper Hansen authored Sep 21, 2023
  
  37db5759
- Fix usage of engine · 133dd7a7
  Casper authored Sep 21, 2023
  
  133dd7a7
- Fix module import · fbeea40b
  Casper Hansen authored Sep 21, 2023
  
  fbeea40b
20 Sep, 2023 6 commits
- Add bigcode fused modules (WIP) · b49168f8
  Casper Hansen authored Sep 20, 2023
  
  b49168f8
- Update setup · e3b8d9c4
  Casper authored Sep 20, 2023
  
  e3b8d9c4
- Removed unused module · bf76e108
  Casper Hansen authored Sep 20, 2023
  
  bf76e108
- Use default attention shapes · d973a0e0
  Casper Hansen authored Sep 20, 2023
  
  d973a0e0
- Fix attention shapes. Add repeat interleave. · 2d593b84
  Casper Hansen authored Sep 20, 2023
  
  2d593b84
- Support kv_heads · 3256ffec
  Casper Hansen authored Sep 20, 2023
  
  3256ffec
19 Sep, 2023 1 commit
- Use GEMM v2 kernel for context processing · f3a71d1d
  Casper Hansen authored Sep 19, 2023
  
  f3a71d1d
15 Sep, 2023 2 commits
- fix bug · 8e7059a7
  qwopqwop200 authored Sep 15, 2023
  
  8e7059a7
- suppport windows · e1884728
  qwopqwop200 authored Sep 15, 2023
  
  e1884728
13 Sep, 2023 1 commit
- Set batch size for attention shapes · d3625d1c
  Casper Hansen authored Sep 13, 2023
  
  d3625d1c
12 Sep, 2023 7 commits
- Falcon 40B attention shapes [WIP] · 28d52d81
  Casper Hansen authored Sep 12, 2023
  
  28d52d81
- xq view key · bc4f93c2
  Casper Hansen authored Sep 12, 2023
  
  bc4f93c2
- Fix Falcon inputs ids · ba4da393
  Casper Hansen authored Sep 12, 2023
  
  ba4da393
- Create custom attention shape for Falcon 7B · ac770875
  Casper Hansen authored Sep 12, 2023
  
  ac770875
- xk_reshape key · 7f8f9f16
  Casper Hansen authored Sep 12, 2023
  
  7f8f9f16
- Refactor view/reshaping into a predefined dict · a8c9afd5
  Casper Hansen authored Sep 12, 2023
  
  a8c9afd5
- Create Falcon block and model for fusing · 4517b3f2
  Casper Hansen authored Sep 12, 2023
  
  4517b3f2
11 Sep, 2023 6 commits
- Update MPTBlock, fuse with MPTModel · dd41a223
  Casper Hansen authored Sep 11, 2023
  
  dd41a223
- Fix attention, support alibi · 7631add1
  Casper Hansen authored Sep 11, 2023
  
  7631add1
- Create MPTModel class · 90f54dbb
  Casper Hansen authored Sep 11, 2023
  
  90f54dbb
- Initialize with device · e80663bb
  Casper Hansen authored Sep 11, 2023
  
  e80663bb
- Fuse MPT block · 950851b3
  Casper Hansen authored Sep 11, 2023
  
  950851b3
- Switch to torch SDPA · d7badefc
  Casper Hansen authored Sep 11, 2023
  
  d7badefc
09 Sep, 2023 1 commit
- Fix alibi, rotary arg, neox, arg · 73c5e2bf
  Casper Hansen authored Sep 09, 2023
  
  73c5e2bf