Commits · db8ba3228798fe856c480b5363903310240fb718 · OpenDAS / AutoAWQ

"vscode:/vscode.git/clone" did not exist on "0f218731e341a8e3418053adc87b1742ccdeab0a"

13 Sep, 2023 16 commits
- Add safetensors support · db8ba322
  Casper Hansen authored Sep 13, 2023
  
  db8ba322
- Implement saving sharded weights + safetensors · 97d38e29
  Casper Hansen authored Sep 13, 2023
  
  97d38e29
- Update README.md · d76125bf
  Casper authored Sep 13, 2023
  
  d76125bf
- Merge pull request #40 from casper-hansen/new_kernel · 1b0af2d3
  Casper authored Sep 13, 2023
```
[NEW] GEMV kernel implementation
```
  1b0af2d3
- Update README · f264ebb3
  Casper Hansen authored Sep 13, 2023
  
  f264ebb3
- Fix Falcon benchmark format · dc99d2fc
  Casper Hansen authored Sep 13, 2023
  
  dc99d2fc
- Update benchmarks · 7cf3c790
  Casper Hansen authored Sep 13, 2023
  
  7cf3c790
- More consistent benchmark · 86ea8df1
  Casper Hansen authored Sep 13, 2023
  
  86ea8df1
- Guide on quantized vs non-quantized · 98f6d7b9
  Casper Hansen authored Sep 13, 2023
  
  98f6d7b9
- Add explanation of from_quantized variables · c88b2e25
  Casper Hansen authored Sep 13, 2023
  
  c88b2e25
- Update README.md · 196119b4
  Casper authored Sep 13, 2023
  
  196119b4
- Set batch size for attention shapes · d3625d1c
  Casper Hansen authored Sep 13, 2023
  
  d3625d1c
- Set the batch_size · fdff74d6
  Casper Hansen authored Sep 13, 2023
  
  fdff74d6
- Set AWQ_BATCH_SIZE environment variable · 6e0bde15
  Casper Hansen authored Sep 13, 2023
  
  6e0bde15
- Catch out of memory exception · a2aa804c
  Casper Hansen authored Sep 13, 2023
  
  a2aa804c
- Support batch size benchmark, stop if OOM · 0091f1e2
  Casper Hansen authored Sep 13, 2023
  
  0091f1e2
12 Sep, 2023 8 commits
- Falcon 40B attention shapes [WIP] · 28d52d81
  Casper Hansen authored Sep 12, 2023
  
  28d52d81
- xq view key · bc4f93c2
  Casper Hansen authored Sep 12, 2023
  
  bc4f93c2
- Fix Falcon inputs ids · ba4da393
  Casper Hansen authored Sep 12, 2023
  
  ba4da393
- Falcon: Enable fused attention · f03f72de
  Casper Hansen authored Sep 12, 2023
  
  f03f72de
- Create custom attention shape for Falcon 7B · ac770875
  Casper Hansen authored Sep 12, 2023
  
  ac770875
- xk_reshape key · 7f8f9f16
  Casper Hansen authored Sep 12, 2023
  
  7f8f9f16
- Refactor view/reshaping into a predefined dict · a8c9afd5
  Casper Hansen authored Sep 12, 2023
  
  a8c9afd5
- Create Falcon block and model for fusing · 4517b3f2
  Casper Hansen authored Sep 12, 2023
  
  4517b3f2
11 Sep, 2023 9 commits
- Use CUDA stream · e120c9b6
  Casper Hansen authored Sep 11, 2023
  
  e120c9b6
- Remove xformers · 1aa8aebd
  Casper Hansen authored Sep 11, 2023
  
  1aa8aebd
- Update MPTBlock, fuse with MPTModel · dd41a223
  Casper Hansen authored Sep 11, 2023
  
  dd41a223
- Fix attention, support alibi · 7631add1
  Casper Hansen authored Sep 11, 2023
  
  7631add1
- Create MPTModel class · 90f54dbb
  Casper Hansen authored Sep 11, 2023
  
  90f54dbb
- Initialize with device · e80663bb
  Casper Hansen authored Sep 11, 2023
  
  e80663bb
- Remove fusing attention, only blocks · ac3e86df
  Casper Hansen authored Sep 11, 2023
  
  ac3e86df
- Fuse MPT block · 950851b3
  Casper Hansen authored Sep 11, 2023
  
  950851b3
- Switch to torch SDPA · d7badefc
  Casper Hansen authored Sep 11, 2023
  
  d7badefc
09 Sep, 2023 7 commits
- Falcon fused layers · 54f02854
  Casper Hansen authored Sep 09, 2023
  
  54f02854
- Fix alibi, rotary arg, neox, arg · 73c5e2bf
  Casper Hansen authored Sep 09, 2023
  
  73c5e2bf
- Add xformers requirement · 06bea896
  Casper Hansen authored Sep 09, 2023
  
  06bea896
- Fuse MPT · f3695d60
  Casper Hansen authored Sep 09, 2023
  
  f3695d60
- Update module name · 5bd6fbc7
  Casper Hansen authored Sep 09, 2023
  
  5bd6fbc7
- Implement ALiBi. · adc5304b
  Casper Hansen authored Sep 09, 2023
  
  adc5304b
- Default to GEMM · 48be2ee2
  Casper Hansen authored Sep 09, 2023
  
  48be2ee2