Commits · 724bda5884aba6a6d9d049f474aeeefb8b1ad070 · OpenDAS / AutoAWQ

20 Sep, 2023 13 commits
- Working for OPT · 724bda58
  Casper Hansen authored Sep 20, 2023
  
  724bda58
- Make code cleaner · 356cbc92
  Casper authored Sep 20, 2023
  
  356cbc92
- Add desc · 9204d6e8
  Casper authored Sep 20, 2023
  
  9204d6e8
- Added type hinting · 44470a7a
  Casper authored Sep 20, 2023
  
  44470a7a
- Initial quantization refactoring · 45c22ee5
  Casper authored Sep 20, 2023
  
  45c22ee5
- Merge pull request #60 from casper-hansen/kv_heads · a5e8b048
  Casper authored Sep 20, 2023
```
Support kv_heads
```
  a5e8b048
- Merge branch 'main' into kv_heads · a024e893
  Casper authored Sep 20, 2023
  
  a024e893
- Removed unused module · bf76e108
  Casper Hansen authored Sep 20, 2023
  
  bf76e108
- Use default attention shapes · d973a0e0
  Casper Hansen authored Sep 20, 2023
  
  d973a0e0
- Fix attention shapes. Add repeat interleave. · 2d593b84
  Casper Hansen authored Sep 20, 2023
  
  2d593b84
- Avoid fusing for larger Falcon models · e3936a44
  Casper Hansen authored Sep 20, 2023
  
  e3936a44
- Support kv_heads · 3256ffec
  Casper Hansen authored Sep 20, 2023
  
  3256ffec
- Merge pull request #58 from casper-hansen/gemmv2 · 63a12504
  Casper authored Sep 20, 2023
```
2x faster context processing with GEMV
```
  63a12504
19 Sep, 2023 3 commits
- Use GEMM v2 kernel for context processing · f3a71d1d
  Casper Hansen authored Sep 19, 2023
  
  f3a71d1d
- Move kernel into existing cuda file · 2fa3a5d1
  Casper Hansen authored Sep 19, 2023
  
  2fa3a5d1
- Add GEMM v2 kernel · 7cd461ff
  Casper Hansen authored Sep 19, 2023
  
  7cd461ff
15 Sep, 2023 7 commits
- Merge pull request #47 from casper-hansen/safetensors · 8788fe10
  Casper authored Sep 15, 2023
```
Safetensors and model sharding
```
  8788fe10
- Rename safetensors generation example · 4ecd8594
  Casper Hansen authored Sep 15, 2023
  
  4ecd8594
- Default to empty string for model file · 435b3b4b
  Casper Hansen authored Sep 15, 2023
  
  435b3b4b
- Merge pull request #27 from boehm-e/main · 2a3e0fa1
  Casper authored Sep 15, 2023
```
Allow user to use custom calibration data for quantization
```
  2a3e0fa1
- Add wikitext example · b9ab9a64
  Casper authored Sep 15, 2023
  
  b9ab9a64
- Move shuffling up for datasets loaded with load_dataset · d832a218
  Casper authored Sep 15, 2023
  
  d832a218
- Actually pass args to calib function · a9cef34b
  Casper authored Sep 15, 2023
  
  a9cef34b
14 Sep, 2023 4 commits
- Custom data example · 69d31edc
  Casper authored Sep 14, 2023
  
  69d31edc
- Pass split and text_column arguments to calib_data function · 84e82744
  Casper authored Sep 14, 2023
  
  84e82744
- Merge branch 'main' into pr/27 · 077f39a0
  Casper authored Sep 14, 2023
  
  077f39a0
- Allow huggingface datasets · faedf517
  Casper authored Sep 14, 2023
  
  faedf517
13 Sep, 2023 13 commits
- Add notes to example · 720a1fce
  Casper Hansen authored Sep 13, 2023
  
  720a1fce
- Correct example · affd1906
  Casper Hansen authored Sep 13, 2023
  
  affd1906
- Unify using variable safetensors · 219ccb33
  Casper Hansen authored Sep 13, 2023
  
  219ccb33
- Add generation example for safetensors · bb455d7c
  Casper Hansen authored Sep 13, 2023
  
  bb455d7c
- Fix ignore patterns · 7ba31254
  Casper Hansen authored Sep 13, 2023
  
  7ba31254
- Add safetensors support · db8ba322
  Casper Hansen authored Sep 13, 2023
  
  db8ba322
- Implement saving sharded weights + safetensors · 97d38e29
  Casper Hansen authored Sep 13, 2023
  
  97d38e29
- Update README.md · d76125bf
  Casper authored Sep 13, 2023
  
  d76125bf
- Merge pull request #40 from casper-hansen/new_kernel · 1b0af2d3
  Casper authored Sep 13, 2023
```
[NEW] GEMV kernel implementation
```
  1b0af2d3
- Update README · f264ebb3
  Casper Hansen authored Sep 13, 2023
  
  f264ebb3
- Fix Falcon benchmark format · dc99d2fc
  Casper Hansen authored Sep 13, 2023
  
  dc99d2fc
- Update benchmarks · 7cf3c790
  Casper Hansen authored Sep 13, 2023
  
  7cf3c790
- More consistent benchmark · 86ea8df1
  Casper Hansen authored Sep 13, 2023
  
  86ea8df1