Commits · abbe65adfc42b5fb2ee1b57a73ea76f24f81252c · OpenDAS / bitsandbytes

10 Mar, 2023 3 commits
- beta2 is actually accessible in kOptimizerStatic8bit1StateBlockwise · abbe65ad
  Phil Wang authored Mar 10, 2023
  
  abbe65ad
- do the epsilon beta2 switcharoo within the cuda code, and not within the... · c99b44f7
  Phil Wang authored Mar 10, 2023
```
do the epsilon beta2 switcharoo within the cuda code, and not within the python class (so that the state dict still makes sense)
```
  c99b44f7
- swap the order in which momentum and parameters are updated in ops.cu · 8618bed0
  Phil Wang authored Mar 10, 2023
  
  8618bed0
09 Mar, 2023 6 commits
- missed adagrad · c5582724
  Phil Wang authored Mar 09, 2023
  
  c5582724
- fix weight decay for lion to be decoupled, using a switch · af034309
  Phil Wang authored Mar 09, 2023
  
  af034309
- remove something rmsprop specific · ead570a4
  Phil Wang authored Mar 09, 2023
  
  ead570a4
- use epsilon as beta2 for lion, complete most of the logic in kernel.cu for all functions · c83888aa
  Phil Wang authored Mar 09, 2023
  
  c83888aa
- add a sign function, for lion · 64bb1ae8
  Phil Wang authored Mar 09, 2023
  
  64bb1ae8
- do a bunch of typical bookkeeping before getting to main lion logic · cb4c3c8c
  Phil Wang authored Mar 09, 2023
  
  cb4c3c8c
20 Nov, 2022 1 commit
- Added additional blocksizes: {64, 128, 256}. · c059bd28
  Tim Dettmers authored Nov 20, 2022
  
  c059bd28
08 Nov, 2022 1 commit
- Fixed bug in cpu quant; faster GPU dequant. · 08fa2e7b
  Tim Dettmers authored Nov 07, 2022
  
  08fa2e7b
07 Nov, 2022 1 commit
- Added blocksizes 2048, 1024, and 512 to blockwise quant. · 6bc2b992
  Tim Dettmers authored Nov 06, 2022
  
  6bc2b992
27 Oct, 2022 1 commit
- Remove trailing whitespace & ensure newline at EOF · 1eec77d3
  Tom Aarsen authored Oct 27, 2022
  
  1eec77d3
13 Sep, 2022 1 commit
- Fixed cpu blockwise quantization for small input tensors. · c05dd42d
  Tim Dettmers authored Sep 13, 2022
  
  c05dd42d
11 Sep, 2022 1 commit
- Fixed 2^31 max size issue for cpu blockwise quant. · 19a7adca
  Tim Dettmers authored Sep 11, 2022
  
  19a7adca
23 Aug, 2022 1 commit
- Fixed issue where Pascal was not displaying proper error. · ee5b947e
  Tim Dettmers authored Aug 23, 2022
  
  ee5b947e
17 Aug, 2022 1 commit
- Enhanced error handling in CUDA SETUP failures. · a6664de0
  Tim Dettmers authored Aug 16, 2022
  
  a6664de0
16 Aug, 2022 2 commits
- Added fused bias in dequant_mm. · dede3430
  Tim Dettmers authored Aug 16, 2022
  
  dede3430
- Removed storage() from get_ptr; added boilerplate for bias dequant_mm. · 1ed2fa2f
  Tim Dettmers authored Aug 16, 2022
  
  1ed2fa2f
06 Aug, 2022 1 commit
- Removed faulty asserts. · a4532c59
  Tim Dettmers authored Aug 06, 2022
  
  a4532c59
03 Aug, 2022 2 commits
- Added fixes for the case that matmullt dim A is zero, e.g. [0, 768]. · 451fd950
  Tim Dettmers authored Aug 03, 2022
  
  451fd950
- Added CUDA block assert and is_on_gpu check. · 2f01865a
  Tim Dettmers authored Aug 03, 2022
  
  2f01865a
27 Jul, 2022 2 commits
- Added col_ampere outlier extraction kernel. · 32fa459e
  Tim Dettmers authored Jul 26, 2022
  
  32fa459e
- Working outlier extraction for Turing. · bcab99ec
  Tim Dettmers authored Jul 26, 2022
  
  bcab99ec
26 Jul, 2022 3 commits
- Boilerplate and test for extract_outliers. · cbb901ac
  Tim Dettmers authored Jul 26, 2022
  
  cbb901ac
- Fixed cpuonly build. · 953b7285
  Tim Dettmers authored Jul 26, 2022
  
  953b7285
- Some progress on build script; added multi-cuda install script. · 9268dc9d
  Tim Dettmers authored Jul 25, 2022
  
  9268dc9d
25 Jul, 2022 1 commit
- Fixed makefile; fixed Ampere igemmlt_8 bug. · 8b1fd32e
  Tim Dettmers authored Jul 25, 2022
  
  8b1fd32e
22 Jul, 2022 2 commits
- Fixed rowcol synchronization bug. · 7d2ecd30
  Tim Dettmers authored Jul 22, 2022
  
  7d2ecd30
- Most tests passing. · c771b3a7
  Tim Dettmers authored Jul 22, 2022
  
  c771b3a7
01 Jul, 2022 5 commits
- Reduce diff · 025824d2
  Max Ryabinin authored Jul 01, 2022
  
  025824d2
- Reduce diff · 575aa698
  Max Ryabinin authored Jul 01, 2022
  
  575aa698
- Reduce diff · 4d1d5b56
  Max Ryabinin authored Jul 01, 2022
  
  4d1d5b56
- Reduce diff · 31ce1b37
  Max Ryabinin authored Jul 01, 2022
  
  31ce1b37
- Add a CPU-only build option · 8258b436
  Max Ryabinin authored Jul 01, 2022
  
  8258b436
29 Nov, 2021 1 commit
- Added AdamW. #10 #13 · 2f8083bd
  Tim Dettmers authored Nov 28, 2021
  
  2f8083bd
10 Nov, 2021 1 commit
- Added adagrad with tests (no clipping). · 8b3c0f35
  Tim Dettmers authored Nov 10, 2021
  
  8b3c0f35
22 Oct, 2021 1 commit
- Added compilation from source instructions; easier compilation. · 0fb378b4
  Tim Dettmers authored Oct 21, 2021
  
  0fb378b4
21 Oct, 2021 2 commits
- Added skip_zeros; tests are passing. · a6eae2e7
  Tim Dettmers authored Oct 20, 2021
  
  a6eae2e7
- Initial plumbing for skip_zeros. · bb34fd50
  Tim Dettmers authored Oct 20, 2021
  
  bb34fd50