GitLab

This PR adds initial FSDP support for training QLoRA models. It enables basic FSDP and CPU Offload support, with low memory training via FSDP.sync_module_states option unsupported.

This PR builds off of #840 commit 8278fca and BNB FSDP by @TimDettmers and @Titus-von-Koeller.

An example of using this PR to finetune QLoRA models with FSDP can be found in the demo repo: AnswerDotAi/fsdp_qlora.

* Minimal changes for fp32 4bit storage from BNB commit 8278fca

* Params4bit with selectable storage dtype

* possible fix for double quantizing linear weight & quant storage dtype

* minor fixes in Params4bit for peft tests

* remove redundant

* add float16

* update test

* Remove float16 quant cast as there are fp32, bf16, & fp16 quant kernels

---------
Co-authored-by: Kerem Turgutlu <keremturgutlu@gmail.com>

Delete .github/workflows/delete_doc_commment.yml

fix array index out of bounds in kgetColRowStats

Fix typo "quanitze"

Added scipy to requirements.txt

Make sure bitsandbytes handles permission errors in the right order

Add version attribute as per Python convention

Fix parameter name in error message

Update README.md

Update README.md

Fixed missing `Embedding` export

Fix typo in test_optim.py

improve `make clean` target

doc: Fix typo in how_to_use_nonpytorch_cuda.md

Robustness fix: don't break in case of directories without read permission

Add env var related to google systems to ignored list

Small fix to getting started code in README

Fixed wget link for installing cuda

FIX missing closing quote in README example

Fix issue #901: Failing PEFT integration tests by reversing PR#867 from jph00/patch-2

tests.

This reverts commit 2ee289fb, reversing
changes made to 744d36f7.

Fix for 4bit without compress_statistics.

chore: update dev setup