Commit 78007346 authored by Titus von Koeller's avatar Titus von Koeller
Browse files

Changelog: add explanation r. QLoRA mem savings

parent a7c08afd
### 0.43.2 ### 0.43.2
This release is quite significant as the QLoRA bug fix big implications for higher `seqlen` and batch sizes.
For each sequence (i.e. batch size increase of one) we expect memory savings of:
- 405B: 39GB for `seqlen=1024`, and 4888GB for `seqlen=128,00`
- 70B: 10.1GB for `seqlen=1024` and 1258GB for `seqlen=128,00`
This was due to activations being unnecessary for frozen parameters, yet the memory for them was still erroneously allocated due to the now fixed bug.
#### Improvements: #### Improvements:
- docs: FSDP+QLoRA and CPU install guide (#1211 #1227, thanks @stevhliu) - docs: FSDP+QLoRA and CPU install guide (#1211 #1227, thanks @stevhliu)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment