Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
bitsandbytes
Commits
78007346
Commit
78007346
authored
Jul 23, 2024
by
Titus von Koeller
Browse files
Changelog: add explanation r. QLoRA mem savings
parent
a7c08afd
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
8 additions
and
0 deletions
+8
-0
CHANGELOG.md
CHANGELOG.md
+8
-0
No files found.
CHANGELOG.md
View file @
78007346
### 0.43.2
This release is quite significant as the QLoRA bug fix big implications for higher
`seqlen`
and batch sizes.
For each sequence (i.e. batch size increase of one) we expect memory savings of:
-
405B: 39GB for
`seqlen=1024`
, and 4888GB for
`seqlen=128,00`
-
70B: 10.1GB for
`seqlen=1024`
and 1258GB for
`seqlen=128,00`
This was due to activations being unnecessary for frozen parameters, yet the memory for them was still erroneously allocated due to the now fixed bug.
#### Improvements:
-
docs: FSDP+QLoRA and CPU install guide (#1211 #1227, thanks @stevhliu)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment