Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
flash-attention
Commits
ece8f05d
"doc/vscode:/vscode.git/clone" did not exist on "b4d0ea7ee6a12e7e0c798d34c2f3894c5caf33ec"
Commit
ece8f05d
authored
Dec 15, 2022
by
Tri Dao
Browse files
[Docs] Mention PubMedGPT
parent
04c4c610
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
0 deletions
+5
-0
usage.md
usage.md
+5
-0
No files found.
usage.md
View file @
ece8f05d
...
...
@@ -45,6 +45,11 @@ yields the fastest BERT training on cloud instances in MLPerf training 2.0 (June
## Language model training & inference
-
[
PubMedGPT 2.7B
](
https://crfm.stanford.edu/2022/12/15/pubmedgpt.html
)
, a
domain-specific LLM for biomedicine, by Stanford CRFM, trained on
[
MosaicML
](
https://www.mosaicml.com/blog/introducing-pubmed-gpt
)
Cloud. Just
using FlashAttention nearly halves the total training time.
-
Meta's
[
AITemplate
](
https://ai.facebook.com/blog/gpu-inference-engine-nvidia-amd-open-source/
)
uses FlashAttention as part of their approach to speed up Transformer
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment