Skip to content

GitLab

"tests/vscode:/vscode.git/clone" did not exist on "92f6791095491e44c5712e14f00f2e19b52dc9f6"
Unverified Commit 160c64c6 authored by LuGY's avatar LuGY Committed by GitHub
Browse files

[example] fix bucket size in example of gpt gemini (#4028)

parent 727c4598
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
examples/language/gpt/gemini/train_gpt_demo.py
View file @ 160c64c6
...@@ -250,7 +250,7 @@ def main(): ...@@ -250,7 +250,7 @@ def main():
plugin = None plugin = None
if args.distplan.startswith("CAI_ZeRO"): if args.distplan.startswith("CAI_ZeRO"):
plugin = LowLevelZeroPlugin(stage=zero_stage, plugin = LowLevelZeroPlugin(stage=zero_stage,
reduce_bucket_size_in_m=12 * 1024 * 1024, reduce_bucket_size_in_m=12,
overlap_communication=True, overlap_communication=True,
verbose=True) verbose=True)
elif args.distplan == "CAI_Gemini": elif args.distplan == "CAI_Gemini":
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment