Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
ColossalAI
Commits
0ff8406b
Unverified
Commit
0ff8406b
authored
Mar 03, 2023
by
ver217
Committed by
GitHub
Mar 03, 2023
Browse files
[chatgpt] allow shard init and display warning (#2986)
parent
f5ca0397
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
2 deletions
+3
-2
applications/ChatGPT/chatgpt/trainer/strategies/colossalai.py
...ications/ChatGPT/chatgpt/trainer/strategies/colossalai.py
+3
-2
No files found.
applications/ChatGPT/chatgpt/trainer/strategies/colossalai.py
View file @
0ff8406b
...
...
@@ -79,8 +79,9 @@ class ColossalAIStrategy(DDPStrategy):
self
.
stage
=
stage
# TODO(ver217): support shard_init when using from_pretrained()
if
shard_init
:
warnings
.
warn
(
f
'Shard init is not supported yet. Ignore.'
)
shard_init
=
False
warnings
.
warn
(
f
'Shard init is not supported model.from_pretrained() yet. Please load weights after strategy.prepare()'
)
self
.
shard_init
=
shard_init
self
.
gemini_config
=
dict
(
device
=
get_current_device
(),
placement_policy
=
placement_policy
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment