Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
ced7284a
Unverified
Commit
ced7284a
authored
Mar 31, 2021
by
Philipp Schmid
Committed by
GitHub
Mar 31, 2021
Browse files
Sagemaker test fix (#10987)
* wrong makefile command * ddp test fix
parent
645f45c4
Changes
2
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
3 additions
and
3 deletions
+3
-3
tests/sagemaker/README.md
tests/sagemaker/README.md
+2
-2
tests/sagemaker/test_multi_node_data_parallel.py
tests/sagemaker/test_multi_node_data_parallel.py
+1
-1
No files found.
tests/sagemaker/README.md
View file @
ced7284a
...
...
@@ -18,7 +18,7 @@ git+https://github.com/huggingface/transformers.git@v4.5.0.rc0 # install master
After we adjusted the
`requirements.txt`
we can run Amazon SageMaker tests with:
```
bash
AWS_PROFILE
=
<enter-your-profile> make sagemaker
-test
AWS_PROFILE
=
<enter-your-profile> make
test-
sagemaker
```
These tests take around 10-15 minutes to finish. Preferably make a screenshot of the successfully ran tests.
...
...
@@ -88,7 +88,7 @@ tensorflow-gpu==2.5.0 # for tensorflow
After we adjusted the
`requirements.txt`
we can run Amazon SageMaker tests with.
```
bash
AWS_PROFILE
=
<enter-your-profile> make sagemaker
-test
AWS_PROFILE
=
<enter-your-profile> make
test-
sagemaker
```
These tests take around 10-15 minutes to finish. Preferably make a screenshot of the successfully ran tests.
...
...
tests/sagemaker/test_multi_node_data_parallel.py
View file @
ced7284a
...
...
@@ -58,7 +58,7 @@ class MultiNodeTest(unittest.TestCase):
def
create_estimator
(
self
,
instance_count
):
job_name
=
f
"
{
self
.
env
.
base_job_name
}
-
{
instance_count
}
-
{
'ddp'
if
'ddp'
in
self
.
script
else
'smd'
}
"
# distributed data settings
distribution
=
{
"smdistributed"
:
{
"dataparallel"
:
{
"enabled"
:
True
}}}
distribution
=
{
"smdistributed"
:
{
"dataparallel"
:
{
"enabled"
:
True
}}}
if
self
.
script
!=
"run_ddp.py"
else
None
# creates estimator
return
HuggingFace
(
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment