Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
781220ac
".circleci/vscode:/vscode.git/clone" did not exist on "1ecbc249b5eed3a2e0e765ac0046e2bf9c231884"
Unverified
Commit
781220ac
authored
Feb 08, 2021
by
Stas Bekman
Committed by
GitHub
Feb 08, 2021
Browse files
transition to new tests dir (#10080)
parent
84acf0c7
Changes
2
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
7 additions
and
2 deletions
+7
-2
examples/tests/deepspeed/ds_config.json
examples/tests/deepspeed/ds_config.json
+0
-0
examples/tests/deepspeed/test_deepspeed.py
examples/tests/deepspeed/test_deepspeed.py
+7
-2
No files found.
examples/
seq2seq
/ds_config.json
→
examples/
tests/deepspeed
/ds_config.json
View file @
781220ac
File moved
examples/
seq2seq
/test_deepspeed.py
→
examples/
tests/deepspeed
/test_deepspeed.py
View file @
781220ac
...
...
@@ -12,6 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import
json
import
os
import
unittest
...
...
@@ -19,13 +20,17 @@ from transformers.integrations import is_deepspeed_available
from
transformers.testing_utils
import
TestCasePlus
,
execute_subprocess_async
,
require_torch_multi_gpu
from
transformers.trainer_callback
import
TrainerState
from
transformers.trainer_utils
import
set_seed
from
utils
import
load_json
set_seed
(
42
)
MBART_TINY
=
"sshleifer/tiny-mbart"
def
load_json
(
path
):
with
open
(
path
)
as
f
:
return
json
.
load
(
f
)
# a candidate for testing_utils
def
require_deepspeed
(
test_case
):
"""
...
...
@@ -122,7 +127,7 @@ class TestDeepSpeed(TestCasePlus):
ds_args
=
f
"--deepspeed
{
self
.
test_file_dir_str
}
/ds_config.json"
.
split
()
distributed_args
=
f
"""
{
self
.
test_file_dir
}
/finetune_trainer.py
{
self
.
test_file_dir
}
/
../../seq2seq/
finetune_trainer.py
"""
.
split
()
cmd
=
[
"deepspeed"
]
+
distributed_args
+
args
+
ds_args
# keep for quick debug
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment