Commit 29cbab98 authored by Aymeric Augustin's avatar Aymeric Augustin
Browse files

Parallelize tests on Circle CI.

Set the number of CPUs manually based on the Circle CI resource class,
or else we're getting 36 CPUs, which is far too much (perhaps that's
the underlying hardware and not what Circle CI allocates to us).

Don't parallelize the custom tokenizers tests because they take less
than one second to run and parallelization actually makes them slower.
parent a4c9338b
...@@ -11,9 +11,9 @@ jobs: ...@@ -11,9 +11,9 @@ jobs:
- run: sudo pip install torch - run: sudo pip install torch
- run: sudo pip install tensorflow - run: sudo pip install tensorflow
- run: sudo pip install --progress-bar off . - run: sudo pip install --progress-bar off .
- run: sudo pip install pytest codecov pytest-cov - run: sudo pip install pytest codecov pytest-cov pytest-xdist
- run: sudo pip install tensorboardX scikit-learn - run: sudo pip install tensorboardX scikit-learn
- run: python -m pytest -sv ./transformers/tests/ --cov - run: python -m pytest -n 8 -s -v ./transformers/tests/ --cov
- run: codecov - run: codecov
build_py3_torch: build_py3_torch:
working_directory: ~/transformers working_directory: ~/transformers
...@@ -25,10 +25,10 @@ jobs: ...@@ -25,10 +25,10 @@ jobs:
- checkout - checkout
- run: sudo pip install torch - run: sudo pip install torch
- run: sudo pip install --progress-bar off . - run: sudo pip install --progress-bar off .
- run: sudo pip install pytest codecov pytest-cov - run: sudo pip install pytest codecov pytest-cov pytest-xdist
- run: sudo pip install tensorboardX scikit-learn - run: sudo pip install tensorboardX scikit-learn
- run: python -m pytest -sv ./transformers/tests/ --cov - run: python -m pytest -n 8 -s -v ./transformers/tests/ --cov
- run: python -m pytest -sv ./examples/ - run: python -m pytest -n 8 -s -v ./examples/
- run: codecov - run: codecov
build_py3_tf: build_py3_tf:
working_directory: ~/transformers working_directory: ~/transformers
...@@ -40,9 +40,9 @@ jobs: ...@@ -40,9 +40,9 @@ jobs:
- checkout - checkout
- run: sudo pip install tensorflow - run: sudo pip install tensorflow
- run: sudo pip install --progress-bar off . - run: sudo pip install --progress-bar off .
- run: sudo pip install pytest codecov pytest-cov - run: sudo pip install pytest codecov pytest-cov pytest-xdist
- run: sudo pip install tensorboardX scikit-learn - run: sudo pip install tensorboardX scikit-learn
- run: python -m pytest -sv ./transformers/tests/ --cov - run: python -m pytest -n 8 -s -v ./transformers/tests/ --cov
- run: codecov - run: codecov
build_py3_custom_tokenizers: build_py3_custom_tokenizers:
working_directory: ~/transformers working_directory: ~/transformers
...@@ -51,7 +51,7 @@ jobs: ...@@ -51,7 +51,7 @@ jobs:
steps: steps:
- checkout - checkout
- run: sudo pip install --progress-bar off . - run: sudo pip install --progress-bar off .
- run: sudo pip install pytest - run: sudo pip install pytest pytest-xdist
- run: sudo pip install mecab-python3 - run: sudo pip install mecab-python3
- run: RUN_CUSTOM_TOKENIZERS=1 python -m pytest -sv ./transformers/tests/tokenization_bert_japanese_test.py - run: RUN_CUSTOM_TOKENIZERS=1 python -m pytest -sv ./transformers/tests/tokenization_bert_japanese_test.py
deploy_doc: deploy_doc:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment