Unverified Commit daa5dd12 authored by Stas Bekman's avatar Stas Bekman Committed by GitHub
Browse files

add a summary report flag for run_examples on CI (#6035)

Currently, it's hard to derive which example tests were run on CI, and which weren't. Adding `-rA` flag to `pytest`, will now include a summary like:

```
==================================================================== short test summary info =====================================================================
PASSED examples/test_examples.py::ExamplesTests::test_generation
PASSED examples/test_examples.py::ExamplesTests::test_run_glue
PASSED examples/test_examples.py::ExamplesTests::test_run_language_modeling
PASSED examples/test_examples.py::ExamplesTests::test_run_squad
FAILED examples/test_examples.py::ExamplesTests::test_run_pl_glue - AttributeError: 'Namespace' object has no attribute 'gpus'
============================================================ 1 failed, 4 passed, 8 warnings in 42.96s ============================================================
```
which makes it easier to validate whether some example is being covered by CI or not.
parent c69ea5ef
...@@ -73,7 +73,7 @@ jobs: ...@@ -73,7 +73,7 @@ jobs:
- checkout - checkout
- run: sudo pip install .[sklearn,torch,testing] - run: sudo pip install .[sklearn,torch,testing]
- run: sudo pip install -r examples/requirements.txt - run: sudo pip install -r examples/requirements.txt
- run: python -m pytest -n 8 --dist=loadfile -s ./examples/ | tee output.txt - run: python -m pytest -n 8 --dist=loadfile -rA -s ./examples/ | tee output.txt
- store_artifacts: - store_artifacts:
path: ~/transformers/output.txt path: ~/transformers/output.txt
destination: test_output.txt destination: test_output.txt
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment