• Stas Bekman's avatar
    add a summary report flag for run_examples on CI (#6035) · daa5dd12
    Stas Bekman authored
    Currently, it's hard to derive which example tests were run on CI, and which weren't. Adding `-rA` flag to `pytest`, will now include a summary like:
    
    ```
    ==================================================================== short test summary info =====================================================================
    PASSED examples/test_examples.py::ExamplesTests::test_generation
    PASSED examples/test_examples.py::ExamplesTests::test_run_glue
    PASSED examples/test_examples.py::ExamplesTests::test_run_language_modeling
    PASSED examples/test_examples.py::ExamplesTests::test_run_squad
    FAILED examples/test_examples.py::ExamplesTests::test_run_pl_glue - AttributeError: 'Namespace' object has no attribute 'gpus'
    ============================================================ 1 failed, 4 passed, 8 warnings in 42.96s ============================================================
    ```
    which makes it easier to validate whether some example is being covered by CI or not.
    daa5dd12
config.yml 4.99 KB