Update contribution instructions.

Also provide shortcuts in a Makefile.

CONTRIBUTING.md

@@ -100,7 +100,8 @@ Follow these steps to start contributing:
 
 1. Fork the [repository](https://github.com/huggingface/transformers) by
    clicking on the 'Fork' button on the repository's page. This creates a copy of the code
-   under your github user account.
+   under your GitHub user account.
+
 2. Clone your fork to your local disk, and add the base repository as a remote:
 
    ```bash
@@ -123,8 +124,38 @@ Follow these steps to start contributing:
    $ pip install -e .[dev]
    ```
 
-5. Develop the features on your branch. Add changed files using `git add` and
-   then `git commit` to record your changes locally:
+   Right now, we need an unreleased version of `isort` to avoid a
+   [bug](https://github.com/timothycrosley/isort/pull/1000):
+
+   ```bash
+   $ pip install -U git+git://github.com/timothycrosley/isort.git@e63ae06ec7d70b06df9e528357650281a3d3ec22#egg=isort
+   ```
+
+5. Develop the features on your branch.
+
+   As you work on the features, you should make sure that the test suite
+   passes:
+
+   ```bash
+   $ make test
+   ```
+
+   `transformers` relies on `black` and `isort` to format its source code
+   consistently. After you make changes, format them with:
+
+   ```bash
+   $ make style
+   ```
+
+   `transformers` also uses `flake8` to check for coding mistakes. Quality
+   control runs in CI, however you can also run the same checks with:
+
+   ```bash
+   $ make quality
+   ```
+
+   Once you're happy with your changes, add changed files using `git add` and
+   make a commit with `git commit` to record your changes locally:
 
    ```bash
    $ git add modified_file.py
@@ -132,9 +163,10 @@ Follow these steps to start contributing:
    ```
 
    Please write [good commit
-   messages](https://chris.beams.io/posts/git-commit/). It
-   is a good idea to sync your copy of the code with the original repository
-   regularly. This way you can quickly account for changes:
+   messages](https://chris.beams.io/posts/git-commit/).
+
+   It is a good idea to sync your copy of the code with the original
+   repository regularly. This way you can quickly account for changes:
 
    ```bash
    $ git fetch upstream
@@ -148,7 +180,7 @@ Follow these steps to start contributing:
    ```
 
 6. Once you are satisfied (**and the checklist below is happy too**), go to the
-   webpage of your fork on Github. Click on 'Pull request' to send your changes
+   webpage of your fork on GitHub. Click on 'Pull request' to send your changes
    to the project maintainers for review.
 
 7. It's ok if maintainers ask you for changes. It happens to core contributors
@@ -171,6 +203,53 @@ Follow these steps to start contributing:
 6. All public methods must have informative docstrings;
 
 
+### Tests
+
+You can run 🤗 Transformers tests with `unittest` or `pytest`.
+
+We like `pytest` and `pytest-xdist` because it's faster. From the root of the
+repository, here's how to run tests with `pytest` for the library:
+
+```bash
+$ python -m pytest -n auto --dist=loadfile -s -v ./tests/
+```
+
+and for the examples:
+
+```bash
+$ pip install -r examples/requirements.txt  # only needed the first time
+$ python -m pytest -n auto --dist=loadfile -s -v ./examples/
+```
+
+In fact, that's how `make test` and `make test-examples` are implemented!
+
+You can specify a smaller set of tests in order to test only the feature
+you're working on.
+
+By default, slow tests are skipped. Set the `RUN_SLOW` environment variable to
+`yes` to run them. This will download many gigabytes of models — make sure you
+have enough disk space and a good Internet connection, or a lot of patience!
+
+```bash
+$ RUN_SLOW=yes python -m pytest -n auto --dist=loadfile -s -v ./tests/
+$ RUN_SLOW=yes python -m pytest -n auto --dist=loadfile -s -v ./examples/
+```
+
+Likewise, set the `RUN_CUSTOM_TOKENIZERS` environment variable to `yes` to run
+tests for custom tokenizers, which don't run by default either.
+
+🤗 Transformers uses `pytest` as a test runner only. It doesn't use any
+`pytest`-specific features in the test suite itself.
+
+This means `unittest` is fully supported. Here's how to run tests with
+`unittest`:
+
+```bash
+$ python -m unittest discover -s tests -t . -v
+$ python -m unittest discover -s examples -t examples -v
+```
+
+
 ### Style guide
 
 For documentation strings, `transformers` follows the [google

Makefile

-.PHONY: style
+.PHONY: quality style test test-examples
+
+# Check that source code meets quality standards
+
+quality:
+	black --check --line-length 119 examples templates tests src utils
+	isort --check-only --recursive examples templates tests src utils
+	flake8 examples templates tests src utils
+
+# Format source code automatically
 
 style:
 	black --line-length 119 examples templates tests src utils
 	isort --recursive examples templates tests src utils
+
+# Run tests for the library
+
+test:
+	python -m pytest -n auto --dist=loadfile -s -v ./tests/
+
+# Run tests for examples
+
+test-examples:
+	python -m pytest -n auto --dist=loadfile -s -v ./examples/

README.md

@@ -66,6 +66,12 @@ Choose the right framework for every part of a model's lifetime
 
 This repo is tested on Python 3.5+, PyTorch 1.0.0+ and TensorFlow 2.0.0-rc1
 
+You should install 🤗 Transformers in a [virtual environment](https://docs.python.org/3/library/venv.html). If you're unfamiliar with Python virtual environments, check out the [user guide](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/).
+
+Create a virtual environment with the version of Python you're going to use and activate it.
+
+Now, if you want to use 🤗 Transformers, you can install it with pip. If you'd like to play with the examples, you must install it from source.
+
 ### With pip
 
 First you need to install one of, or both, TensorFlow 2.0 and PyTorch.
@@ -85,43 +91,48 @@ Please refer to [TensorFlow installation page](https://www.tensorflow.org/instal
 When TensorFlow 2.0 and/or PyTorch has been installed, you can install from source by cloning the repository and running:
 
 ```bash
+git clone https://github.com/huggingface/transformers
+cd transformers
 pip install [--editable] .
 ```
 
+When you update the repository, you should upgrade the transformers installation and its dependencies as follows:
+
+```bash
+git pull
+pip install --upgrade [--editable] .
+```
+
 ### Run the examples
 
 Examples are included in the repository but are not shipped with the library.
-Therefore, in order to run the latest versions of the examples you also need to install from source. To do so, create a new virtual environment and follow these steps:
 
-```bash
-git clone https://github.com/huggingface/transformers
-cd transformers
-pip install [--editable] .
-```
+Therefore, in order to run the latest versions of the examples, you need to install from source, as described above.
 
-### Tests
+Look at the [README](https://github.com/huggingface/transformers/blob/master/examples/README.md) for how to run examples.
 
-A series of tests are included for the library and the example scripts. Library tests can be found in the [tests folder](https://github.com/huggingface/transformers/tree/master/tests) and examples tests in the [examples folder](https://github.com/huggingface/transformers/tree/master/examples).
+### Tests
 
-These tests can be run using `unittest` or `pytest` (install pytest if needed with `pip install pytest`).
+A series of tests are included for the library and for some example scripts. Library tests can be found in the [tests folder](https://github.com/huggingface/transformers/tree/master/tests) and examples tests in the [examples folder](https://github.com/huggingface/transformers/tree/master/examples).
 
 Depending on which framework is installed (TensorFlow 2.0 and/or PyTorch), the irrelevant tests will be skipped. Ensure that both frameworks are installed if you want to execute all tests.
 
-You can run the tests from the root of the cloned repository with the commands:
+Here's the easiest way to run tests for the library:
 
 ```bash
-python -m unittest discover -s tests -t . -v
-python -m unittest discover -s examples -t examples -v
+pip install -e .[testing]
+make test
 ```
 
-or
+and for the examples:
 
 ```bash
-python -m pytest -sv ./tests/
-python -m pytest -sv ./examples/
+pip install -e .[testing]
+pip install -r examples/requirements.txt
+make test-examples
 ```
 
-By default, slow tests are skipped. Set the `RUN_SLOW` environment variable to `yes` to run them.
+For details, refer to the [contributing guide](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#tests).
 
 ### Do you want to run a Transformer model on a mobile device?
 

docs/source/installation.md

@@ -24,23 +24,7 @@ pip install [--editable] .
 
 An extensive test suite is included to test the library behavior and several examples. Library tests can be found in the [tests folder](https://github.com/huggingface/transformers/tree/master/tests) and examples tests in the [examples folder](https://github.com/huggingface/transformers/tree/master/examples).
 
-Tests can be run using `unittest` or `pytest` (install pytest if needed with `pip install pytest`).
-
-Run all the tests from the root of the cloned repository with the commands:
-
-```bash
-python -m unittest discover -s tests -t . -v
-python -m unittest discover -s examples -t examples -v
-```
-
-or
-
-``` bash
-python -m pytest -sv ./tests/
-python -m pytest -sv ./examples/
-```
-
-By default, slow tests are skipped. Set the `RUN_SLOW` environment variable to `yes` to run them.
+Refer to the [contributing guide](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#tests) for details about running tests.
 
 ## OpenAI GPT original tokenization workflow