GitLab

If a user or contributor ran `pip install -e .` on transformers < 3.0,
pip created a transformers.egg-info directory next to the transformers
directory at the root of the repository.

In transformers 3.0, the source is in a `src` subdirectory.
`pip install -e .` creates a transformers.egg-info directory there.
However, pip will still pick transformers.egg-info from the previous
location. This is a bug: https://github.com/pypa/pip/issues/5466

Users and contributors are likely to hit this problem because the
documentation for transformers 3.0 relies heavily on extra_requires
which didn't exist in earlier versions, so aren't defined in a stale
transformers.egg-info directory.

If such a directory exists, remove it. It's autogenerated, gitignored
and not supposed to contain anything of value.

Also provide shortcuts in a Makefile.

It's superseded by the "docs" extras.

It's redundant with setup.py and, also, incomplete (e.g. numpy).

It was generated once, likely in a non-reproducible way (pip freeze
in a contributor's local environment), and never updated.

Take advantage of this to simplify the Circle CI configuration.

Don't bother with tensorboardX: it's a fallback for PyTorch < 1.1.0.

Every other file uses this pattern.

requirements.txt isn't up to date.

This should fix API docs, which went AWOL with yesterday's changes.

Fix doc link in README

fix error due to wrong argument name to Tensor.scatter()

Remove support for Python 2

On Python 3, `open is io.open`.

Improve repository structure

Implement some Python best practices

This is the same change as for (TF)CommonTestCases for modeling.

It isn't imported anywhere.

I suspect the wrapper classes were created in order to prevent the
abstract base class (TF)CommonModelTester from being included in test
discovery and running, because that would fail.

I solved this by replacing the abstract base class with a mixin.

Code changes are just de-indenting and automatic reformattings
performed by black to use the extra line space.

This construct isn't used anymore these days.

Running python tests/test_foo.py puts the tests/ directory on
PYTHONPATH, which isn't representative of how we run tests.

Use python -m unittest tests/test_foo.py instead.

This prevents transformers from being importable simply because the CWD
is the root of the git repository, while not being importable from other
directories. That led to inconsistent behavior, especially in examples.

Once you fetch this commit, in your dev environment, you must run:

    $ pip uninstall transformers
    $ pip install -e .

These libraries aren't always installed in the virtual environment where
isort is running. Declaring them properly avoids mixing these
third-party imports with local imports.

I don't want to consider it a dependency of transformers, but it's
usually there in local development and usually not there in CI.