- 20 Feb, 2020 3 commits
-
-
Scott Gigante authored
-
Funtowicz Morgan authored
* Remove warning when pad_to_max_length is not set. Signed-off-by:
Morgan Funtowicz <morgan@huggingface.co> * Move RoberTa warning to RoberTa and not GPT2 base tokenizer. Signed-off-by:
-
Funtowicz Morgan authored
Signed-off-by:
-
- 19 Feb, 2020 6 commits
-
-
Funtowicz Morgan authored
* Correctly return the tuple of generated file(s) when calling save_pretrained Signed-off-by:
-
Lysandre authored
-
Lysandre authored
-
Funtowicz Morgan authored
* Override build_inputs_with_special_tokens for fast impl + unittest. Signed-off-by:
-
Lysandre authored
Welcome Rust Tokenizers
-
Funtowicz Morgan authored
* Implemented fast version of tokenizers Signed-off-by:
🚀 Signed-off-by:MOI Anthony <xn1t0x@gmail.com>
-
- 14 Feb, 2020 1 commit
-
-
Julien Chaumond authored
-
- 13 Feb, 2020 2 commits
-
-
Joe Davison authored
* Preserve spaces in GPT-2 tokenizers Preserves spaces after special tokens in GPT-2 and inhereted (RoBERTa) tokenizers, enabling correct BPE encoding. Automatically inserts a space in front of first token in encode function when adding special tokens. * Add tokenization preprocessing method * Add framework argument to pipeline factory Also fixes pipeline test issue. Each test input now treated as a distinct sequence.
-
Sam Shleifer authored
* activations.py contains a mapping from string to activation function * resolves some `gelu` vs `gelu_new` ambiguity
-
- 11 Feb, 2020 2 commits
-
-
Oleksiy Syvokon authored
PyTorch < 1.3 requires multiplication operands to be of the same type. This was violated when using default attention mask (i.e., attention_mask=None in arguments) given BERT in the decoder mode. In particular, this was breaking Model2Model and made tutorial from the quickstart failing.
-
jiyeon authored
-
- 10 Feb, 2020 3 commits
- 07 Feb, 2020 8 commits
-
-
Lysandre authored
-
Lysandre authored
-
VictorSanh authored
-
VictorSanh authored
-
monologg authored
-
Ari authored
-
thomwolf authored
-
thomwolf authored
-
- 06 Feb, 2020 3 commits
-
-
thomwolf authored
-
thomwolf authored
-
dchurchwell authored
Changed vocabulary save function. Variable name was inconsistent, causing an error to be thrown when passing a file name instead of a directory.
-
- 05 Feb, 2020 1 commit
-
-
James Betker authored
This prevents the model from being saved, and who knows what else.
-
- 04 Feb, 2020 2 commits
- 03 Feb, 2020 3 commits
-
-
Julien Chaumond authored
cc @mfuntowicz does this seem correct?
-
Lysandre authored
-
Lysandre authored
Masked indices should have -1 and not -100. Updating documentation + scripts that were forgotten
-
- 01 Feb, 2020 1 commit
-
-
Bram Vanroy authored
* add "info" command to CLI As a convenience, add the info directive to CLI. Running `python transformers-cli info` will return a string containing the transformers version, platform, python version, PT/TF version and GPU support * Swap f-strings for .format Still supporting 3.5 so can't use f-strings (sad face) * Add reference in issue to CLI * Add the expected fields to issue template This way, people can still add the information manually if they want. (Though I fear they'll just ignore it.) * Remove heading from output * black-ify * order of imports Should ensure isort test passes * use is_X_available over import..pass * style * fix copy-paste bug * Rename command info -> env Also adds the command to CONTRIBUTING.md in "Did you find a bug" section
-
- 31 Jan, 2020 5 commits
-
-
Lysandre authored
-
Lysandre authored
cc @julien-c
-
Lysandre authored
The FlauBERT configuration file inherits from XLMConfig, and is recognized as such when loading from AutoModels as the XLMConfig is checked before the FlaubertConfig. Changing the order solves this problem, but a test should be added.
-
Lysandre authored
-
Julien Chaumond authored
* [Umberto] model shortcuts cc @loretoparisi @simonefrancia see #2485 * Ensure that tokenizers will be correctly configured
-
