GitLab

adding conversion script

adding first draft of modeling & tokenization

adding placeholder for test files

bunch of changes

registering the tokenizer/model/etc

tests

change link; something is very VERY wrong here

weird end-of-word thingy going on

i think the tokenization works now ; wrote the unit tests

overall structure works;load w next

the monster is alive!

works after some cleanup as well

adding emacs autosave to gitignore

currently only supporting the 48 layer one; seems to infer fine on my macbook

cleanup

fixing some documentation

fixing some documentation

tests passing?

now works on CUDA also

adding greedy?

adding greedy sampling

works well

fix link

Fix some typos

Fix tensorflow_dataset glue support

Update run_lm_finetuning.py

The previous method, just as phrased, did not exist in the class.

`glue_convert_examples_to_features` assumed that tensorflow_dataset
examples contains the features `'sentence1'` and `'sentence2'`. This
commit encapsulates the choice of features in the glue processor and
uses that to parse examples.

Just some typos

faster dataset building

Add small  note about the output of hidden states (closes #1332)

Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>

Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>