GitLab

Fix Language Modeling Loss

fix sample_doc

Fix cosine schedule

Fix links in README

Fix typo in example code

Fixes to the TensorFlow conversion tool

Modify 'unambigiously' to 'unambiguously'

If the value of rand_end is returned from the randint function, the value of sampled_doc_index that matches current_idx is returned from searchsorted.

example:
cumsum_max = {int64} 30
doc_cumsum = {ndarray} [ 5  7 11 19 30]
doc_lengths = {list} <class 'list'>: [5, 2, 4, 8, 11]
if current_idx  = 1,
rand_start = 7
rand_end = 35
sentence_index = randint(7, 35) % cumsum_max
if randint return 35, sentence_index becomes 5.
if sentence_index is 5, np.searchsorted returns 1 equal to current_index.

fix lm_finetuning's link

Added remaining GLUE tasks to 'run_classifier.py'

Remove padding_idx from position_embeddings and token_type_embeddings

 add tqdm to the process of eval in examples/run_swag.py

Minor change in README

Add full language model fine-tuning

order.

data on disc as a memmap rather than in memory