Skip to content

GitLab

Unverified Commit 8f8bbd4a authored by Thomas Wolf's avatar Thomas Wolf Committed by GitHub
Browse files

Merge pull request #244 from deepset-ai/prettify_lm_masking 

Avoid confusion of inplace LM masking
parents e2d53d95 ce75b169
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
examples/run_lm_finetuning.py
View file @ 8f8bbd4a
...@@ -328,8 +328,8 @@ def convert_example_to_features(example, max_seq_length, tokenizer): ...@@ -328,8 +328,8 @@ def convert_example_to_features(example, max_seq_length, tokenizer):
# Account for [CLS], [SEP], [SEP] with "- 3" # Account for [CLS], [SEP], [SEP] with "- 3"
_truncate_seq_pair(tokens_a, tokens_b, max_seq_length - 3) _truncate_seq_pair(tokens_a, tokens_b, max_seq_length - 3)
t1_random, t1_label = random_word(tokens_a, tokenizer) tokens_a, t1_label = random_word(tokens_a, tokenizer)
t2_random, t2_label = random_word(tokens_b, tokenizer) tokens_b, t2_label = random_word(tokens_b, tokenizer)
# concatenate lm labels and account for CLS, SEP, SEP # concatenate lm labels and account for CLS, SEP, SEP
lm_label_ids = ([-1] + t1_label + [-1] + t2_label + [-1]) lm_label_ids = ([-1] + t1_label + [-1] + t2_label + [-1])
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment