Skip to content

GitLab

Commit d4614729 authored by Lysandre's avatar Lysandre
Browse files

return for SQuAD [BLACKED]

parent f24a228a
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 172 additions and 110 deletions
transformers/data/processors/glue.py
View file @ d4614729
...@@ -133,7 +133,7 @@ def glue_convert_examples_to_features(examples, tokenizer, ...@@ -133,7 +133,7 @@ def glue_convert_examples_to_features(examples, tokenizer,
if is_tf_available() and is_tf_dataset: if is_tf_available() and is_tf_dataset:
def gen(): def gen():
for ex in features: for ex in features:
yield ({'input_ids': ex.input_ids, yield ({'input_ids': ex.input_ids,
'attention_mask': ex.attention_mask, 'attention_mask': ex.attention_mask,
'token_type_ids': ex.token_type_ids}, 'token_type_ids': ex.token_type_ids},
ex.label) ex.label)
......
transformers/data/processors/squad.py
View file @ d4614729
This diff is collapsed.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment