Skip to content

GitLab

Commit 2d8ec5a6 authored by LysandreJik's avatar LysandreJik
Browse files

Changed warning to be more explicit 

Co-authored by: julien_c <chaumond@gmail.com>
parent 75635072
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
pytorch_transformers/tokenization_utils.py
View file @ 2d8ec5a6
...@@ -691,7 +691,7 @@ class PreTrainedTokenizer(object): ...@@ -691,7 +691,7 @@ class PreTrainedTokenizer(object):
return self.add_special_tokens_sentences_pair(first_sentence_tokens, second_sentence_tokens, output_mask) return self.add_special_tokens_sentences_pair(first_sentence_tokens, second_sentence_tokens, output_mask)
else: else:
if output_mask: if output_mask:
logger.warning("Can't output mask if no special tokens are involved. Please call the method with add_special_tokens set to True.") logger.warning("Can't output mask if you're not joining two sequences.")
return first_sentence_tokens, second_sentence_tokens return first_sentence_tokens, second_sentence_tokens
def add_special_tokens_single_sentence(self, token_ids): def add_special_tokens_single_sentence(self, token_ids):
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment