Unverified Commit c4734840 authored by Thomas Wolf's avatar Thomas Wolf Committed by GitHub
Browse files

Fix the tokenization warning noted in #5505 (#5550)

* fix warning

* style and quality
parent 1bbc28be
......@@ -588,8 +588,8 @@ class PreTrainedTokenizer(PreTrainedTokenizerBase):
first_ids,
second_ids,
add_special_tokens=add_special_tokens,
padding_strategy=PaddingStrategy.DO_NOT_PAD, # we pad in batch afterward
truncation_strategy=truncation_strategy,
padding=PaddingStrategy.DO_NOT_PAD.value, # we pad in batch afterward
truncation=truncation_strategy.value,
max_length=max_length,
stride=stride,
pad_to_multiple_of=None, # we pad in batch afterward
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment