Unverified Commit fd690283 authored by Sylvain Gugger's avatar Sylvain Gugger Committed by GitHub
Browse files

Properly indent block_size (#12070)

parent 49bee0ae
......@@ -364,7 +364,7 @@ def main():
f"The tokenizer picked seems to have a very large `model_max_length` ({tokenizer.model_max_length}). "
"Picking 1024 instead. You can change that default value by passing --block_size xxx."
)
block_size = 1024
block_size = 1024
else:
if data_args.block_size > tokenizer.model_max_length:
logger.warning(
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment