Skip to content

GitLab

Commit caf1d116 authored by LysandreJik's avatar LysandreJik
Browse files

Closing bracket in DistilBERT's token count.

parent fe8fb10b
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
examples/distillation/scripts/token_counts.py
View file @ caf1d116
...@@ -24,7 +24,7 @@ from utils import logger ...@@ -24,7 +24,7 @@ from utils import logger
if __name__ == '__main__': if __name__ == '__main__':
parser = argparse.ArgumentParser(description="Token Counts for smoothing the masking probabilities in MLM (cf XLM/word2vec)") parser = argparse.ArgumentParser(description="Token Counts for smoothing the masking probabilities in MLM (cf XLM/word2vec)")
parser.add_argument("--data_file", type=str, default="data/dump.bert-base-uncased.pickle", parser.add_argument("--data_file", type=str, default="data/dump.bert-base-uncased.pickle",
help="The binarized dataset." help="The binarized dataset.")
parser.add_argument("--token_counts_dump", type=str, default="data/token_counts.bert-base-uncased.pickle", parser.add_argument("--token_counts_dump", type=str, default="data/token_counts.bert-base-uncased.pickle",
help="The dump file.") help="The dump file.")
parser.add_argument("--vocab_size", default=30522, type=int) parser.add_argument("--vocab_size", default=30522, type=int)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment