"...git@developer.sourcefind.cn:chenpangpang/transformers.git" did not exist on "a64bcb564dbc2a6329235016613a888ca21d513b"
modify double considering special tokens in `language_modeling.py` (#11275)
* Update language_modeling.py in "class TextDatasetForNextSentencePrediction(Dataset)", double considering "self.tokenizer.num_special_tokens_to_add(pair=True)" so, i remove self.block_size, and add parameter for "def create_examples_from_document". like "class LineByLineWithSOPTextDataset" do * Update language_modeling.py
Showing
Please register or sign in to comment