Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "45d5af2416f53940e48100754bdfbb6360c4e586"
Switch branch/tag
Find file Normal viewHistoryPermalink
__init__.py 97 Bytes
Newer Older
Jared Casper's avatar
Initial commit of multiprocess preprocess and extracted copy of fairseq's indexed_dataset.  
Jared Casper committed
1
2 
from . import indexed_dataset
from .bert_tokenization import FullTokenizer as FullBertTokenizer
Mohammad Shoeybi's avatar
moved albert to bert  
Mohammad Shoeybi committed
3