Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
vocab.txt 85 Bytes
Newer Older
Patrick von Platen's avatar
[AutoTokenizer] Allow creation of tokenizers by tokenizer type (#13668)  
Patrick von Platen committed
1
2
3
4
5
6
7
8
9
10 
[PAD]
[SEP]
[MASK]
[CLS]
[unused3]
[unused4]
[unused5]
[unused6]
[unused7]
[unused8]