-
Arthur authored
[`TokenizerFast`] `can_save_slow_tokenizer` as a property for when `vocab_file`'s folder was removed (#25626) * pad token should be None by default * fix tests * nits * check if isfile vocabfile * add warning if sp model folder was deleted * save SPM when missing folder for sloz * update the ` can_save_slow_tokenizer` to be a property * first batch * second batch * missing one
3b39b906