-
Arthur authored
* update `tokenizers` version pin * force tokenizers>=0.15 * use 0.14 Co-authored-by:
Lysandre <lysandre@huggingface.co> --------- Co-authored-by:
Lysandre <lysandre@huggingface.co>
3d1a7bf4
* update `tokenizers` version pin * force tokenizers>=0.15 * use 0.14 Co-authored-by:Lysandre <lysandre@huggingface.co> --------- Co-authored-by:
Lysandre <lysandre@huggingface.co>