examples/flax/language-modeling/run_mlm_flax.py · 1e8140caad248ba7ea8797f0575f7931bd4b12a6 · chenpangpang / transformers

Fix RESOURCE_EXHAUSTED error when dealing with large datasets in Flax example scripts (#18069) · 1e8140ca

Duong A. Nguyen authored Jul 11, 2022

* Fix RESOURCE_EXHAUSTED error for large datasets on Flax example scripts

* using np.permutation for creating batch_idx

* train_samples_idx -> training_samples_idx

* fix type hints

1e8140ca

run_mlm_flax.py 36.8 KB

Replace run_mlm_flax.py