examples/lm_finetuning/pregenerate_training_data.py · 7d1ae644ef367dee1bcdf07daec1d78058d6f16a · chenpangpang / transformers · GitLab

Find file Blame History Permalink

Added a --reduce_memory option to the training script to keep training · 7d1ae644
Matthew Carrigan authored Mar 21, 2019
```
data on disc as a memmap rather than in memory
```
7d1ae644

pregenerate_training_data.py 12.4 KB