GitLab

Fix --reduce_memory in finetune_on_pregenerated

Fix indentation weirdness in GPT-2 example.

Learning rate schedules improvement + extension

Pulling commits from main repo

- removed unused plotting code
- using ABC for LRSchedule
- added some schedule object init tests

Updating network handling

Fix indentation for unconditional generation

Fix gradient overflow issue during attention mask

[run_gpt2.py] temperature should be a float, not int

Gpt2 tokenization

This fix is in reference to issue #382. GPT2 can now be trained in mixed precision, which I've confirmed with testing. I also tested unconditional generation on multiple seeds before and after changing 1e10 to 1e4 and there was no difference. Please let me know if there is anything else I can do to make this pull request better. Thanks for all your work!

Better serialization for Tokenizers and Configuration classes - Also fix #466

Clean up GPT and GPT-2 losses computation