Unverified Commit abb1fa3f authored by Patrick von Platen's avatar Patrick von Platen Committed by GitHub
Browse files

Update README.md

parent 0ccbfd28
...@@ -5,8 +5,7 @@ Crime and Punishment is a novel written by Fyodor Dostoevsky and was translated ...@@ -5,8 +5,7 @@ Crime and Punishment is a novel written by Fyodor Dostoevsky and was translated
Crime and Punishment training data was taken from `gs://trax-ml/reformer/crime-and-punishment-2554.txt` and contains Crime and Punishment training data was taken from `gs://trax-ml/reformer/crime-and-punishment-2554.txt` and contains
roughly 0.5M tokens. roughly 0.5M tokens.
The ReformerLM model was trained in flax using colab notebook proposed by authors: https://colab.research.google.com/github/google/trax/blob/master/trax/models/reformer/text_generation.ipynb . The ReformerLM model was trained in flax using colab notebook proposed by authors: https://colab.research.google.com/github/google/trax/blob/master/trax/models/reformer/text_generation.ipynb and the weights were converted to Hugging Face's PyTorch ReformerLM model `ReformerModelWithLMHead`.
and the weights were converted to Hugging Face's PyTorch ReformerLM model `ReformerModelWithLMHead`.
The model is a language model that operates on small sub-word units. Text can be generated as follows: The model is a language model that operates on small sub-word units. Text can be generated as follows:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment