------------------------------------ GPT模型汇总 ------------------------------------ 下表汇总介绍了目前PaddleNLP支持的GPT模型对应预训练权重。 关于模型的具体细节可以参考对应链接。 +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | Pretrained Weight | Language | Details of the model | +==================================================================================+==============+==================================================================================+ |``gpt-cpm-large-cn`` | Chinese | 32-layer, 2560-hidden, | | | | 32-heads, 2.6B parameters. | | | | Trained on Chinese text. | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ |``gpt-cpm-small-cn-distill`` | Chinese | 12-layer, 768-hidden, | | | | 12-heads, 109M parameters. | | | | The model distilled from | | | | the GPT model ``gpt-cpm-large-cn`` | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ |``gpt2-en`` | English | 12-layer, 768-hidden, | | | | 12-heads, 117M parameters. | | | | Trained on English text. | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ |``gpt2-medium-en`` | English | 24-layer, 1024-hidden, | | | | 16-heads, 345M parameters. | | | | Trained on English text. | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ |``gpt2-large-en`` | English | 36-layer, 1280-hidden, | | | | 20-heads, 774M parameters. | | | | Trained on English text. | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ |``gpt2-xl-en`` | English | 48-layer, 1600-hidden, | | | | 25-heads, 1558M parameters. | | | | Trained on English text. | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ |``microsoft/DialoGPT-medium`` | English | 24-layer, 1024-hidden, | | | | 16-heads, 354M parameters. | | | | Trained on English text. | | | | | | | | Please refer to: | | | | `microsoft/DialoGPT-medium`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ |``uer/gpt2-chinese-poem`` | Chinese | 12-layer, 768-hidden, | | | | 12-heads, 103M parameters. | | | | Trained on Chinese poetry corpus. | | | | | | | | Please refer to: | | | | `uer/gpt2-chinese-poem`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``distilgpt2`` | English | Please refer to: | | | | `distilgpt2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``w11wo/javanese-gpt2-small-imdb`` | Javanese | Please refer to: | | | | `w11wo/javanese-gpt2-small-imdb`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``remotejob/tweetsDISTILGPT2fi_v4`` | English | Please refer to: | | | | `remotejob/tweetsDISTILGPT2fi_v4`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``TrLOX/gpt2-tdk`` | English | Please refer to: | | | | `TrLOX/gpt2-tdk`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``huggingtweets/slime_machine`` | English | Please refer to: | | | | `huggingtweets/slime_machine`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``microsoft/DialoGPT-small`` | English | Please refer to: | | | | `microsoft/DialoGPT-small`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``sberbank-ai/rugpt3large_based_on_gpt2`` | Russian | Please refer to: | | | | `sberbank-ai/rugpt3large_based_on_gpt2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``sshleifer/tiny-gpt2`` | English | Please refer to: | | | | `sshleifer/tiny-gpt2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``microsoft/DialoGPT-large`` | English | Please refer to: | | | | `microsoft/DialoGPT-large`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``sberbank-ai/rugpt3small_based_on_gpt2`` | Russian | Please refer to: | | | | `sberbank-ai/rugpt3small_based_on_gpt2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``uw-hai/polyjuice`` | English | Please refer to: | | | | `uw-hai/polyjuice`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``NYTK/text-generation-poem-petofi-gpt2-small-hungarian`` | Hungarian | Please refer to: | | | | `NYTK/text-generation-poem-petofi-gpt2-small-hungarian`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``microsoft/DialogRPT-human-vs-rand`` | English | Please refer to: | | | | `microsoft/DialogRPT-human-vs-rand`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``hf-internal-testing/tiny-random-gpt2`` | English | Please refer to: | | | | `hf-internal-testing/tiny-random-gpt2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``Grossmend/rudialogpt3_medium_based_on_gpt2`` | Russian | Please refer to: | | | | `Grossmend/rudialogpt3_medium_based_on_gpt2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``pranavpsv/genre-story-generator-v2`` | English | Please refer to: | | | | `pranavpsv/genre-story-generator-v2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``microsoft/DialogRPT-updown`` | English | Please refer to: | | | | `microsoft/DialogRPT-updown`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``microsoft/DialogRPT-human-vs-machine`` | English | Please refer to: | | | | `microsoft/DialogRPT-human-vs-machine`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``pierreguillou/gpt2-small-portuguese`` | Portuguese | Please refer to: | | | | `pierreguillou/gpt2-small-portuguese`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``mrm8488/GPT-2-finetuned-covid-bio-medrxiv`` | English | Please refer to: | | | | `mrm8488/GPT-2-finetuned-covid-bio-medrxiv`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``anonymous-german-nlp/german-gpt2`` | German | Please refer to: | | | | `anonymous-german-nlp/german-gpt2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``microsoft/CodeGPT-small-py`` | English | Please refer to: | | | | `microsoft/CodeGPT-small-py`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``antoiloui/belgpt2`` | French | Please refer to: | | | | `antoiloui/belgpt2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``benjamin/gerpt2`` | German | Please refer to: | | | | `benjamin/gerpt2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``asi/gpt-fr-cased-small`` | French | Please refer to: | | | | `asi/gpt-fr-cased-small`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``microsoft/CodeGPT-small-java-adaptedGPT2`` | English | Please refer to: | | | | `microsoft/CodeGPT-small-java-adaptedGPT2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``GroNLP/gpt2-small-dutch`` | Dutch | Please refer to: | | | | `GroNLP/gpt2-small-dutch`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``lvwerra/gpt2-imdb`` | English | Please refer to: | | | | `lvwerra/gpt2-imdb`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``DeepESP/gpt2-spanish`` | Spanish | Please refer to: | | | | `DeepESP/gpt2-spanish`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``microsoft/CodeGPT-small-py-adaptedGPT2`` | English | Please refer to: | | | | `microsoft/CodeGPT-small-py-adaptedGPT2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``microsoft/DialogRPT-width`` | English | Please refer to: | | | | `microsoft/DialogRPT-width`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``dbddv01/gpt2-french-small`` | French | Please refer to: | | | | `dbddv01/gpt2-french-small`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``GroNLP/gpt2-small-italian`` | Italian | Please refer to: | | | | `GroNLP/gpt2-small-italian`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``flax-community/gpt2-medium-persian`` | Persian | Please refer to: | | | | `flax-community/gpt2-medium-persian`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``microsoft/DialogRPT-depth`` | English | Please refer to: | | | | `microsoft/DialogRPT-depth`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``Nokia/nlgp-natural`` | English | Please refer to: | | | | `Nokia/nlgp-natural`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``macedonizer/hr-gpt2`` | English | Please refer to: | | | | `macedonizer/hr-gpt2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``mrm8488/GPT-2-finetuned-common_gen`` | English | Please refer to: | | | | `mrm8488/GPT-2-finetuned-common_gen`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``pranavpsv/gpt2-genre-story-generator`` | English | Please refer to: | | | | `pranavpsv/gpt2-genre-story-generator`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``rbhushan/distilgpt2-finetuned-wikitext2`` | English | Please refer to: | | | | `rbhushan/distilgpt2-finetuned-wikitext2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``readerbench/RoGPT2-large`` | Romanian | Please refer to: | | | | `readerbench/RoGPT2-large`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``flax-community/gpt2-small-indonesian`` | Indonesian | Please refer to: | | | | `flax-community/gpt2-small-indonesian`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``HooshvareLab/gpt2-fa`` | Persian | Please refer to: | | | | `HooshvareLab/gpt2-fa`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``cahya/gpt2-small-indonesian-522M`` | Indonesian | Please refer to: | | | | `cahya/gpt2-small-indonesian-522M`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``DingleyMaillotUrgell/homer-bot`` | English | Please refer to: | | | | `DingleyMaillotUrgell/homer-bot`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``datificate/gpt2-small-spanish`` | Spanish | Please refer to: | | | | `datificate/gpt2-small-spanish`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``ericzhou/tsundere_v1`` | English | Please refer to: | | | | `ericzhou/tsundere_v1`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``huggingtweets/wwm_shakespeare`` | English | Please refer to: | | | | `huggingtweets/wwm_shakespeare`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``SIC98/GPT2-python-code-generator`` | English | Please refer to: | | | | `SIC98/GPT2-python-code-generator`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``GroNLP/gpt2-small-italian-embeddings`` | Italian | Please refer to: | | | | `GroNLP/gpt2-small-italian-embeddings`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``huggingtweets/hel_ql-shahdashrf_-sinnerslayerr-witheredstrings`` | English | Please refer to: | | | | `huggingtweets/hel_ql-shahdashrf_-sinnerslayerr-witheredstrings`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``salesken/grammar_correction`` | English | Please refer to: | | | | `salesken/grammar_correction`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``flax-community/gpt2-medium-indonesian`` | Indonesian | Please refer to: | | | | `flax-community/gpt2-medium-indonesian`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``gorkemgoknar/gpt2-small-turkish`` | Turkish | Please refer to: | | | | `gorkemgoknar/gpt2-small-turkish`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``deepparag/DumBot`` | English | Please refer to: | | | | `deepparag/DumBot`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``jcblaise/gpt2-tagalog`` | Tagalog | Please refer to: | | | | `jcblaise/gpt2-tagalog`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``BigSalmon/InformalToFormalLincoln21`` | English | Please refer to: | | | | `BigSalmon/InformalToFormalLincoln21`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``LorenzoDeMattei/GePpeTto`` | English | Please refer to: | | | | `LorenzoDeMattei/GePpeTto`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``macedonizer/sr-gpt2`` | English | Please refer to: | | | | `macedonizer/sr-gpt2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``indonesian-nlp/gpt2`` | English | Please refer to: | | | | `indonesian-nlp/gpt2`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``ceostroff/harry-potter-gpt2-fanfiction`` | English | Please refer to: | | | | `ceostroff/harry-potter-gpt2-fanfiction`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``akhooli/gpt2-small-arabic-poetry`` | Arabic | Please refer to: | | | | `akhooli/gpt2-small-arabic-poetry`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``asi/gpt-fr-cased-base`` | French | Please refer to: | | | | `asi/gpt-fr-cased-base`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``congcongwang/gpt2_medium_fine_tuned_coder`` | English | Please refer to: | | | | `congcongwang/gpt2_medium_fine_tuned_coder`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ | ``cambridgeltl/simctg_wikitext103`` | English | Please refer to: | | | | `cambridgeltl/simctg_wikitext103`_ | +----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+ .. _microsoft/DialoGPT-medium: https://huggingface.co/microsoft/DialoGPT-medium .. _uer/gpt2-chinese-poem: https://huggingface.co/uer/gpt2-chinese-poem .. _distilgpt2: https://huggingface.co/distilgpt2 .. _w11wo/javanese-gpt2-small-imdb: https://huggingface.co/w11wo/javanese-gpt2-small-imdb .. _remotejob/tweetsDISTILGPT2fi_v4: https://huggingface.co/remotejob/tweetsDISTILGPT2fi_v4 .. _TrLOX/gpt2-tdk: https://huggingface.co/TrLOX/gpt2-tdk .. _huggingtweets/slime_machine: https://huggingface.co/huggingtweets/slime_machine .. _microsoft/DialoGPT-small: https://huggingface.co/microsoft/DialoGPT-small .. _sberbank-ai/rugpt3large_based_on_gpt2: https://huggingface.co/sberbank-ai/rugpt3large_based_on_gpt2 .. _sshleifer/tiny-gpt2: https://huggingface.co/sshleifer/tiny-gpt2 .. _microsoft/DialoGPT-large: https://huggingface.co/microsoft/DialoGPT-large .. _sberbank-ai/rugpt3small_based_on_gpt2: https://huggingface.co/sberbank-ai/rugpt3small_based_on_gpt2 .. _uw-hai/polyjuice: https://huggingface.co/uw-hai/polyjuice .. _NYTK/text-generation-poem-petofi-gpt2-small-hungarian: https://huggingface.co/NYTK/text-generation-poem-petofi-gpt2-small-hungarian .. _microsoft/DialogRPT-human-vs-rand: https://huggingface.co/microsoft/DialogRPT-human-vs-rand .. _hf-internal-testing/tiny-random-gpt2: https://huggingface.co/hf-internal-testing/tiny-random-gpt2 .. _Grossmend/rudialogpt3_medium_based_on_gpt2: https://huggingface.co/Grossmend/rudialogpt3_medium_based_on_gpt2 .. _pranavpsv/genre-story-generator-v2: https://huggingface.co/pranavpsv/genre-story-generator-v2 .. _microsoft/DialogRPT-updown: https://huggingface.co/microsoft/DialogRPT-updown .. _microsoft/DialogRPT-human-vs-machine: https://huggingface.co/microsoft/DialogRPT-human-vs-machine .. _pierreguillou/gpt2-small-portuguese: https://huggingface.co/pierreguillou/gpt2-small-portuguese .. _mrm8488/GPT-2-finetuned-covid-bio-medrxiv: https://huggingface.co/mrm8488/GPT-2-finetuned-covid-bio-medrxiv .. _anonymous-german-nlp/german-gpt2: https://huggingface.co/anonymous-german-nlp/german-gpt2 .. _microsoft/CodeGPT-small-py: https://huggingface.co/microsoft/CodeGPT-small-py .. _antoiloui/belgpt2: https://huggingface.co/antoiloui/belgpt2 .. _benjamin/gerpt2: https://huggingface.co/benjamin/gerpt2 .. _asi/gpt-fr-cased-small: https://huggingface.co/asi/gpt-fr-cased-small .. _microsoft/CodeGPT-small-java-adaptedGPT2: https://huggingface.co/microsoft/CodeGPT-small-java-adaptedGPT2 .. _GroNLP/gpt2-small-dutch: https://huggingface.co/GroNLP/gpt2-small-dutch .. _lvwerra/gpt2-imdb: https://huggingface.co/lvwerra/gpt2-imdb .. _DeepESP/gpt2-spanish: https://huggingface.co/DeepESP/gpt2-spanish .. _microsoft/CodeGPT-small-py-adaptedGPT2: https://huggingface.co/microsoft/CodeGPT-small-py-adaptedGPT2 .. _microsoft/DialogRPT-width: https://huggingface.co/microsoft/DialogRPT-width .. _dbddv01/gpt2-french-small: https://huggingface.co/dbddv01/gpt2-french-small .. _GroNLP/gpt2-small-italian: https://huggingface.co/GroNLP/gpt2-small-italian .. _flax-community/gpt2-medium-persian: https://huggingface.co/flax-community/gpt2-medium-persian .. _microsoft/DialogRPT-depth: https://huggingface.co/microsoft/DialogRPT-depth .. _Nokia/nlgp-natural: https://huggingface.co/Nokia/nlgp-natural .. _macedonizer/hr-gpt2: https://huggingface.co/macedonizer/hr-gpt2 .. _mrm8488/GPT-2-finetuned-common_gen: https://huggingface.co/mrm8488/GPT-2-finetuned-common_gen .. _pranavpsv/gpt2-genre-story-generator: https://huggingface.co/pranavpsv/gpt2-genre-story-generator .. _rbhushan/distilgpt2-finetuned-wikitext2: https://huggingface.co/rbhushan/distilgpt2-finetuned-wikitext2 .. _readerbench/RoGPT2-large: https://huggingface.co/readerbench/RoGPT2-large .. _flax-community/gpt2-small-indonesian: https://huggingface.co/flax-community/gpt2-small-indonesian .. _HooshvareLab/gpt2-fa: https://huggingface.co/HooshvareLab/gpt2-fa .. _cahya/gpt2-small-indonesian-522M: https://huggingface.co/cahya/gpt2-small-indonesian-522M .. _DingleyMaillotUrgell/homer-bot: https://huggingface.co/DingleyMaillotUrgell/homer-bot .. _datificate/gpt2-small-spanish: https://huggingface.co/datificate/gpt2-small-spanish .. _ericzhou/tsundere_v1: https://huggingface.co/ericzhou/tsundere_v1 .. _huggingtweets/wwm_shakespeare: https://huggingface.co/huggingtweets/wwm_shakespeare .. _SIC98/GPT2-python-code-generator: https://huggingface.co/SIC98/GPT2-python-code-generator .. _GroNLP/gpt2-small-italian-embeddings: https://huggingface.co/GroNLP/gpt2-small-italian-embeddings .. _huggingtweets/hel_ql-shahdashrf_-sinnerslayerr-witheredstrings: https://huggingface.co/huggingtweets/hel_ql-shahdashrf_-sinnerslayerr-witheredstrings .. _salesken/grammar_correction: https://huggingface.co/salesken/grammar_correction .. _flax-community/gpt2-medium-indonesian: https://huggingface.co/flax-community/gpt2-medium-indonesian .. _gorkemgoknar/gpt2-small-turkish: https://huggingface.co/gorkemgoknar/gpt2-small-turkish .. _deepparag/DumBot: https://huggingface.co/deepparag/DumBot .. _jcblaise/gpt2-tagalog: https://huggingface.co/jcblaise/gpt2-tagalog .. _BigSalmon/InformalToFormalLincoln21: https://huggingface.co/BigSalmon/InformalToFormalLincoln21 .. _LorenzoDeMattei/GePpeTto: https://huggingface.co/LorenzoDeMattei/GePpeTto .. _macedonizer/sr-gpt2: https://huggingface.co/macedonizer/sr-gpt2 .. _indonesian-nlp/gpt2: https://huggingface.co/indonesian-nlp/gpt2 .. _ceostroff/harry-potter-gpt2-fanfiction: https://huggingface.co/ceostroff/harry-potter-gpt2-fanfiction .. _akhooli/gpt2-small-arabic-poetry: https://huggingface.co/akhooli/gpt2-small-arabic-poetry .. _asi/gpt-fr-cased-base: https://huggingface.co/asi/gpt-fr-cased-base .. _congcongwang/gpt2_medium_fine_tuned_coder: https://huggingface.co/congcongwang/gpt2_medium_fine_tuned_coder .. _cambridgeltl/simctg_wikitext103: https://huggingface.co/cambridgeltl/simctg_wikitext103