[model_cards] Migrate cards from this repo to model repos on huggingface.co (#9013)

* rm all model cards * Update the .rst @sgugger it is still not super crystal clear/streamlined so let me know if any ideas to make it simpler * Add a rootlevel README.md with simple instructions/context * Update docs/source/model_sharing.rst Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * make style * rm all model cards Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

[model_cards] Migrate cards from this repo to model repos on huggingface.co (#9013)
* rm all model cards * Update the .rst @sgugger it is still not super crystal clear/streamlined so let me know if any ideas to make it simpler * Add a rootlevel README.md with simple instructions/context * Update docs/source/model_sharing.rst Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * make style * rm all model cards Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
3552d0e0 · Julien Chaumond · GitHub · 29e45979 · 29e45979 · 29e45979
Unverified Commit 3552d0e0 authored Dec 12, 2020 by Julien Chaumond Committed by GitHub Dec 11, 2020
20 changed files
--- a/model_cards/microsoft/layoutlm-large-uncased/README.md
+++ b/model_cards/microsoft/layoutlm-large-uncased/README.md
-# LayoutLM
-## Model description
-LayoutLM is a simple but effective pre-training method of text and layout for document image understanding and information extraction tasks, such as form understanding and receipt understanding. LayoutLM archives the SOTA results on multiple datasets. For more details, please refer to our paper: 
-[LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318)
-Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou, [KDD 2020](https://www.kdd.org/kdd2020/accepted-papers)
-## Training data
-We pre-train LayoutLM on IIT-CDIP Test Collection 1.0\* dataset with two settings. 
-* LayoutLM-Base, Uncased (11M documents, 2 epochs): 12-layer, 768-hidden, 12-heads, 113M parameters 
-* LayoutLM-Large, Uncased (11M documents, 2 epochs): 24-layer, 1024-hidden, 16-heads, 343M parameters **(This Model)**
-## Citation
-If you find LayoutLM useful in your research, please cite the following paper:
-``` latex
-@misc{xu2019layoutlm,
-    title={LayoutLM: Pre-training of Text and Layout for Document Image Understanding},
-    author={Yiheng Xu and Minghao Li and Lei Cui and Shaohan Huang and Furu Wei and Ming Zhou},
-    year={2019},
-    eprint={1912.13318},
-    archivePrefix={arXiv},
-    primaryClass={cs.CL}
-}
-```
--- a/model_cards/microsoft/prophetnet-large-uncased-cnndm/README.md
+++ b/model_cards/microsoft/prophetnet-large-uncased-cnndm/README.md
---
-language: en
-datasets:
- cnn_dailymail
---
-## prophetnet-large-uncased-cnndm
-Fine-tuned weights(converted from [original fairseq version repo](https://github.com/microsoft/ProphetNet)) for [ProphetNet](https://arxiv.org/abs/2001.04063) on summarization task CNN/DailyMail.  
-ProphetNet is a new pre-trained language model for sequence-to-sequence learning with a novel self-supervised objective called future n-gram prediction.  
-ProphetNet is able to predict more future tokens with a n-stream decoder. The original implementation is Fairseq version at [github repo](https://github.com/microsoft/ProphetNet).   
-### Usage
-```
-from transformers import ProphetNetTokenizer, ProphetNetForConditionalGeneration, ProphetNetConfig
-model = ProphetNetForConditionalGeneration.from_pretrained('microsoft/prophetnet-large-uncased-cnndm')
-tokenizer = ProphetNetTokenizer.from_pretrained('microsoft/prophetnet-large-uncased-cnndm')
-ARTICLE_TO_SUMMARIZE = "USTC was founded in Beijing by the Chinese Academy of Sciences (CAS) in September 1958. The Director of CAS, Mr. Guo Moruo was appointed the first president of USTC. USTC's founding mission was to develop a high-level science and technology workforce, as deemed critical for development of China's economy, defense, and science and technology education. The establishment was hailed as \"A Major Event in the History of Chinese Education and Science.\" CAS has supported USTC by combining most of its institutes with the departments of the university. USTC is listed in the top 16 national key universities, becoming the youngest national key university.".lower()
-inputs = tokenizer([ARTICLE_TO_SUMMARIZE], max_length=100, return_tensors='pt')
-# Generate Summary
-summary_ids = model.generate(inputs['input_ids'], num_beams=4, max_length=512, early_stopping=True)
-tokenizer.batch_decode(summary_ids, skip_special_tokens=True)
-# should give: 'ustc was founded in beijing by the chinese academy of sciences in 1958. [X_SEP] ustc\'s mission was to develop a high - level science and technology workforce. [X_SEP] the establishment was hailed as " a major event in the history of chinese education and science "'
-```
-Here, [X_SEP] is used as a special token to seperate sentences.
-### Citation
-```bibtex
-@article{yan2020prophetnet,
-  title={Prophetnet: Predicting future n-gram for sequence-to-sequence pre-training},
-  author={Yan, Yu and Qi, Weizhen and Gong, Yeyun and Liu, Dayiheng and Duan, Nan and Chen, Jiusheng and Zhang, Ruofei and Zhou, Ming},
-  journal={arXiv preprint arXiv:2001.04063},
-  year={2020}
-}
-```
--- a/model_cards/microsoft/prophetnet-large-uncased-squad-qg/README.md
+++ b/model_cards/microsoft/prophetnet-large-uncased-squad-qg/README.md
---
-language: en
-datasets:
- squad
---
-## 
-prophetnet-large-uncased-squad-qg
-Fine-tuned weights(converted from [original fairseq version repo](https://github.com/microsoft/ProphetNet)) for [ProphetNet](https://arxiv.org/abs/2001.04063) on question generation 
-SQuAD 1.1.  
-ProphetNet is a new pre-trained language model for sequence-to-sequence learning with a novel self-supervised objective called future n-gram prediction.  
-ProphetNet is able to predict more future tokens with a n-stream decoder. The original implementation is Fairseq version at [github repo](https://github.com/microsoft/ProphetNet).   
-### Usage
-```
-from transformers import ProphetNetTokenizer, ProphetNetForConditionalGeneration, ProphetNetConfig
-model = ProphetNetForConditionalGeneration.from_pretrained('microsoft/prophetnet-large-uncased-squad-qg')
-tokenizer = ProphetNetTokenizer.from_pretrained('microsoft/prophetnet-large-uncased-squad-qg')
-FACT_TO_GENERATE_QUESTION_FROM = ""Bill Gates [SEP] Microsoft was founded by Bill Gates and Paul Allen on April 4, 1975."
-inputs = tokenizer([FACT_TO_GENERATE_QUESTION_FROM], return_tensors='pt')
-# Generate Summary
-question_ids = model.generate(inputs['input_ids'], num_beams=5, early_stopping=True)
-tokenizer.batch_decode(question_ids, skip_special_tokens=True)
-# should give: 'along with paul allen, who founded microsoft?'
-```
-### Citation
-```bibtex
-@article{yan2020prophetnet,
-  title={Prophetnet: Predicting future n-gram for sequence-to-sequence pre-training},
-  author={Yan, Yu and Qi, Weizhen and Gong, Yeyun and Liu, Dayiheng and Duan, Nan and Chen, Jiusheng and Zhang, Ruofei and Zhou, Ming},
-  journal={arXiv preprint arXiv:2001.04063},
-  year={2020}
-}
-```
--- a/model_cards/microsoft/prophetnet-large-uncased/README.md
+++ b/model_cards/microsoft/prophetnet-large-uncased/README.md
---
-language: en
---
-## prophetnet-large-uncased
-Pretrained weights for [ProphetNet](https://arxiv.org/abs/2001.04063).  
-ProphetNet is a new pre-trained language model for sequence-to-sequence learning with a novel self-supervised objective called future n-gram prediction.  
-ProphetNet is able to predict more future tokens with a n-stream decoder. The original implementation is Fairseq version at [github repo](https://github.com/microsoft/ProphetNet).   
-### Usage
-This pre-trained model can be fine-tuned on *sequence-to-sequence* tasks. The model could *e.g.* be trained on headline generation as follows:
-```python 
-from transformers import ProphetNetForConditionalGeneration, ProphetNetTokenizer
-model = ProphetNetForConditionalGeneration.from_pretrained("microsoft/prophetnet-large-uncased")
-tokenizer = ProphetNetTokenizer.from_pretrained("microsoft/prophetnet-large-uncased")
-input_str = "the us state department said wednesday it had received no formal word from bolivia that it was expelling the us ambassador there but said the charges made against him are `` baseless ."
-target_str = "us rejects charges against its ambassador in bolivia"
-input_ids = tokenizer(input_str, return_tensors="pt").input_ids
-labels = tokenizer(target_str, return_tensors="pt").input_ids
-loss = model(input_ids, labels=labels).loss
-```
-### Citation
-```bibtex
-@article{yan2020prophetnet,
-  title={Prophetnet: Predicting future n-gram for sequence-to-sequence pre-training},
-  author={Yan, Yu and Qi, Weizhen and Gong, Yeyun and Liu, Dayiheng and Duan, Nan and Chen, Jiusheng and Zhang, Ruofei and Zhou, Ming},
-  journal={arXiv preprint arXiv:2001.04063},
-  year={2020}
-}
-```
--- a/model_cards/microsoft/xprophetnet-large-wiki100-cased-xglue-ntg/README.md
+++ b/model_cards/microsoft/xprophetnet-large-wiki100-cased-xglue-ntg/README.md
-## xprophetnet-large-wiki100-cased-xglue-ntg
-Cross-lingual version [ProphetNet](https://arxiv.org/abs/2001.04063), pretrained on [wiki100 xGLUE dataset](https://arxiv.org/abs/2004.01401) and finetuned on xGLUE cross-lingual News Titles Generation task.  
-ProphetNet is a new pre-trained language model for sequence-to-sequence learning with a novel self-supervised objective called future n-gram prediction.  
-ProphetNet is able to predict more future tokens with a n-stream decoder. The original implementation is Fairseq version at [github repo](https://github.com/microsoft/ProphetNet).   
-xProphetNet is also served as the baseline model for xGLUE cross-lingual natural language generation tasks.  
-For xGLUE corss-lingual NLG tasks, xProphetNet is finetuned with English data, but inference with both English and other zero-shot language data.  
-### Usage
-A quick usage is like: 
-```
-from transformers import XLMProphetNetTokenizer, XLMProphetNetForConditionalGeneration, ProphetNetConfig
-model = ProphetNetForConditionalGeneration.from_pretrained('microsoft/xprophetnet-large-wiki100-cased-xglue-ntg')
-tokenizer = ProphetNetTokenizer.from_pretrained('microsoft/xprophetnet-large-wiki100-cased-xglue-ntg')
-EN_SENTENCE = "Microsoft Corporation intends to officially end free support for the Windows 7 operating system after January 14, 2020, according to the official portal of the organization. From that day, users of this system will not be able to receive security updates, which could make their computers vulnerable to cyber attacks."
-RU_SENTENCE = "орпорация Microsoft намерена официально прекратить бесплатную поддержку операционной системы Windows 7 после 14 января 2020 года, сообщается на официальном портале организации . С указанного дня пользователи этой системы не смогут получать обновления безопасности, из-за чего их компьютеры могут стать уязвимыми к кибератакам."
-ZH_SENTENCE = "根据该组织的官方门户网站，微软公司打算在2020年1月14日之后正式终止对Windows 7操作系统的免费支持。从那时起，该系统的用户将无法接收安全更新，这可能会使他们的计算机容易受到网络攻击。"
-inputs = tokenizer([EN_SENTENCE, RU_SENTENCE, ZH_SENTENCE], padding=True, max_length=256, return_tensors='pt')
-summary_ids = model.generate(inputs['input_ids'], num_beams=4, max_length=100, early_stopping=True)
-tokenizer.batch_decode(summary_ids, skip_special_tokens=True)
-# should give:
-# 'Microsoft to end Windows 7 free support after January 14, 2020'
-# 'Microsoft намерена прекратить бесплатную поддержку Windows 7 после 14 января 2020 года'
-# '微软终止对Windows 7操作系统的免费支持'
-```
-### Citation
-```bibtex
-@article{yan2020prophetnet,
-  title={Prophetnet: Predicting future n-gram for sequence-to-sequence pre-training},
-  author={Yan, Yu and Qi, Weizhen and Gong, Yeyun and Liu, Dayiheng and Duan, Nan and Chen, Jiusheng and Zhang, Ruofei and Zhou, Ming},
-  journal={arXiv preprint arXiv:2001.04063},
-  year={2020}
-}
-```
--- a/model_cards/microsoft/xprophetnet-large-wiki100-cased-xglue-qg/README.md
+++ b/model_cards/microsoft/xprophetnet-large-wiki100-cased-xglue-qg/README.md
-## xprophetnet-large-wiki100-cased-xglue-ntg
-Cross-lingual version [ProphetNet](https://arxiv.org/abs/2001.04063), pretrained on [wiki100 xGLUE dataset](https://arxiv.org/abs/2004.01401) and finetuned on xGLUE cross-lingual Question Generation task.  
-ProphetNet is a new pre-trained language model for sequence-to-sequence learning with a novel self-supervised objective called future n-gram prediction.  
-ProphetNet is able to predict more future tokens with a n-stream decoder. The original implementation is Fairseq version at [github repo](https://github.com/microsoft/ProphetNet).   
-xProphetNet is also served as the baseline model for xGLUE cross-lingual natural language generation tasks.  
-For xGLUE corss-lingual NLG tasks, xProphetNet is finetuned with English data, but inference with both English and other zero-shot language data.  
-### Usage
-A quick usage is like: 
-```
-from transformers import ProphetNetTokenizer, ProphetNetForConditionalGeneration, ProphetNetConfig
-model = ProphetNetForConditionalGeneration.from_pretrained('microsoft/xprophetnet-large-wiki100-cased-xglue-qg')
-tokenizer = ProphetNetTokenizer.from_pretrained('microsoft/xprophetnet-large-wiki100-cased-xglue-qg')
-EN_SENTENCE = "Google left China in 2010"
-ZH_SENTENCE = "Google在2010年离开中国"
-inputs = tokenizer([EN_SENTENCE, ZH_SENTENCE], padding=True, max_length=256, return_tensors='pt')
-summary_ids = model.generate(inputs['input_ids'], num_beams=4, max_length=100, early_stopping=True)
-print([tokenizer.decode(g) for g in summary_ids])  
-```
-### Citation
-```bibtex
-@article{yan2020prophetnet,
-  title={Prophetnet: Predicting future n-gram for sequence-to-sequence pre-training},
-  author={Yan, Yu and Qi, Weizhen and Gong, Yeyun and Liu, Dayiheng and Duan, Nan and Chen, Jiusheng and Zhang, Ruofei and Zhou, Ming},
-  journal={arXiv preprint arXiv:2001.04063},
-  year={2020}
-}
-```
--- a/model_cards/microsoft/xprophetnet-large-wiki100-cased/README.md
+++ b/model_cards/microsoft/xprophetnet-large-wiki100-cased/README.md
---
-language: multilingual
---
-## xprophetnet-large-wiki100-cased
-Cross-lingual version [ProphetNet](https://arxiv.org/abs/2001.04063), pretrained on [wiki100 xGLUE dataset](https://arxiv.org/abs/2004.01401).  
-ProphetNet is a new pre-trained language model for sequence-to-sequence learning with a novel self-supervised objective called future n-gram prediction.  
-ProphetNet is able to predict more future tokens with a n-stream decoder. The original implementation is Fairseq version at [github repo](https://github.com/microsoft/ProphetNet).   
-xProphetNet is also served as the baseline model for xGLUE cross-lingual natural language generation tasks.  
-For xGLUE corss-lingual NLG tasks, xProphetNet is finetuned with English data, but inference with both English and other zero-shot language data. 
-### Usage
-This pre-trained model can be fine-tuned on *sequence-to-sequence* tasks. The model could *e.g.* be trained on English headline generation as follows:
-```python 
-from transformers import XLMProphetNetForConditionalGeneration, XLMProphetNetTokenizer
-model = XLMProphetNetForConditionalGeneration.from_pretrained("microsoft/xprophetnet-large-wiki100-cased")
-tokenizer = XLMProphetNetTokenizer.from_pretrained("microsoft/xprophetnet-large-wiki100-cased")
-input_str = "the us state department said wednesday it had received no formal word from bolivia that it was expelling the us ambassador there but said the charges made against him are `` baseless ."
-target_str = "us rejects charges against its ambassador in bolivia"
-input_ids = tokenizer(input_str, return_tensors="pt").input_ids
-labels = tokenizer(target_str, return_tensors="pt").input_ids
-loss = model(input_ids, labels=labels).loss
-```
-Note that since this model is a multi-lingual model it can be fine-tuned on all kinds of other languages.
-### Citation
-```bibtex
-@article{yan2020prophetnet,
-  title={Prophetnet: Predicting future n-gram for sequence-to-sequence pre-training},
-  author={Yan, Yu and Qi, Weizhen and Gong, Yeyun and Liu, Dayiheng and Duan, Nan and Chen, Jiusheng and Zhang, Ruofei and Zhou, Ming},
-  journal={arXiv preprint arXiv:2001.04063},
-  year={2020}
-}
-```
--- a/model_cards/monilouise/ner_pt_br/README.md
+++ b/model_cards/monilouise/ner_pt_br/README.md
---
-language: 
- pt
-tags:
- ner
-metrics:
- f1
- accuracy
- precision
- recall
---
-# RiskData Brazilian Portuguese NER
-## Model description
-This is a finetunned version from [Neuralmind BERTimbau] (https://github.com/neuralmind-ai/portuguese-bert/blob/master/README.md) for Portuguese language.
-For more details, please see, (https://github.com/SecexSaudeTCU/noticias_ner).
-## Intended uses & limitations
-#### How to use
-```python
-from transformers import BertForTokenClassification, DistilBertTokenizerFast, pipeline
-model = BertForTokenClassification.from_pretrained('monilouise/ner_pt_br')
-tokenizer = DistilBertTokenizerFast.from_pretrained('neuralmind/bert-base-portuguese-cased'
-                                                    , model_max_length=512
-                                                    , do_lower_case=False
-                                                    )
-nlp = pipeline('ner', model=model, tokenizer=tokenizer, grouped_entities=True)
-result = nlp("O Tribunal de Contas da União é localizado em Brasília e foi fundado por Rui Barbosa.")
-```
-#### Limitations and bias
- The finetunned model was trained on a corpus with around 180 news articles crawled from Google News.  The original project's purpose was to recognize named entities in news 
-related to fraud and corruption, classifying these entities in four classes: PERSON, ORGANIZATION, PUBLIC INSITUITION and LOCAL (PESSOA, ORGANIZAÇÃO, INSTITUIÇÃO PÚBLICA and LOCAL).
-## Training data
-The training data can be found at (https://github.com/SecexSaudeTCU/noticias_ner/blob/master/dados/labeled_4_labels.jsonl).
-## Training procedure
-## Eval results
-accuracy: 0.98, 
-precision: 0.86 
-recall: 0.91
-f1: 0.88
-The score was calculated using this code:
-```python
-    def align_predictions(predictions: np.ndarray, label_ids: np.ndarray) -> Tuple[List[int], List[int]]:
-        preds = np.argmax(predictions, axis=2)
-        batch_size, seq_len = preds.shape
-        out_label_list = [[] for _ in range(batch_size)]
-        preds_list = [[] for _ in range(batch_size)]
-        for i in range(batch_size):
-            for j in range(seq_len):
-                if label_ids[i, j] != nn.CrossEntropyLoss().ignore_index:
-                    out_label_list[i].append(id2tag[label_ids[i][j]])
-                    preds_list[i].append(id2tag[preds[i][j]])
-        return preds_list, out_label_list
-    def compute_metrics(p: EvalPrediction) -> Dict:
-        preds_list, out_label_list = align_predictions(p.predictions, p.label_ids)
-        return {
-            "accuracy_score": accuracy_score(out_label_list, preds_list),
-            "precision": precision_score(out_label_list, preds_list),
-            "recall": recall_score(out_label_list, preds_list),
-            "f1": f1_score(out_label_list, preds_list),
-        }
-```
-### BibTeX entry and citation info
-For further information about BERTimbau language model:
-```bibtex
-@inproceedings{souza2020bertimbau,
-    author    = {Souza, F{\'a}bio and Nogueira, Rodrigo and Lotufo, Roberto},
-    title     = {{BERT}imbau: pretrained {BERT} models for {B}razilian {P}ortuguese},
-    booktitle = {9th Brazilian Conference on Intelligent Systems, {BRACIS}, Rio Grande do Sul, Brazil, October 20-23 (to appear)},
-    year      = {2020}
-}
-@article{souza2019portuguese,
-    title={Portuguese Named Entity Recognition using BERT-CRF},
-    author={Souza, F{\'a}bio and Nogueira, Rodrigo and Lotufo, Roberto},
-    journal={arXiv preprint arXiv:1909.10649},
-    url={http://arxiv.org/abs/1909.10649},
-    year={2019}
-}
-```
--- a/model_cards/monologg/koelectra-base-discriminator/README.md
+++ b/model_cards/monologg/koelectra-base-discriminator/README.md
---
-language: ko
---
-# KoELECTRA (Base Discriminator)
-Pretrained ELECTRA Language Model for Korean (`koelectra-base-discriminator`)
-For more detail, please see [original repository](https://github.com/monologg/KoELECTRA/blob/master/README_EN.md).
-## Usage
-### Load model and tokenizer
-```python
->>> from transformers import ElectraModel, ElectraTokenizer
->>> model = ElectraModel.from_pretrained("monologg/koelectra-base-discriminator")
->>> tokenizer = ElectraTokenizer.from_pretrained("monologg/koelectra-base-discriminator")
-```
-### Tokenizer example
-```python
->>> from transformers import ElectraTokenizer
->>> tokenizer = ElectraTokenizer.from_pretrained("monologg/koelectra-base-discriminator")
->>> tokenizer.tokenize("[CLS] 한국어 ELECTRA를 공유합니다. [SEP]")
-['[CLS]', '한국어', 'E', '##L', '##EC', '##T', '##RA', '##를', '공유', '##합니다', '.', '[SEP]']
->>> tokenizer.convert_tokens_to_ids(['[CLS]', '한국어', 'E', '##L', '##EC', '##T', '##RA', '##를', '공유', '##합니다', '.', '[SEP]'])
-[2, 18429, 41, 6240, 15229, 6204, 20894, 5689, 12622, 10690, 18, 3]
-```
-## Example using ElectraForPreTraining
-```python
-import torch
-from transformers import ElectraForPreTraining, ElectraTokenizer
-discriminator = ElectraForPreTraining.from_pretrained("monologg/koelectra-base-discriminator")
-tokenizer = ElectraTokenizer.from_pretrained("monologg/koelectra-base-discriminator")
-sentence = "나는 방금 밥을 먹었다."
-fake_sentence = "나는 내일 밥을 먹었다."
-fake_tokens = tokenizer.tokenize(fake_sentence)
-fake_inputs = tokenizer.encode(fake_sentence, return_tensors="pt")
-discriminator_outputs = discriminator(fake_inputs)
-predictions = torch.round((torch.sign(discriminator_outputs[0]) + 1) / 2)
-print(list(zip(fake_tokens, predictions.tolist()[1:-1])))
-```
--- a/model_cards/monologg/koelectra-base-generator/README.md
+++ b/model_cards/monologg/koelectra-base-generator/README.md
---
-language: ko
---
-# KoELECTRA (Base Generator)
-Pretrained ELECTRA Language Model for Korean (`koelectra-base-generator`)
-For more detail, please see [original repository](https://github.com/monologg/KoELECTRA/blob/master/README_EN.md).
-## Usage
-### Load model and tokenizer
-```python
->>> from transformers import ElectraModel, ElectraTokenizer
->>> model = ElectraModel.from_pretrained("monologg/koelectra-base-generator")
->>> tokenizer = ElectraTokenizer.from_pretrained("monologg/koelectra-base-generator")
-```
-### Tokenizer example
-```python
->>> from transformers import ElectraTokenizer
->>> tokenizer = ElectraTokenizer.from_pretrained("monologg/koelectra-base-generator")
->>> tokenizer.tokenize("[CLS] 한국어 ELECTRA를 공유합니다. [SEP]")
-['[CLS]', '한국어', 'E', '##L', '##EC', '##T', '##RA', '##를', '공유', '##합니다', '.', '[SEP]']
->>> tokenizer.convert_tokens_to_ids(['[CLS]', '한국어', 'E', '##L', '##EC', '##T', '##RA', '##를', '공유', '##합니다', '.', '[SEP]'])
-[2, 18429, 41, 6240, 15229, 6204, 20894, 5689, 12622, 10690, 18, 3]
-```
-## Example using ElectraForMaskedLM
-```python
-from transformers import pipeline
-fill_mask = pipeline(
-    "fill-mask",
-    model="monologg/koelectra-base-generator",
-    tokenizer="monologg/koelectra-base-generator"
-)
-print(fill_mask("나는 {} 밥을 먹었다.".format(fill_mask.tokenizer.mask_token)))
-```
--- a/model_cards/monologg/koelectra-small-discriminator/README.md
+++ b/model_cards/monologg/koelectra-small-discriminator/README.md
---
-language: ko
---
-# KoELECTRA (Small Discriminator)
-Pretrained ELECTRA Language Model for Korean (`koelectra-small-discriminator`)
-For more detail, please see [original repository](https://github.com/monologg/KoELECTRA/blob/master/README_EN.md).
-## Usage
-### Load model and tokenizer
-```python
->>> from transformers import ElectraModel, ElectraTokenizer
->>> model = ElectraModel.from_pretrained("monologg/koelectra-small-discriminator")
->>> tokenizer = ElectraTokenizer.from_pretrained("monologg/koelectra-small-discriminator")
-```
-### Tokenizer example
-```python
->>> from transformers import ElectraTokenizer
->>> tokenizer = ElectraTokenizer.from_pretrained("monologg/koelectra-small-discriminator")
->>> tokenizer.tokenize("[CLS] 한국어 ELECTRA를 공유합니다. [SEP]")
-['[CLS]', '한국어', 'E', '##L', '##EC', '##T', '##RA', '##를', '공유', '##합니다', '.', '[SEP]']
->>> tokenizer.convert_tokens_to_ids(['[CLS]', '한국어', 'E', '##L', '##EC', '##T', '##RA', '##를', '공유', '##합니다', '.', '[SEP]'])
-[2, 18429, 41, 6240, 15229, 6204, 20894, 5689, 12622, 10690, 18, 3]
-```
-## Example using ElectraForPreTraining
-```python
-import torch
-from transformers import ElectraForPreTraining, ElectraTokenizer
-discriminator = ElectraForPreTraining.from_pretrained("monologg/koelectra-small-discriminator")
-tokenizer = ElectraTokenizer.from_pretrained("monologg/koelectra-small-discriminator")
-sentence = "나는 방금 밥을 먹었다."
-fake_sentence = "나는 내일 밥을 먹었다."
-fake_tokens = tokenizer.tokenize(fake_sentence)
-fake_inputs = tokenizer.encode(fake_sentence, return_tensors="pt")
-discriminator_outputs = discriminator(fake_inputs)
-predictions = torch.round((torch.sign(discriminator_outputs[0]) + 1) / 2)
-print(list(zip(fake_tokens, predictions.tolist()[1:-1])))
-```
--- a/model_cards/monologg/koelectra-small-generator/README.md
+++ b/model_cards/monologg/koelectra-small-generator/README.md
---
-language: ko
---
-# KoELECTRA (Small Generator)
-Pretrained ELECTRA Language Model for Korean (`koelectra-small-generator`)
-For more detail, please see [original repository](https://github.com/monologg/KoELECTRA/blob/master/README_EN.md).
-## Usage
-### Load model and tokenizer
-```python
->>> from transformers import ElectraModel, ElectraTokenizer
->>> model = ElectraModel.from_pretrained("monologg/koelectra-small-generator")
->>> tokenizer = ElectraTokenizer.from_pretrained("monologg/koelectra-small-generator")
-```
-### Tokenizer example
-```python
->>> from transformers import ElectraTokenizer
->>> tokenizer = ElectraTokenizer.from_pretrained("monologg/koelectra-small-generator")
->>> tokenizer.tokenize("[CLS] 한국어 ELECTRA를 공유합니다. [SEP]")
-['[CLS]', '한국어', 'E', '##L', '##EC', '##T', '##RA', '##를', '공유', '##합니다', '.', '[SEP]']
->>> tokenizer.convert_tokens_to_ids(['[CLS]', '한국어', 'E', '##L', '##EC', '##T', '##RA', '##를', '공유', '##합니다', '.', '[SEP]'])
-[2, 18429, 41, 6240, 15229, 6204, 20894, 5689, 12622, 10690, 18, 3]
-```
-## Example using ElectraForMaskedLM
-```python
-from transformers import pipeline
-fill_mask = pipeline(
-    "fill-mask",
-    model="monologg/koelectra-small-generator",
-    tokenizer="monologg/koelectra-small-generator"
-)
-print(fill_mask("나는 {} 밥을 먹었다.".format(fill_mask.tokenizer.mask_token)))
-```
--- a/model_cards/monsoon-nlp/dv-wave/README.md
+++ b/model_cards/monsoon-nlp/dv-wave/README.md
---
-language: dv
---
-# dv-wave
-This is a second attempt at a Dhivehi language model trained with
-Google Research's [ELECTRA](https://github.com/google-research/electra).
-Tokenization and pre-training CoLab: https://colab.research.google.com/drive/1ZJ3tU9MwyWj6UtQ-8G7QJKTn-hG1uQ9v?usp=sharing
-Using SimpleTransformers to classify news https://colab.research.google.com/drive/1KnyQxRNWG_yVwms_x9MUAqFQVeMecTV7?usp=sharing
-V1: similar performance to mBERT on news classification task after finetuning for 3 epochs (52%)
-V2: fixed tokenizers ```do_lower_case=False``` and ```strip_accents=False``` to preserve vowel signs of Dhivehi
-  dv-wave: 89% to mBERT: 52%
-## Corpus
-Trained on @Sofwath's 307MB corpus of Dhivehi text: https://github.com/Sofwath/DhivehiDatasets - this repo also contains the news classification task CSV
-[OSCAR](https://oscar-corpus.com/) was considered but has not been added to pretraining; as of
-this writing their web crawl has 126MB of Dhivehi text (79MB deduped).
-## Vocabulary
-Included as vocab.txt in the upload - vocab_size is 29874
--- a/model_cards/moumeneb1/flaubert-base-cased-ecology_crisis/README.md
+++ b/model_cards/moumeneb1/flaubert-base-cased-ecology_crisis/README.md
-# Flaubert-base-cased-ecology_crisis
-An adapted [__Flaubert/Flaubert_base-cased model__](https://github.com/getalp/Flaubert) Trained further on a Language modeling Task of unlabeled French tweets used to create the [CrisisDataset](https://github.com/DiegoKoz/french_ecological_crisis), The intermediate task of masqued language modeling helped us improve the results on our [paper](http://www.sciencedirect.com/science/article/pii/S0306457320300650) compared to the standard flaubert-base-cased model.
-If you use this pretrained model on your work, please cite us as follows 🤗
-```
-@article{Kozlowski-et-al2020,
-title = "A three-level classification of French tweets in ecological crises",
-journal = "Information Processing & Management",
-volume = "57",
-number = "5",
-pages = "102284",
-year = "2020",
-issn = "0306-4573",
-doi = "https://doi.org/10.1016/j.ipm.2020.102284",
-url = "http://www.sciencedirect.com/science/article/pii/S0306457320300650",
-author = "Diego Kozlowski and Elisa Lannelongue and Frédéric Saudemont and Farah Benamara and Alda Mari and Véronique Moriceau and Abdelmoumene Boumadane",
-keywords = "Crisis response from social media, Machine learning, Natural language processing, Transfer learning",
-}
-```
--- a/model_cards/mrm8488/CodeBERTaPy/README.md
+++ b/model_cards/mrm8488/CodeBERTaPy/README.md
---
-language: code
-thumbnail:
---
-# CodeBERTaPy
-CodeBERTaPy is a RoBERTa-like model trained on the [CodeSearchNet](https://github.blog/2019-09-26-introducing-the-codesearchnet-challenge/) dataset from GitHub for `python` by [Manuel Romero](https://twitter.com/mrm8488)
-The **tokenizer** is a Byte-level BPE tokenizer trained on the corpus using Hugging Face `tokenizers`.
-Because it is trained on a corpus of code (vs. natural language), it encodes the corpus efficiently (the sequences are between 33% to 50% shorter, compared to the same corpus tokenized by gpt2/roberta).
-The (small) **model** is a 6-layer, 84M parameters, RoBERTa-like Transformer model – that’s the same number of layers & heads as DistilBERT – initialized from the default initialization settings and trained from scratch on the full `python` corpus for 4 epochs.
-## Quick start: masked language modeling prediction
-```python
-PYTHON_CODE = """
-fruits = ['apples', 'bananas', 'oranges']
-for idx, <mask> in enumerate(fruits):
-  print("index is %d and value is %s" % (idx, val))
-""".lstrip()
-```
-### Does the model know how to complete simple Python code?
-```python
-from transformers import pipeline
-fill_mask = pipeline(
-    "fill-mask",
-    model="mrm8488/CodeBERTaPy",
-    tokenizer="mrm8488/CodeBERTaPy"
-)
-fill_mask(PYTHON_CODE)
-## Top 5 predictions:
-'val' # prob  0.980728805065155
-'value'
-'idx'
-',val'
-'_'
-```
-### Yes! That was easy 🎉 Let's try with another Flask like example
-```python
-PYTHON_CODE2 = """
-@app.route('/<name>')
-def hello_name(name):
-    return "Hello {}!".format(<mask>)
-if __name__ == '__main__':
-    app.run()
-""".lstrip()
-fill_mask(PYTHON_CODE2)
-## Top 5 predictions:
-'name' # prob  0.9961813688278198
-' name'
-'url'
-'description'
-'self'
-```
-### Yeah! It works 🎉 Let's try with another Tensorflow/Keras like example
-```python
-PYTHON_CODE3="""
-model = keras.Sequential([
-    keras.layers.Flatten(input_shape=(28, 28)),
-    keras.layers.<mask>(128, activation='relu'),
-    keras.layers.Dense(10, activation='softmax')
-])
-""".lstrip()
-fill_mask(PYTHON_CODE3)
-## Top 5 predictions:
-'Dense' # prob   0.4482928514480591
-'relu'
-'Flatten'
-'Activation'
-'Conv'
-```
-> Great! 🎉
-## This work is heavily inspired on [CodeBERTa](https://github.com/huggingface/transformers/blob/master/model_cards/huggingface/CodeBERTa-small-v1/README.md) by huggingface team
-<br>
-## CodeSearchNet citation
-<details>
-```bibtex
-@article{husain_codesearchnet_2019,
-	title = {{CodeSearchNet} {Challenge}: {Evaluating} the {State} of {Semantic} {Code} {Search}},
-	shorttitle = {{CodeSearchNet} {Challenge}},
-	url = {http://arxiv.org/abs/1909.09436},
-	urldate = {2020-03-12},
-	journal = {arXiv:1909.09436 [cs, stat]},
-	author = {Husain, Hamel and Wu, Ho-Hsiang and Gazit, Tiferet and Allamanis, Miltiadis and Brockschmidt, Marc},
-	month = sep,
-	year = {2019},
-	note = {arXiv: 1909.09436},
-}
-```
-</details>
-> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488)
-> Made with <span style="color: #e25555;">&hearts;</span> in Spain
--- a/model_cards/mrm8488/GPT-2-finetuned-CORD19/README.md
+++ b/model_cards/mrm8488/GPT-2-finetuned-CORD19/README.md
---
-language: en
-thumbnail:
---
-# GPT-2 + CORD19 dataset : 🦠 ✍ ⚕
-**GPT-2** fine-tuned on **biorxiv_medrxiv**, **comm_use_subset** and **custom_license** files from [CORD-19](https://www.kaggle.com/allen-institute-for-ai/CORD-19-research-challenge) dataset.
-## Datasets details
-| Dataset                | # Files |
-| ---------------------- | ----- |
-| biorxiv_medrxiv        | 885  |
-| comm_use_subset         | 9K   |
-| custom_license         | 20.6K   |
-## Model training
-The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command:
-```bash
-export TRAIN_FILE=/path/to/dataset/train.txt
-python run_language_modeling.py \
-    --model_type gpt2 \
-    --model_name_or_path gpt2 \
-    --do_train \
-    --train_data_file $TRAIN_FILE \
-    --num_train_epochs 4 \
-    --output_dir model_output \
-    --overwrite_output_dir \
-    --save_steps 10000 \
-    --per_gpu_train_batch_size 3
-```
-<img alt="training loss" src="https://svgshare.com/i/JTf.svg' title='GTP-2-finetuned-CORDS19-loss" width="600" height="300" />
-## Model in action / Example of usage ✒
-You can get the following script [here](https://github.com/huggingface/transformers/blob/master/examples/text-generation/run_generation.py)
-```bash
-python run_generation.py \
-    --model_type gpt2 \
-    --model_name_or_path mrm8488/GPT-2-finetuned-CORD19 \
-    --length 200
-```
-```txt
-# Input: the effects of COVID-19 on the lungs
-# Output: === GENERATED SEQUENCE 1 ===
-the effects of COVID-19 on the lungs are currently debated (86). The role of this virus in the pathogenesis of pneumonia and lung cancer is still debated. MERS-CoV is also known to cause acute respiratory distress syndrome (87) and is associated with increased expression of pulmonary fibrosis markers (88). Thus, early airway inflammation may play an important role in the pathogenesis of coronavirus pneumonia and may contribute to the severe disease and/or mortality observed in coronavirus patients.
-Pneumonia is an acute, often fatal disease characterized by severe edema, leakage of oxygen and bronchiolar inflammation. Viruses include coronaviruses, and the role of oxygen depletion is complicated by lung injury and fibrosis in the lung, in addition to susceptibility to other lung diseases. The progression of the disease may be variable, depending on the lung injury, pathologic role, prognosis, and the immune status of the patient. Inflammatory responses to respiratory viruses cause various pathologies of the respiratory
-```
-> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/)
-> Made with <span style="color: #e25555;">&hearts;</span> in Spain
--- a/model_cards/mrm8488/GPT-2-finetuned-common_gen/README.md
+++ b/model_cards/mrm8488/GPT-2-finetuned-common_gen/README.md
---
-language: en
-datasets:
- common_gen
-widget:
- text: "<|endoftext|> apple, tree, pick:"
---
-# GPT-2 fine-tuned on CommonGen
-[GPT-2](https://huggingface.co/gpt2) fine-tuned on [CommonGen](https://inklab.usc.edu/CommonGen/index.html) for *Generative Commonsense Reasoning*.
-## Details of GPT-2
-GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This
-means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots
-of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely,
-it was trained to guess the next word in sentences.
-More precisely, inputs are sequences of continuous text of a certain length and the targets are the same sequence,
-shifted one token (word or piece of word) to the right. The model uses internally a mask-mechanism to make sure the
-predictions for the token `i` only uses the inputs from `1` to `i` but not the future tokens.
-This way, the model learns an inner representation of the English language that can then be used to extract features
-useful for downstream tasks. The model is best at what it was pretrained for however, which is generating texts from a
-prompt.
-## Details of the dataset 📚 
-CommonGen is a constrained text generation task, associated with a benchmark dataset, to explicitly test machines for the ability of generative commonsense reasoning. Given a set of common concepts; the task is to generate a coherent sentence describing an everyday scenario using these concepts.
-CommonGen is challenging because it inherently requires 1) relational reasoning using background commonsense knowledge, and 2) compositional generalization ability to work on unseen concept combinations. Our dataset, constructed through a combination of crowd-sourcing from AMT and existing caption corpora, consists of 30k concept-sets and 50k sentences in total.
-| Dataset  | Split | # samples |
-| -------- | ----- | --------- |
-| common_gen | train  | 67389   |
-| common_gen | valid  | 4018    |
-| common_gen | test   | 1497    |
-## Model fine-tuning 🏋️‍
-You can find the fine-tuning script [here](https://github.com/huggingface/transformers/tree/master/examples/language-modeling)
-## Model in Action 🚀
-```bash
-python ./transformers/examples/text-generation/run_generation.py \
-    --model_type=gpt2 \
-    --model_name_or_path="mrm8488/GPT-2-finetuned-common_gen" \
-    --num_return_sequences 1 \
-    --prompt "<|endoftext|> kid, room, dance:" \
-    --stop_token "."
-```
-> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/)
-> Made with <span style="color: #e25555;">&hearts;</span> in Spain
--- a/model_cards/mrm8488/GPT-2-finetuned-covid-bio-medrxiv/README.md
+++ b/model_cards/mrm8488/GPT-2-finetuned-covid-bio-medrxiv/README.md
---
-language: en
-thumbnail:
---
-# GPT-2 + bio/medrxiv files from CORD19: 🦠 ✍ ⚕
-**GPT-2** fine-tuned on **biorxiv_medrxiv** files from [CORD-19](https://www.kaggle.com/allen-institute-for-ai/CORD-19-research-challenge) dataset.
-## Datasets details:
-| Dataset                | # Files |
-| ---------------------- | ----- |
-| biorxiv_medrxiv        | 885  |
-## Model training:
-The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command:
-```bash
-export TRAIN_FILE=/path/to/dataset/train.txt
-python run_language_modeling.py \
-    --model_type gpt2 \
-    --model_name_or_path gpt2 \
-    --do_train \
-    --train_data_file $TRAIN_FILE \
-    --num_train_epochs 4 \
-    --output_dir model_output \
-    --overwrite_output_dir \
-    --save_steps 2000 \
-    --per_gpu_train_batch_size 3
-```
-## Model in action / Example of usage: ✒
-You can get the following script [here](https://github.com/huggingface/transformers/blob/master/examples/text-generation/run_generation.py)
-```bash
-python run_generation.py \
-    --model_type gpt2 \
-    --model_name_or_path mrm8488/GPT-2-finetuned-CORD19 \
-    --length 200
-```
-```txt
-👵👴🦠
-# Input: Old people with COVID-19 tends to suffer 
-# Output: === GENERATED SEQUENCE 1 ===
-Old people with COVID-19 tends to suffer more symptom onset time and death. It is well known that many people with COVID-19 have high homozygous ZIKV infection in the face of severe symptoms in both severe and severe cases.
-The origin of Wuhan Fever was investigated by Prof. Shen Jiang at the outbreak of Wuhan Fever [34]. As Huanan Province is the epicenter of this outbreak, Huanan, the epicenter of epidemic Wuhan Fever, is the most potential location for the direct transmission of infection (source: Zhongzhen et al., 2020). A negative risk ratio indicates more frequent underlying signs in the people in Huanan Province with COVID-19 patients. Further analysis of reported Huanan Fever onset data in the past two years indicated that the intensity of exposure is the key risk factor for developing MERS-CoV infection in this region, especially among children and elderly. To be continued to develop infected patients would be a very important area for
-```
-![Model in action](https://media.giphy.com/media/TgUdO72Iwk9h7hhm7G/giphy.gif)
-> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/)
-> Made with <span style="color: #e25555;">&hearts;</span> in Spain
--- a/model_cards/mrm8488/GuaPeTe-2-tiny/README.md
+++ b/model_cards/mrm8488/GuaPeTe-2-tiny/README.md
---
-language: es
-widget:
- text: "Murcia es la huerta de Europa porque"
---
-#GuaPeTe-2-tiny: A proof of concept tiny GPT-2 like model trained on Spanish Wikipedia corpus
--- a/model_cards/mrm8488/RoBERTinha/README.md
+++ b/model_cards/mrm8488/RoBERTinha/README.md
---
-language: gl
-widget:
- text: "Galicia é unha <mask> autónoma española."
- text: "A lingua oficial de Galicia é o <mask>."
---
-# RoBERTinha: RoBERTa-like Language model trained on OSCAR Galician corpus