Unverified Commit aeba4f95 authored by Darigov Research's avatar Darigov Research Committed by GitHub
Browse files

Adds terms to Glossary (#10443)

* feat: Adds three definitions to glossary from @cronoik

Needed a definition for transformer which in turn needed 2 more definitions

To do with issue https://github.com/huggingface/transformers/issues/9078

* fix: Adjusts definition of neural network to make it easier to read
parent 256482ac
...@@ -21,6 +21,7 @@ General terms ...@@ -21,6 +21,7 @@ General terms
- CLM: causal language modeling, a pretraining task where the model reads the texts in order and has to predict the - CLM: causal language modeling, a pretraining task where the model reads the texts in order and has to predict the
next word. It's usually done by reading the whole sentence but using a mask inside the model to hide the future next word. It's usually done by reading the whole sentence but using a mask inside the model to hide the future
tokens at a certain timestep. tokens at a certain timestep.
- deep learning: machine learning algorithms which uses neural networks with several layers.
- MLM: masked language modeling, a pretraining task where the model sees a corrupted version of the texts, usually done - MLM: masked language modeling, a pretraining task where the model sees a corrupted version of the texts, usually done
by masking some tokens randomly, and has to predict the original text. by masking some tokens randomly, and has to predict the original text.
- multimodal: a task that combines texts with another kind of inputs (for instance images). - multimodal: a task that combines texts with another kind of inputs (for instance images).
...@@ -33,10 +34,12 @@ General terms ...@@ -33,10 +34,12 @@ General terms
involve a self-supervised objective, which can be reading the text and trying to predict the next word (see CLM) or involve a self-supervised objective, which can be reading the text and trying to predict the next word (see CLM) or
masking some words and trying to predict them (see MLM). masking some words and trying to predict them (see MLM).
- RNN: recurrent neural network, a type of model that uses a loop over a layer to process texts. - RNN: recurrent neural network, a type of model that uses a loop over a layer to process texts.
- self-attention: each element of the input finds out which other elements of the input they should attend to.
- seq2seq or sequence-to-sequence: models that generate a new sequence from an input, like translation models, or - seq2seq or sequence-to-sequence: models that generate a new sequence from an input, like translation models, or
summarization models (such as :doc:`Bart </model_doc/bart>` or :doc:`T5 </model_doc/t5>`). summarization models (such as :doc:`Bart </model_doc/bart>` or :doc:`T5 </model_doc/t5>`).
- token: a part of a sentence, usually a word, but can also be a subword (non-common words are often split in subwords) - token: a part of a sentence, usually a word, but can also be a subword (non-common words are often split in subwords)
or a punctuation symbol. or a punctuation symbol.
- transformer: self-attention based deep learning model architecture.
Model inputs Model inputs
----------------------------------------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------------------------------------
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment