Unverified Commit b732e7e1 authored by Stas Bekman's avatar Stas Bekman Committed by GitHub
Browse files

[doc] multiple corrections to "Summary of the tasks" (#6509)

* [doc] multiple corrections to "Summary of the tasks"

* fix indentation

* correction

* fix links, add links to examples/seq2seq/README.md instead of non-existing script
parent 2a77813d
...@@ -15,7 +15,7 @@ checkpoints are usually pre-trained on a large corpus of data and fine-tuned on ...@@ -15,7 +15,7 @@ checkpoints are usually pre-trained on a large corpus of data and fine-tuned on
following: following:
- Not all models were fine-tuned on all tasks. If you want to fine-tune a model on a specific task, you can leverage - Not all models were fine-tuned on all tasks. If you want to fine-tune a model on a specific task, you can leverage
one of the `run_$TASK.py` script in the one of the `run_$TASK.py` scripts in the
`examples <https://github.com/huggingface/transformers/tree/master/examples>`_ directory. `examples <https://github.com/huggingface/transformers/tree/master/examples>`_ directory.
- Fine-tuned models were fine-tuned on a specific dataset. This dataset may or may not overlap with your use-case - Fine-tuned models were fine-tuned on a specific dataset. This dataset may or may not overlap with your use-case
and domain. As mentioned previously, you may leverage the and domain. As mentioned previously, you may leverage the
...@@ -25,8 +25,7 @@ following: ...@@ -25,8 +25,7 @@ following:
In order to do an inference on a task, several mechanisms are made available by the library: In order to do an inference on a task, several mechanisms are made available by the library:
- Pipelines: very easy-to-use abstractions, which require as little as two lines of code. - Pipelines: very easy-to-use abstractions, which require as little as two lines of code.
- Using a model directly with a tokenizer (PyTorch/TensorFlow): the full inference using the model. Less abstraction, - Direct model use: Less abstractions, but more flexibility and power via a direct access to a tokenizer (PyTorch/TensorFlow) and full inference capacity.
but much more powerful.
Both approaches are showcased here. Both approaches are showcased here.
...@@ -44,10 +43,11 @@ Sequence Classification ...@@ -44,10 +43,11 @@ Sequence Classification
Sequence classification is the task of classifying sequences according to a given number of classes. An example Sequence classification is the task of classifying sequences according to a given number of classes. An example
of sequence classification is the GLUE dataset, which is entirely based on that task. If you would like to fine-tune of sequence classification is the GLUE dataset, which is entirely based on that task. If you would like to fine-tune
a model on a GLUE sequence classification task, you may leverage the a model on a GLUE sequence classification task, you may leverage the
`run_glue.py <https://github.com/huggingface/transformers/tree/master/examples/text-classification/run_glue.py>`_ or `run_glue.py <https://github.com/huggingface/transformers/tree/master/examples/text-classification/run_glue.py>`_ and
`run_pl_glue.py <https://github.com/huggingface/transformers/tree/master/examples/text-classification/run_pl_glue.py>`_ or
`run_tf_glue.py <https://github.com/huggingface/transformers/tree/master/examples/text-classification/run_tf_glue.py>`_ scripts. `run_tf_glue.py <https://github.com/huggingface/transformers/tree/master/examples/text-classification/run_tf_glue.py>`_ scripts.
Here is an example using the pipelines do to sentiment analysis: identifying if a sequence is positive or negative. Here is an example of using pipelines to do sentiment analysis: identifying if a sequence is positive or negative.
It leverages a fine-tuned model on sst2, which is a GLUE task. It leverages a fine-tuned model on sst2, which is a GLUE task.
This returns a label ("POSITIVE" or "NEGATIVE") alongside a score, as follows: This returns a label ("POSITIVE" or "NEGATIVE") alongside a score, as follows:
...@@ -70,15 +70,17 @@ This returns a label ("POSITIVE" or "NEGATIVE") alongside a score, as follows: ...@@ -70,15 +70,17 @@ This returns a label ("POSITIVE" or "NEGATIVE") alongside a score, as follows:
Here is an example of doing a sequence classification using a model to determine if two sequences are paraphrases Here is an example of doing a sequence classification using a model to determine if two sequences are paraphrases
of each other. The process is the following: of each other. The process is the following:
- Instantiate a tokenizer and a model from the checkpoint name. The model is identified as a BERT model and loads it 1. Instantiate a tokenizer and a model from the checkpoint name. The model is
with the weights stored in the checkpoint. identified as a BERT model and loads it with the weights stored in the
- Build a sequence from the two sentences, with the correct model-specific separators token type ids checkpoint.
and attention masks (:func:`~transformers.PreTrainedTokenizer.encode` and 2. Build a sequence from the two sentences, with the correct model-specific
:func:`~transformers.PreTrainedTokenizer.__call__` take care of this) separators token type ids and attention masks
- Pass this sequence through the model so that it is classified in one of the two available classes: 0 (:func:`~transformers.PreTrainedTokenizer.encode` and
(not a paraphrase) and 1 (is a paraphrase) :func:`~transformers.PreTrainedTokenizer.__call__` take care of this).
- Compute the softmax of the result to get probabilities over the classes 3. Pass this sequence through the model so that it is classified in one of the
- Print the results two available classes: 0 (not a paraphrase) and 1 (is a paraphrase).
4. Compute the softmax of the result to get probabilities over the classes.
5. Print the results.
.. code-block:: .. code-block::
...@@ -154,9 +156,12 @@ Extractive Question Answering ...@@ -154,9 +156,12 @@ Extractive Question Answering
Extractive Question Answering is the task of extracting an answer from a text given a question. An example of a Extractive Question Answering is the task of extracting an answer from a text given a question. An example of a
question answering dataset is the SQuAD dataset, which is entirely based on that task. If you would like to fine-tune question answering dataset is the SQuAD dataset, which is entirely based on that task. If you would like to fine-tune
a model on a SQuAD task, you may leverage the `run_squad.py`. a model on a SQuAD task, you may leverage the
`run_squad.py <https://github.com/huggingface/transformers/tree/master/examples/question-answering/run_squad.py>`_ and
`run_tf_squad.py <https://github.com/huggingface/transformers/tree/master/examples/question-answering/run_tf_squad.py>`_ scripts.
Here is an example using the pipelines do to question answering: extracting an answer from a text given a question.
Here is an example of using pipelines to do question answering: extracting an answer from a text given a question.
It leverages a fine-tuned model on SQuAD. It leverages a fine-tuned model on SQuAD.
.. code-block:: .. code-block::
...@@ -171,7 +176,7 @@ It leverages a fine-tuned model on SQuAD. ...@@ -171,7 +176,7 @@ It leverages a fine-tuned model on SQuAD.
... a model on a SQuAD task, you may leverage the examples/question-answering/run_squad.py script. ... a model on a SQuAD task, you may leverage the examples/question-answering/run_squad.py script.
... """ ... """
This returns an answer extracted from the text, a confidence score, alongside "start" and "end" values which This returns an answer extracted from the text, a confidence score, alongside "start" and "end" values, which
are the positions of the extracted answer in the text. are the positions of the extracted answer in the text.
.. code-block:: .. code-block::
...@@ -187,16 +192,19 @@ are the positions of the extracted answer in the text. ...@@ -187,16 +192,19 @@ are the positions of the extracted answer in the text.
Here is an example of question answering using a model and a tokenizer. The process is the following: Here is an example of question answering using a model and a tokenizer. The process is the following:
- Instantiate a tokenizer and a model from the checkpoint name. The model is identified as a BERT model and loads it 1. Instantiate a tokenizer and a model from the checkpoint name. The model is
with the weights stored in the checkpoint. identified as a BERT model and loads it with the weights stored in the
- Define a text and a few questions. checkpoint.
- Iterate over the questions and build a sequence from the text and the current question, with the correct 2. Define a text and a few questions.
model-specific separators token type ids and attention masks 3. Iterate over the questions and build a sequence from the text and the current
- Pass this sequence through the model. This outputs a range of scores across the entire sequence tokens (question and question, with the correct model-specific separators token type ids and
text), for both the start and end positions. attention masks.
- Compute the softmax of the result to get probabilities over the tokens 4. Pass this sequence through the model. This outputs a range of scores across
- Fetch the tokens from the identified start and stop values, convert those tokens to a string. the entire sequence tokens (question and text), for both the start and end
- Print the results positions.
5. Compute the softmax of the result to get probabilities over the tokens.
6. Fetch the tokens from the identified start and stop values, convert those tokens to a string.
7. Print the results.
.. code-block:: .. code-block::
...@@ -291,8 +299,8 @@ Here is an example of question answering using a model and a tokenizer. The proc ...@@ -291,8 +299,8 @@ Here is an example of question answering using a model and a tokenizer. The proc
Language Modeling Language Modeling
---------------------------------------------------- ----------------------------------------------------
Language modeling is the task of fitting a model to a corpus, which can be domain specific. All popular transformer Language modeling is the task of fitting a model to a corpus, which can be domain specific. All popular transformer-based
based models are trained using a variant of language modeling, e.g. BERT with masked language modeling, GPT-2 with models are trained using a variant of language modeling, e.g. BERT with masked language modeling, GPT-2 with
causal language modeling. causal language modeling.
Language modeling can be useful outside of pre-training as well, for example to shift the model distribution to be Language modeling can be useful outside of pre-training as well, for example to shift the model distribution to be
...@@ -305,7 +313,7 @@ Masked Language Modeling ...@@ -305,7 +313,7 @@ Masked Language Modeling
Masked language modeling is the task of masking tokens in a sequence with a masking token, and prompting the model to Masked language modeling is the task of masking tokens in a sequence with a masking token, and prompting the model to
fill that mask with an appropriate token. This allows the model to attend to both the right context (tokens on the fill that mask with an appropriate token. This allows the model to attend to both the right context (tokens on the
right of the mask) and the left context (tokens on the left of the mask). Such a training creates a strong basis right of the mask) and the left context (tokens on the left of the mask). Such a training creates a strong basis
for downstream tasks requiring bi-directional context such as SQuAD (question answering, for downstream tasks, requiring bi-directional context such as SQuAD (question answering,
see `Lewis, Lui, Goyal et al. <https://arxiv.org/abs/1910.13461>`__, part 4.2). see `Lewis, Lui, Goyal et al. <https://arxiv.org/abs/1910.13461>`__, part 4.2).
Here is an example of using pipelines to replace a mask from a sequence: Here is an example of using pipelines to replace a mask from a sequence:
...@@ -316,7 +324,7 @@ Here is an example of using pipelines to replace a mask from a sequence: ...@@ -316,7 +324,7 @@ Here is an example of using pipelines to replace a mask from a sequence:
>>> nlp = pipeline("fill-mask") >>> nlp = pipeline("fill-mask")
This outputs the sequences with the mask filled, the confidence score as well as the token id in the tokenizer This outputs the sequences with the mask filled, the confidence score, and the token id in the tokenizer
vocabulary: vocabulary:
.. code-block:: .. code-block::
...@@ -349,17 +357,19 @@ vocabulary: ...@@ -349,17 +357,19 @@ vocabulary:
'token': 17715, 'token': 17715,
'token_str': 'Ġprototype'}] 'token_str': 'Ġprototype'}]
Here is an example doing masked language modeling using a model and a tokenizer. The process is the following: Here is an example of doing masked language modeling using a model and a tokenizer. The process is the following:
- Instantiate a tokenizer and a model from the checkpoint name. The model is identified as a DistilBERT model and 1. Instantiate a tokenizer and a model from the checkpoint name. The model is
loads it with the weights stored in the checkpoint. identified as a DistilBERT model and loads it with the weights stored in the
- Define a sequence with a masked token, placing the :obj:`tokenizer.mask_token` instead of a word. checkpoint.
- Encode that sequence into IDs and find the position of the masked token in that list of IDs. 2. Define a sequence with a masked token, placing the :obj:`tokenizer.mask_token` instead of a word.
- Retrieve the predictions at the index of the mask token: this tensor has the same size as the vocabulary, and the 3. Encode that sequence into a list of IDs and find the position of the masked token in that list.
values are the scores attributed to each token. The model gives higher score to tokens he deems probable in that 4. Retrieve the predictions at the index of the mask token: this tensor has the
context. same size as the vocabulary, and the values are the scores attributed to each
- Retrieve the top 5 tokens using the PyTorch :obj:`topk` or TensorFlow :obj:`top_k` methods. token. The model gives higher score to tokens it deems probable in that
- Replace the mask token by the tokens and print the results context.
5. Retrieve the top 5 tokens using the PyTorch :obj:`topk` or TensorFlow :obj:`top_k` methods.
6. Replace the mask token by the tokens and print the results
.. code-block:: .. code-block::
...@@ -419,7 +429,7 @@ for generation tasks. ...@@ -419,7 +429,7 @@ for generation tasks.
Usually, the next token is predicted by sampling from the logits of the last hidden state the model produces from the input sequence. Usually, the next token is predicted by sampling from the logits of the last hidden state the model produces from the input sequence.
Here is an example using the tokenizer and model and leveraging the :func:`~transformers.PreTrainedModel.top_k_top_p_filtering` method to sample the next token following an input sequence of tokens. Here is an example of using the tokenizer and model and leveraging the :func:`~transformers.PreTrainedModel.top_k_top_p_filtering` method to sample the next token following an input sequence of tokens.
.. code-block:: .. code-block::
...@@ -473,7 +483,7 @@ Here is an example using the tokenizer and model and leveraging the :func:`~tran ...@@ -473,7 +483,7 @@ Here is an example using the tokenizer and model and leveraging the :func:`~tran
>>> resulting_string = tokenizer.decode(generated.numpy().tolist()[0]) >>> resulting_string = tokenizer.decode(generated.numpy().tolist()[0])
This outputs a (hopefully) coherent next token following the original sequence, which is in our case is the word *has*: This outputs a (hopefully) coherent next token following the original sequence, which in our case is the word *has*:
.. code-block:: .. code-block::
...@@ -485,7 +495,7 @@ In the next section, we show how this functionality is leveraged in :func:`~tran ...@@ -485,7 +495,7 @@ In the next section, we show how this functionality is leveraged in :func:`~tran
Text Generation Text Generation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In text generation (*a.k.a* *open-ended text generation*) the goal is to create a coherent portion of text that is a continuation from the given context. As an example, is it shown how *GPT-2* can be used in pipelines to generate text. As a default all models apply *Top-K* sampling when used in pipelines as configured in their respective configurations (see `gpt-2 config <https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-config.json>`_ for example). In text generation (*a.k.a* *open-ended text generation*) the goal is to create a coherent portion of text that is a continuation from the given context. The following example shows how *GPT-2* can be used in pipelines to generate text. As a default all models apply *Top-K* sampling when used in pipelines, as configured in their respective configurations (see `gpt-2 config <https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-config.json>`_ for example).
.. code-block:: .. code-block::
...@@ -497,10 +507,10 @@ In text generation (*a.k.a* *open-ended text generation*) the goal is to create ...@@ -497,10 +507,10 @@ In text generation (*a.k.a* *open-ended text generation*) the goal is to create
Here the model generates a random text with a total maximal length of *50* tokens from context *"As far as I am concerned, I will"*. Here, the model generates a random text with a total maximal length of *50* tokens from context *"As far as I am concerned, I will"*.
The default arguments of ``PreTrainedModel.generate()`` can directly be overriden in the pipeline as is shown above for the argument ``max_length``. The default arguments of ``PreTrainedModel.generate()`` can be directly overriden in the pipeline, as is shown above for the argument ``max_length``.
Here is an example for text generation using XLNet and its tokenzier. Here is an example of text generation using ``XLNet`` and its tokenzier.
.. code-block:: .. code-block::
...@@ -556,24 +566,27 @@ Here is an example for text generation using XLNet and its tokenzier. ...@@ -556,24 +566,27 @@ Here is an example for text generation using XLNet and its tokenzier.
.. code-block:: .. code-block::
print(generated) >>> print(generated)
Today the weather is really nice and I am planning on anning on taking a nice...... of a great time!<eop>...............
Text generation is currently possible with *GPT-2*, *OpenAi-GPT*, *CTRL*, *XLNet*, *Transfo-XL* and *Reformer* in PyTorch and for most models in Tensorflow as well. As can be seen in the example above *XLNet* and *Transfo-xl* often need to be padded to work well. Text generation is currently possible with *GPT-2*, *OpenAi-GPT*, *CTRL*, *XLNet*, *Transfo-XL* and *Reformer* in PyTorch and for most models in Tensorflow as well. As can be seen in the example above *XLNet* and *Transfo-XL* often need to be padded to work well.
GPT-2 is usually a good choice for *open-ended text generation* because it was trained on millions on webpages with a causal language modeling objective. GPT-2 is usually a good choice for *open-ended text generation* because it was trained on millions of webpages with a causal language modeling objective.
For more information on how to apply different decoding strategies for text generation, please also refer to our generation blog post `here <https://huggingface.co/blog/how-to-generate>`_. For more information on how to apply different decoding strategies for text generation, please also refer to our text generation blog post `here <https://huggingface.co/blog/how-to-generate>`_.
Named Entity Recognition Named Entity Recognition
---------------------------------------------------- ----------------------------------------------------
Named Entity Recognition (NER) is the task of classifying tokens according to a class, for example identifying a Named Entity Recognition (NER) is the task of classifying tokens according to a class, for example, identifying a
token as a person, an organisation or a location. token as a person, an organisation or a location.
An example of a named entity recognition dataset is the CoNLL-2003 dataset, which is entirely based on that task. An example of a named entity recognition dataset is the CoNLL-2003 dataset, which is entirely based on that task.
If you would like to fine-tune a model on an NER task, you may leverage the `ner/run_ner.py` (PyTorch), If you would like to fine-tune a model on an NER task, you may leverage the
`ner/run_pl_ner.py` (leveraging pytorch-lightning) or the `ner/run_tf_ner.py` (TensorFlow) scripts. `run_ner.py <https://github.com/huggingface/transformers/tree/master/examples/token-classification/run_ner.py>`_ (PyTorch),
`run_pl_ner.py <https://github.com/huggingface/transformers/tree/master/examples/token-classification/run_pl_ner.py>`_ (leveraging pytorch-lightning) or the
`run_tf_ner.py <https://github.com/huggingface/transformers/tree/master/examples/token-classification/run_tf_ner.py>`_ (TensorFlow) scripts.
Here is an example using the pipelines do to named entity recognition, trying to identify tokens as belonging to one Here is an example of using pipelines to do named entity recognition, specifically, trying to identify tokens as belonging to one
of 9 classes: of 9 classes:
- O, Outside of a named entity - O, Outside of a named entity
...@@ -599,7 +612,7 @@ It leverages a fine-tuned model on CoNLL-2003, fine-tuned by `@stefan-it <https: ...@@ -599,7 +612,7 @@ It leverages a fine-tuned model on CoNLL-2003, fine-tuned by `@stefan-it <https:
... "close to the Manhattan Bridge which is visible from the window." ... "close to the Manhattan Bridge which is visible from the window."
This outputs a list of all words that have been identified as an entity from the 9 classes defined above. Here is the This outputs a list of all words that have been identified as one of the entities from the 9 classes defined above. Here are the
expected results: expected results:
.. code-block:: .. code-block::
...@@ -620,22 +633,25 @@ expected results: ...@@ -620,22 +633,25 @@ expected results:
{'word': 'Bridge', 'score': 0.990249514579773, 'entity': 'I-LOC'} {'word': 'Bridge', 'score': 0.990249514579773, 'entity': 'I-LOC'}
] ]
Note how the words "Hugging Face" have been identified as an organisation, and "New York City", "DUMBO" and Note, how the tokens of the sequence "Hugging Face" have been identified as an organisation, and "New York City", "DUMBO" and
"Manhattan Bridge" have been identified as locations. "Manhattan Bridge" have been identified as locations.
Here is an example doing named entity recognition using a model and a tokenizer. The process is the following: Here is an example of doing named entity recognition, using a model and a tokenizer. The process is the following:
- Instantiate a tokenizer and a model from the checkpoint name. The model is identified as a BERT model and 1. Instantiate a tokenizer and a model from the checkpoint name. The model is
loads it with the weights stored in the checkpoint. identified as a BERT model and loads it with the weights stored in the
- Define the label list with which the model was trained on. checkpoint.
- Define a sequence with known entities, such as "Hugging Face" as an organisation and "New York City" as a location. 2. Define the label list with which the model was trained on.
- Split words into tokens so that they can be mapped to the predictions. We use a small hack by firstly completely 3. Define a sequence with known entities, such as "Hugging Face" as an organisation and "New York City" as a location.
encoding and decoding the sequence, so that we're left with a string that contains the special tokens. 4. Split words into tokens so that they can be mapped to predictions. We use a
- Encode that sequence into IDs (special tokens are added automatically). small hack by, first, completely encoding and decoding the sequence, so that
- Retrieve the predictions by passing the input to the model and getting the first output. This results in a we're left with a string that contains the special tokens.
distribution over the 9 possible classes for each token. We take the argmax to retrieve the most likely class 5. Encode that sequence into IDs (special tokens are added automatically).
for each token. 6. Retrieve the predictions by passing the input to the model and getting the
- Zip together each token with its prediction and print it. first output. This results in a distribution over the 9 possible classes for
each token. We take the argmax to retrieve the most likely class for each
token.
7. Zip together each token with its prediction and print it.
.. code-block:: .. code-block::
...@@ -697,8 +713,8 @@ Here is an example doing named entity recognition using a model and a tokenizer. ...@@ -697,8 +713,8 @@ Here is an example doing named entity recognition using a model and a tokenizer.
>>> predictions = tf.argmax(outputs, axis=2) >>> predictions = tf.argmax(outputs, axis=2)
This outputs a list of each token mapped to their prediction. Differently from the pipeline, here every token has This outputs a list of each token mapped to its corresponding prediction. Differently from the pipeline, here every token has
a prediction as we didn't remove the "0" class which means that no particular entity was found on that token. The a prediction as we didn't remove the "0"th class, which means that no particular entity was found on that token. The
following array should be the output: following array should be the output:
.. code-block:: .. code-block::
...@@ -709,13 +725,13 @@ following array should be the output: ...@@ -709,13 +725,13 @@ following array should be the output:
Summarization Summarization
---------------------------------------------------- ----------------------------------------------------
Summarization is the task of summarizing a text / an article into a shorter text. Summarization is the task of summarizing a document or an article into a shorter text.
An example of a summarization dataset is the CNN / Daily Mail dataset, which consists of long news articles and was created for the task of summarization. An example of a summarization dataset is the CNN / Daily Mail dataset, which consists of long news articles and was created for the task of summarization.
If you would like to fine-tune a model on a summarization task, you may leverage the ``examples/summarization/bart/run_train.sh`` (leveraging pytorch-lightning) script. If you would like to fine-tune a model on a summarization task, various approaches are described in this
`document <https://github.com/huggingface/transformers/blob/master/examples/seq2seq/README.md>`_.
Here is an example using the pipelines do to summarization. Here is an example of using the pipelines to do summarization. It leverages a Bart model that was fine-tuned on the CNN / Daily Mail data set.
It leverages a Bart model that was fine-tuned on the CNN / Daily Mail data set.
.. code-block:: .. code-block::
...@@ -742,8 +758,8 @@ It leverages a Bart model that was fine-tuned on the CNN / Daily Mail data set. ...@@ -742,8 +758,8 @@ It leverages a Bart model that was fine-tuned on the CNN / Daily Mail data set.
... If convicted, Barrientos faces up to four years in prison. Her next court appearance is scheduled for May 18. ... If convicted, Barrientos faces up to four years in prison. Her next court appearance is scheduled for May 18.
... """ ... """
Because the summarization pipeline depends on the ``PretrainedModel.generate()`` method, we can override the default arguments Because the summarization pipeline depends on the ``PretrainedModel.generate()`` method, we can override the default arguments
of ``PretrainedModel.generate()`` directly in the pipeline as is shown for ``max_length`` and ``min_length`` above. of ``PretrainedModel.generate()`` directly in the pipeline for ``max_length`` and ``min_length`` as shown below.
This outputs the following summary: This outputs the following summary:
.. code-block:: .. code-block::
...@@ -751,14 +767,15 @@ This outputs the following summary: ...@@ -751,14 +767,15 @@ This outputs the following summary:
>>> print(summarizer(ARTICLE, max_length=130, min_length=30, do_sample=False)) >>> print(summarizer(ARTICLE, max_length=130, min_length=30, do_sample=False))
[{'summary_text': 'Liana Barrientos, 39, is charged with two counts of "offering a false instrument for filing in the first degree" In total, she has been married 10 times, with nine of her marriages occurring between 1999 and 2002. She is believed to still be married to four men.'}] [{'summary_text': 'Liana Barrientos, 39, is charged with two counts of "offering a false instrument for filing in the first degree" In total, she has been married 10 times, with nine of her marriages occurring between 1999 and 2002. She is believed to still be married to four men.'}]
Here is an example doing summarization using a model and a tokenizer. The process is the following: Here is an example of doing summarization using a model and a tokenizer. The process is the following:
- Instantiate a tokenizer and a model from the checkpoint name. Summarization is usually done using an encoder-decoder model, such as ``Bart`` or ``T5``. 1. Instantiate a tokenizer and a model from the checkpoint name. Summarization is usually done using an encoder-decoder model, such as ``Bart`` or ``T5``.
- Define the article that should be summarizaed. 2. Define the article that should be summarized.
- Leverage the ``PretrainedModel.generate()`` method. 3. Add the T5 specific prefix "summarize: ".
- Add the T5 specific prefix "summarize: ". 4. Use the ``PretrainedModel.generate()`` method to generate the summary.
In this example we use Google`s T5 model. Even though it was pre-trained only on a multi-task mixed dataset (including CNN / Daily Mail), it yields very good results.
Here Google`s T5 model is used that was only pre-trained on a multi-task mixed data set (including CNN / Daily Mail), but nevertheless yields very good results.
.. code-block:: .. code-block::
>>> ## PYTORCH CODE >>> ## PYTORCH CODE
...@@ -785,12 +802,14 @@ Translation ...@@ -785,12 +802,14 @@ Translation
Translation is the task of translating a text from one language to another. Translation is the task of translating a text from one language to another.
An example of a translation dataset is the WMT English to German dataset, which has English sentences as the input data An example of a translation dataset is the WMT English to German dataset, which has sentences in English as the input data
and German sentences as the target data. and the corresponding sentences in German as the target data.
If you would like to fine-tune a model on a translation task, various approaches are described in this
`document <https://github.com/huggingface/transformers/blob/master/examples/seq2seq/README.md>`_.
Here is an example using the pipelines do to translation. Here is an example of using the pipelines to do translation.
It leverages a T5 model that was only pre-trained on a multi-task mixture dataset (including WMT), but yields impressive It leverages a T5 model that was only pre-trained on a multi-task mixture dataset (including WMT), yet, yielding impressive
translation results nevertheless. translation results.
.. code-block:: .. code-block::
...@@ -800,15 +819,15 @@ translation results nevertheless. ...@@ -800,15 +819,15 @@ translation results nevertheless.
>>> print(translator("Hugging Face is a technology company based in New York and Paris", max_length=40)) >>> print(translator("Hugging Face is a technology company based in New York and Paris", max_length=40))
[{'translation_text': 'Hugging Face ist ein Technologieunternehmen mit Sitz in New York und Paris.'}] [{'translation_text': 'Hugging Face ist ein Technologieunternehmen mit Sitz in New York und Paris.'}]
Because the translation pipeline depends on the ``PretrainedModel.generate()`` method, we can override the default arguments Because the translation pipeline depends on the ``PretrainedModel.generate()`` method, we can override the default arguments
of ``PretrainedModel.generate()`` directly in the pipeline as is shown for ``max_length`` above. of ``PretrainedModel.generate()`` directly in the pipeline as is shown for ``max_length`` above.
Here is an example doing translation using a model and a tokenizer. The process is the following:
- Instantiate a tokenizer and a model from the checkpoint name. Summarization is usually done using an encoder-decoder model, such as ``Bart`` or ``T5``. Here is an example of doing translation using a model and a tokenizer. The process is the following:
- Define the article that should be summarizaed.
- Leverage the ``PretrainedModel.generate()`` method. 1. Instantiate a tokenizer and a model from the checkpoint name. Summarization is usually done using an encoder-decoder model, such as ``Bart`` or ``T5``.
- Add the T5 specific prefix "translate English to German: " 2. Define the article that should be summarizaed.
3. Add the T5 specific prefix "translate English to German: "
4. Use the ``PretrainedModel.generate()`` method to perform the translation.
.. code-block:: .. code-block::
...@@ -820,10 +839,6 @@ Here is an example doing translation using a model and a tokenizer. The process ...@@ -820,10 +839,6 @@ Here is an example doing translation using a model and a tokenizer. The process
>>> inputs = tokenizer.encode("translate English to German: Hugging Face is a technology company based in New York and Paris", return_tensors="pt") >>> inputs = tokenizer.encode("translate English to German: Hugging Face is a technology company based in New York and Paris", return_tensors="pt")
>>> outputs = model.generate(inputs, max_length=40, num_beams=4, early_stopping=True) >>> outputs = model.generate(inputs, max_length=40, num_beams=4, early_stopping=True)
>>> print(outputs)
tensor([[ 0, 11560, 3896, 8881, 229, 236, 3, 14366, 15377, 181,
11216, 16, 368, 1060, 64, 1919, 5]])
>>> ## TENSORFLOW CODE >>> ## TENSORFLOW CODE
>>> from transformers import TFAutoModelWithLMHead, AutoTokenizer >>> from transformers import TFAutoModelWithLMHead, AutoTokenizer
...@@ -833,7 +848,9 @@ Here is an example doing translation using a model and a tokenizer. The process ...@@ -833,7 +848,9 @@ Here is an example doing translation using a model and a tokenizer. The process
>>> inputs = tokenizer.encode("translate English to German: Hugging Face is a technology company based in New York and Paris", return_tensors="tf") >>> inputs = tokenizer.encode("translate English to German: Hugging Face is a technology company based in New York and Paris", return_tensors="tf")
>>> outputs = model.generate(inputs, max_length=40, num_beams=4, early_stopping=True) >>> outputs = model.generate(inputs, max_length=40, num_beams=4, early_stopping=True)
>>> print(outputs) As with the pipeline example, we get the same translation:
tf.Tensor(
[[ 0 11560 3896 8881 229 236 3 14366 15377 181 11216 16 .. code-block::
368 1060 64 1919 5]], shape=(1, 17), dtype=int32)
>>> print(tokenizer.decode(outputs[0]))
Hugging Face ist ein Technologieunternehmen mit Sitz in New York und Paris.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment