Unverified Commit 78a53d59 authored by Maria Khalusova's avatar Maria Khalusova Committed by GitHub
Browse files

Adding task guides to resources (#21704)



* added resources: links to task guides that support these models

* minor polishing

* conflict resolved

* link fix

* Update docs/source/en/model_doc/vision-encoder-decoder.mdx
Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>

---------
Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
parent 03aaac35
...@@ -32,6 +32,10 @@ Tips: ...@@ -32,6 +32,10 @@ Tips:
This model was contributed by [Younes Belkada](https://huggingface.co/ybelkada) and [Arthur Zucker](https://huggingface.co/ArtZucker) . This model was contributed by [Younes Belkada](https://huggingface.co/ybelkada) and [Arthur Zucker](https://huggingface.co/ArtZucker) .
The original code can be found [here](https://github.com/google/flaxformer/tree/main/flaxformer/architectures/moe). The original code can be found [here](https://github.com/google/flaxformer/tree/main/flaxformer/architectures/moe).
## Resources
- [Translation task guide](./tasks/translation)
- [Summarization task guide](./tasks/summarization)
## SwitchTransformersConfig ## SwitchTransformersConfig
......
...@@ -333,6 +333,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h ...@@ -333,6 +333,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h
- [`TFT5ForConditionalGeneration`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/tensorflow/summarization) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/summarization-tf.ipynb). - [`TFT5ForConditionalGeneration`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/tensorflow/summarization) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/summarization-tf.ipynb).
- [`FlaxT5ForConditionalGeneration`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/flax/summarization). - [`FlaxT5ForConditionalGeneration`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/flax/summarization).
- [Summarization](https://huggingface.co/course/chapter7/5?fw=pt#summarization) chapter of the 🤗 Hugging Face course. - [Summarization](https://huggingface.co/course/chapter7/5?fw=pt#summarization) chapter of the 🤗 Hugging Face course.
- [Summarization task guide](./tasks/summarization)
<PipelineTag pipeline="fill-mask"/> <PipelineTag pipeline="fill-mask"/>
...@@ -342,6 +343,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h ...@@ -342,6 +343,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h
- [`T5ForConditionalGeneration`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/translation) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/translation.ipynb). - [`T5ForConditionalGeneration`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/translation) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/translation.ipynb).
- [`TFT5ForConditionalGeneration`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/tensorflow/translation) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/translation-tf.ipynb). - [`TFT5ForConditionalGeneration`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/tensorflow/translation) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/translation-tf.ipynb).
- [Translation task guide](./tasks/translation)
<PipelineTag pipeline="question-answering"/> <PipelineTag pipeline="question-answering"/>
......
...@@ -47,7 +47,7 @@ found [here](https://github.com/microsoft/table-transformer). ...@@ -47,7 +47,7 @@ found [here](https://github.com/microsoft/table-transformer).
<PipelineTag pipeline="object-detection"/> <PipelineTag pipeline="object-detection"/>
- A demo notebook for the Table Transformer can be found [here](https://github.com/NielsRogge/Transformers-Tutorials/tree/master/Table%20Transformer). - A demo notebook for the Table Transformer can be found [here](https://github.com/NielsRogge/Transformers-Tutorials/tree/master/Table%20Transformer).
- It turns out padding of images is quite important for detection. An interesting Github thread with replies from the authors can be found [here](https://github.com/microsoft/table-transformer/issues/68). - It turns out padding of images is quite important for detection. An interesting Github thread with replies from the authors can be found [here](https://github.com/microsoft/table-transformer/issues/68).
## TableTransformerConfig ## TableTransformerConfig
......
...@@ -569,6 +569,11 @@ Predicted answer: SUM > 87, 53, 69 ...@@ -569,6 +569,11 @@ Predicted answer: SUM > 87, 53, 69
In case of a conversational set-up, then each table-question pair must be provided **sequentially** to the model, such that the `prev_labels` token types can be overwritten by the predicted `labels` of the previous table-question pair. Again, more info can be found in [this notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/TAPAS/Fine_tuning_TapasForQuestionAnswering_on_SQA.ipynb) (for PyTorch) and [this notebook](https://github.com/kamalkraj/Tapas-Tutorial/blob/master/TAPAS/Fine_tuning_TapasForQuestionAnswering_on_SQA.ipynb) (for TensorFlow). In case of a conversational set-up, then each table-question pair must be provided **sequentially** to the model, such that the `prev_labels` token types can be overwritten by the predicted `labels` of the previous table-question pair. Again, more info can be found in [this notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/TAPAS/Fine_tuning_TapasForQuestionAnswering_on_SQA.ipynb) (for PyTorch) and [this notebook](https://github.com/kamalkraj/Tapas-Tutorial/blob/master/TAPAS/Fine_tuning_TapasForQuestionAnswering_on_SQA.ipynb) (for TensorFlow).
## Documentation resources
- [Text classification task guide](./tasks/sequence_classification)
- [Masked language modeling task guide](./tasks/masked_language_modeling)
## TAPAS specific outputs ## TAPAS specific outputs
[[autodoc]] models.tapas.modeling_tapas.TableQuestionAnsweringOutput [[autodoc]] models.tapas.modeling_tapas.TableQuestionAnsweringOutput
......
...@@ -28,6 +28,9 @@ There are many pretrained variants. Select your pretrained model based on the da ...@@ -28,6 +28,9 @@ There are many pretrained variants. Select your pretrained model based on the da
This model was contributed by [fcakyon](https://huggingface.co/fcakyon). This model was contributed by [fcakyon](https://huggingface.co/fcakyon).
The original code can be found [here](https://github.com/facebookresearch/TimeSformer). The original code can be found [here](https://github.com/facebookresearch/TimeSformer).
## Documentation resources
- [Video classification task guide](./tasks/video_classification)
## TimesformerConfig ## TimesformerConfig
......
...@@ -58,6 +58,10 @@ TransformerXL does **not** work with *torch.nn.DataParallel* due to a bug in PyT ...@@ -58,6 +58,10 @@ TransformerXL does **not** work with *torch.nn.DataParallel* due to a bug in PyT
</Tip> </Tip>
## Documentation resources
- [Text classification task guide](./tasks/sequence_classification)
- [Causal language modeling task guide](./tasks/language_modeling)
## TransfoXLConfig ## TransfoXLConfig
......
...@@ -44,6 +44,10 @@ Tips: ...@@ -44,6 +44,10 @@ Tips:
This model was contributed by [patrickvonplaten](https://huggingface.co/patrickvonplaten). The Authors' code can be This model was contributed by [patrickvonplaten](https://huggingface.co/patrickvonplaten). The Authors' code can be
found [here](https://github.com/microsoft/UniSpeech/tree/main/UniSpeech-SAT). found [here](https://github.com/microsoft/UniSpeech/tree/main/UniSpeech-SAT).
## Documentation resources
- [Audio classification task guide](./tasks/audio_classification)
- [Automatic speech recognition task guide](./tasks/asr)
## UniSpeechSatConfig ## UniSpeechSatConfig
......
...@@ -39,6 +39,10 @@ Tips: ...@@ -39,6 +39,10 @@ Tips:
This model was contributed by [patrickvonplaten](https://huggingface.co/patrickvonplaten). The Authors' code can be This model was contributed by [patrickvonplaten](https://huggingface.co/patrickvonplaten). The Authors' code can be
found [here](https://github.com/microsoft/UniSpeech/tree/main/UniSpeech). found [here](https://github.com/microsoft/UniSpeech/tree/main/UniSpeech).
## Documentation resources
- [Audio classification task guide](./tasks/audio_classification)
- [Automatic speech recognition task guide](./tasks/asr)
## UniSpeechConfig ## UniSpeechConfig
......
...@@ -35,6 +35,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h ...@@ -35,6 +35,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h
- Demo notebooks for UPerNet can be found [here](https://github.com/NielsRogge/Transformers-Tutorials/tree/master/UPerNet). - Demo notebooks for UPerNet can be found [here](https://github.com/NielsRogge/Transformers-Tutorials/tree/master/UPerNet).
- [`UperNetForSemanticSegmentation`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/semantic-segmentation) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/semantic_segmentation.ipynb). - [`UperNetForSemanticSegmentation`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/semantic-segmentation) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/semantic_segmentation.ipynb).
- See also: [Semantic segmentation task guide](./tasks/semantic_segmentation)
If you're interested in submitting a resource to be included here, please feel free to open a Pull Request and we'll review it! The resource should ideally demonstrate something new instead of duplicating an existing resource. If you're interested in submitting a resource to be included here, please feel free to open a Pull Request and we'll review it! The resource should ideally demonstrate something new instead of duplicating an existing resource.
......
...@@ -39,6 +39,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h ...@@ -39,6 +39,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h
<PipelineTag pipeline="image-classification"/> <PipelineTag pipeline="image-classification"/>
- [`VanForImageClassification`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-classification) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/image_classification.ipynb). - [`VanForImageClassification`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-classification) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/image_classification.ipynb).
- See also: [Image classification task guide](./tasks/image_classification)
If you're interested in submitting a resource to be included here, please feel free to open a Pull Request and we'll review it! The resource should ideally demonstrate something new instead of duplicating an existing resource. If you're interested in submitting a resource to be included here, please feel free to open a Pull Request and we'll review it! The resource should ideally demonstrate something new instead of duplicating an existing resource.
......
...@@ -43,7 +43,7 @@ review it! The resource should ideally demonstrate something new instead of dupl ...@@ -43,7 +43,7 @@ review it! The resource should ideally demonstrate something new instead of dupl
**Video classification** **Video classification**
- [A notebook](https://github.com/huggingface/notebooks/blob/main/examples/video_classification.ipynb) that shows how - [A notebook](https://github.com/huggingface/notebooks/blob/main/examples/video_classification.ipynb) that shows how
to fine-tune a VideoMAE model on a custom dataset. to fine-tune a VideoMAE model on a custom dataset.
- [Video classification task page](https://huggingface.co/tasks/video-classification) - [Video classification task guide](./tasks/video-classification)
- [A 🤗 Space](https://huggingface.co/spaces/sayakpaul/video-classification-ucf101-subset) showing how to perform inference with a video classification model. - [A 🤗 Space](https://huggingface.co/spaces/sayakpaul/video-classification-ucf101-subset) showing how to perform inference with a video classification model.
......
...@@ -95,6 +95,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h ...@@ -95,6 +95,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h
- [`ViTForImageClassification`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-classification) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/image_classification.ipynb). - [`ViTForImageClassification`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-classification) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/image_classification.ipynb).
- A blog on fine-tuning [`ViTForImageClassification`] on a custom dataset can be found [here](https://huggingface.co/blog/fine-tune-vit). - A blog on fine-tuning [`ViTForImageClassification`] on a custom dataset can be found [here](https://huggingface.co/blog/fine-tune-vit).
- More demo notebooks to fine-tune [`ViTForImageClassification`] can be found [here](https://github.com/NielsRogge/Transformers-Tutorials/tree/master/VisionTransformer). - More demo notebooks to fine-tune [`ViTForImageClassification`] can be found [here](https://github.com/NielsRogge/Transformers-Tutorials/tree/master/VisionTransformer).
- [Image classification task guide](./tasks/image_classification)
Besides that: Besides that:
......
...@@ -44,6 +44,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h ...@@ -44,6 +44,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h
<PipelineTag pipeline="image-classification"/> <PipelineTag pipeline="image-classification"/>
- [`ViTHybridForImageClassification`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-classification) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/image_classification.ipynb). - [`ViTHybridForImageClassification`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-classification) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/image_classification.ipynb).
- See also: [Image classification task guide](./tasks/image_classification)
If you're interested in submitting a resource to be included here, please feel free to open a Pull Request and we'll review it! The resource should ideally demonstrate something new instead of duplicating an existing resource. If you're interested in submitting a resource to be included here, please feel free to open a Pull Request and we'll review it! The resource should ideally demonstrate something new instead of duplicating an existing resource.
......
...@@ -53,6 +53,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h ...@@ -53,6 +53,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h
<PipelineTag pipeline="image-classification"/> <PipelineTag pipeline="image-classification"/>
- [`ViTMSNForImageClassification`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-classification) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/image_classification.ipynb). - [`ViTMSNForImageClassification`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-classification) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/image_classification.ipynb).
- See also: [Image classification task guide](./tasks/image_classification)
If you're interested in submitting a resource to be included here, please feel free to open a Pull Request and we'll review it! The resource should ideally demonstrate something new instead of duplicating an existing resource. If you're interested in submitting a resource to be included here, please feel free to open a Pull Request and we'll review it! The resource should ideally demonstrate something new instead of duplicating an existing resource.
......
...@@ -33,6 +33,10 @@ an improved word error rate. ...@@ -33,6 +33,10 @@ an improved word error rate.
This model was contributed by [patrickvonplaten](https://huggingface.co/patrickvonplaten). This model was contributed by [patrickvonplaten](https://huggingface.co/patrickvonplaten).
The original code can be found [here](https://github.com/pytorch/fairseq/tree/main/examples/wav2vec). The original code can be found [here](https://github.com/pytorch/fairseq/tree/main/examples/wav2vec).
## Documentation resources
- [Audio classification task guide](./tasks/audio_classification)
- [Automatic speech recognition task guide](./tasks/asr)
## Wav2Vec2ConformerConfig ## Wav2Vec2ConformerConfig
......
...@@ -43,6 +43,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h ...@@ -43,6 +43,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h
- A notebook on how to [leverage a pretrained Wav2Vec2 model for emotion classification](https://colab.research.google.com/github/m3hrdadfi/soxan/blob/main/notebooks/Emotion_recognition_in_Greek_speech_using_Wav2Vec2.ipynb). 🌎 - A notebook on how to [leverage a pretrained Wav2Vec2 model for emotion classification](https://colab.research.google.com/github/m3hrdadfi/soxan/blob/main/notebooks/Emotion_recognition_in_Greek_speech_using_Wav2Vec2.ipynb). 🌎
- [`Wav2Vec2ForCTC`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/audio-classification) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/audio_classification.ipynb). - [`Wav2Vec2ForCTC`] is supported by this [example script](https://github.com/huggingface/transformers/tree/main/examples/pytorch/audio-classification) and [notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/audio_classification.ipynb).
- [Audio classification task guide](./tasks/audio_classification)
<PipelineTag pipeline="automatic-speech-recognition"/> <PipelineTag pipeline="automatic-speech-recognition"/>
...@@ -51,6 +52,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h ...@@ -51,6 +52,7 @@ A list of official Hugging Face and community (indicated by 🌎) resources to h
- A blog post on [finetuning XLS-R for Multi-Lingual ASR with 🤗 Transformers](https://huggingface.co/blog/fine-tune-xlsr-wav2vec2). - A blog post on [finetuning XLS-R for Multi-Lingual ASR with 🤗 Transformers](https://huggingface.co/blog/fine-tune-xlsr-wav2vec2).
- A notebook on how to [create YouTube captions from any video by transcribing audio with Wav2Vec2](https://colab.research.google.com/github/Muennighoff/ytclipcc/blob/main/wav2vec_youtube_captions.ipynb). 🌎 - A notebook on how to [create YouTube captions from any video by transcribing audio with Wav2Vec2](https://colab.research.google.com/github/Muennighoff/ytclipcc/blob/main/wav2vec_youtube_captions.ipynb). 🌎
- [`Wav2Vec2ForCTC`] is supported by a notebook on [how to finetune a speech recognition model in English](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/speech_recognition.ipynb), and [how to finetune a speech recognition model in any language](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/multi_lingual_speech_recognition.ipynb). - [`Wav2Vec2ForCTC`] is supported by a notebook on [how to finetune a speech recognition model in English](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/speech_recognition.ipynb), and [how to finetune a speech recognition model in any language](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/multi_lingual_speech_recognition.ipynb).
- [Automatic speech recognition task guide](./tasks/asr)
🚀 Deploy 🚀 Deploy
......
...@@ -44,6 +44,10 @@ Relevant checkpoints can be found under https://huggingface.co/models?other=wavl ...@@ -44,6 +44,10 @@ Relevant checkpoints can be found under https://huggingface.co/models?other=wavl
This model was contributed by [patrickvonplaten](https://huggingface.co/patrickvonplaten). The Authors' code can be This model was contributed by [patrickvonplaten](https://huggingface.co/patrickvonplaten). The Authors' code can be
found [here](https://github.com/microsoft/unilm/tree/master/wavlm). found [here](https://github.com/microsoft/unilm/tree/master/wavlm).
## Documentation resources
- [Audio classification task guide](./tasks/audio_classification)
- [Automatic speech recognition task guide](./tasks/asr)
## WavLMConfig ## WavLMConfig
......
...@@ -38,6 +38,10 @@ in social value tasks such as hate speech detection in five languages and find i ...@@ -38,6 +38,10 @@ in social value tasks such as hate speech detection in five languages and find i
This model was contributed by [Suraj](https://huggingface.co/valhalla). The original code can be found [here](https://github.com/pytorch/fairseq/tree/main/examples/xglm). This model was contributed by [Suraj](https://huggingface.co/valhalla). The original code can be found [here](https://github.com/pytorch/fairseq/tree/main/examples/xglm).
## Documentation resources
- [Causal language modeling task guide](./tasks/language_modeling)
## XGLMConfig ## XGLMConfig
[[autodoc]] XGLMConfig [[autodoc]] XGLMConfig
......
...@@ -52,6 +52,12 @@ Tips: ...@@ -52,6 +52,12 @@ Tips:
- XLM-ProphetNet's model architecture and pretraining objective is same as ProphetNet, but XLM-ProphetNet was pre-trained on the cross-lingual dataset XGLUE. - XLM-ProphetNet's model architecture and pretraining objective is same as ProphetNet, but XLM-ProphetNet was pre-trained on the cross-lingual dataset XGLUE.
## Documentation resources
- [Causal language modeling task guide](./tasks/language_modeling)
- [Translation task guide](./tasks/translation)
- [Summarization task guide](./tasks/summarization)
## XLMProphetNetConfig ## XLMProphetNetConfig
[[autodoc]] XLMProphetNetConfig [[autodoc]] XLMProphetNetConfig
......
...@@ -28,6 +28,14 @@ Tips: ...@@ -28,6 +28,14 @@ Tips:
This model was contributed by [Soonhwan-Kwon](https://github.com/Soonhwan-Kwon) and [stefan-it](https://huggingface.co/stefan-it). The original code can be found [here](https://github.com/pytorch/fairseq/tree/master/examples/xlmr). This model was contributed by [Soonhwan-Kwon](https://github.com/Soonhwan-Kwon) and [stefan-it](https://huggingface.co/stefan-it). The original code can be found [here](https://github.com/pytorch/fairseq/tree/master/examples/xlmr).
## Documentation resources
- [Text classification task guide](./tasks/sequence_classification)
- [Token classification task guide](./tasks/token_classification)
- [Question answering task guide](./tasks/question_answering)
- [Causal language modeling task guide](./tasks/language_modeling)
- [Masked language modeling task guide](./tasks/masked_language_modeling)
- [Multiple choice task guide](./tasks/multiple_choice)
## XLMRobertaXLConfig ## XLMRobertaXLConfig
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment