Unverified Commit 7e36deec authored by Nicolas Patry's avatar Nicolas Patry Committed by GitHub
Browse files

Fixing some warnings in DeBerta (#8176)

* Fixing some warnings in DeBerta

* Fixing docs with their rewritten version.
parent 05388207
......@@ -15,7 +15,7 @@
""" PyTorch DeBERTa model. """
import math
from collections import Sequence
from collections.abc import Sequence
import torch
from packaging import version
......@@ -480,7 +480,7 @@ class DisentangledSelfAttention(torch.nn.Module):
Parameters:
config (:obj:`str`):
A model config class instance with the configuration to build a new model. The schema is similar to
`BertConfig`, \ for more details, please refer :class:`~transformers.DebertaConfig`
`BertConfig`, for more details, please refer :class:`~transformers.DebertaConfig`
"""
......
......@@ -297,15 +297,15 @@ class GPT2Tokenizer(object):
Args:
vocab_file (:obj:`str`, optional):
The local path of vocabulary package or the release name of vocabulary in `DeBERTa GitHub releases
<https://github.com/microsoft/DeBERTa/releases>`_, \ e.g. "bpe_encoder", default: `None`.
<https://github.com/microsoft/DeBERTa/releases>`_, e.g. "bpe_encoder", default: `None`.
If it's `None`, then it will download the vocabulary in the latest release from GitHub. The vocabulary file
is a \ state dictionary with three items, "dict_map", "vocab", "encoder" which correspond to three files
used in `RoBERTa`, i.e. `dict.txt`, `vocab.txt` and `encoder.json`. \ The difference between our wrapped
GPT2 tokenizer and RoBERTa wrapped tokenizer are,
is a state dictionary with three items, "dict_map", "vocab", "encoder" which correspond to three files used
in `RoBERTa`, i.e. `dict.txt`, `vocab.txt` and `encoder.json`. The difference between our wrapped GPT2
tokenizer and RoBERTa wrapped tokenizer are,
- Special tokens, unlike `RoBERTa` which use `<s>`, `</s>` as the `start` token and `end` token of a
sentence. We use `[CLS]` and `[SEP]` as the `start` and `end`\ token of input sentence which is the same
sentence. We use `[CLS]` and `[SEP]` as the `start` and `end` token of input sentence which is the same
as `BERT`.
- We remapped the token ids in our dictionary with regarding to the new special tokens, `[PAD]` => 0,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment