Unverified Commit cfb777f2 authored by Joao Gante's avatar Joao Gante Committed by GitHub
Browse files

Docs - Guide to add a new TensorFlow model (#19256)


Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: default avatarMatt <Rocketknight1@users.noreply.github.com>
parent 6a08162a
...@@ -31,7 +31,7 @@ ...@@ -31,7 +31,7 @@
- local: sagemaker - local: sagemaker
title: Run training on Amazon SageMaker title: Run training on Amazon SageMaker
- local: converting_tensorflow_models - local: converting_tensorflow_models
title: Converting TensorFlow Checkpoints title: Converting from TensorFlow checkpoints
- local: serialization - local: serialization
title: Export 🤗 Transformers models title: Export 🤗 Transformers models
- local: troubleshooting - local: troubleshooting
...@@ -109,6 +109,8 @@ ...@@ -109,6 +109,8 @@
title: How to contribute to transformers? title: How to contribute to transformers?
- local: add_new_model - local: add_new_model
title: How to add a model to 🤗 Transformers? title: How to add a model to 🤗 Transformers?
- local: add_tensorflow_model
title: How to convert a 🤗 Transformers model to TensorFlow?
- local: add_new_pipeline - local: add_new_pipeline
title: How to add a pipeline to 🤗 Transformers? title: How to add a pipeline to 🤗 Transformers?
- local: testing - local: testing
...@@ -511,4 +513,4 @@ ...@@ -511,4 +513,4 @@
- local: internal/file_utils - local: internal/file_utils
title: General Utilities title: General Utilities
title: Internal Helpers title: Internal Helpers
title: API title: API
\ No newline at end of file
...@@ -106,7 +106,7 @@ own regarding how code should be written :-) ...@@ -106,7 +106,7 @@ own regarding how code should be written :-)
for a good example). for a good example).
2. The code should be fully understandable, even by a non-native English speaker. This means you should pick 2. The code should be fully understandable, even by a non-native English speaker. This means you should pick
descriptive variable names and avoid abbreviations. As an example, `activation` is preferred to `act`. descriptive variable names and avoid abbreviations. As an example, `activation` is preferred to `act`.
One-letter variable names are strongly discouraged unless it's an index in a for loop. One-letter variable names are strongly discouraged unless it's an index in a for loop.
3. More generally we prefer longer explicit code to short magical one. 3. More generally we prefer longer explicit code to short magical one.
4. Avoid subclassing `nn.Sequential` in PyTorch but subclass `nn.Module` and write the forward pass, so that anyone 4. Avoid subclassing `nn.Sequential` in PyTorch but subclass `nn.Module` and write the forward pass, so that anyone
using your code can quickly debug it by adding print statements or breaking points. using your code can quickly debug it by adding print statements or breaking points.
...@@ -222,7 +222,7 @@ cd .. ...@@ -222,7 +222,7 @@ cd ..
5. To port *brand_new_bert*, you will also need access to its original repository: 5. To port *brand_new_bert*, you will also need access to its original repository:
```bash ```bash
git clone https://github.com/org_that_created_brand_new_bert_org/brand_new_bert.git git clone https://github.com/org_that_created_brand_new_bert_org/brand_new_bert.git
cd brand_new_bert cd brand_new_bert
pip install -e . pip install -e .
``` ```
...@@ -683,10 +683,11 @@ work left to be done should be a cakewalk 😊. ...@@ -683,10 +683,11 @@ work left to be done should be a cakewalk 😊.
At this point, you have successfully added a new model. However, it is very much possible that the model does not yet At this point, you have successfully added a new model. However, it is very much possible that the model does not yet
fully comply with the required design. To make sure, the implementation is fully compatible with 🤗 Transformers, all fully comply with the required design. To make sure, the implementation is fully compatible with 🤗 Transformers, all
common tests should pass. The Cookiecutter should have automatically added a test file for your model, probably under common tests should pass. The Cookiecutter should have automatically added a test file for your model, probably under
the same `tests/test_modeling_brand_new_bert.py`. Run this test file to verify that all common tests pass: the same `tests/models/brand_new_bert/test_modeling_brand_new_bert.py`. Run this test file to verify that all common
tests pass:
```bash ```bash
pytest tests/test_modeling_brand_new_bert.py pytest tests/models/brand_new_bert/test_modeling_brand_new_bert.py
``` ```
Having fixed all common tests, it is now crucial to ensure that all the nice work you have done is well tested, so that Having fixed all common tests, it is now crucial to ensure that all the nice work you have done is well tested, so that
...@@ -700,7 +701,7 @@ Cookiecutter, called `BrandNewBertModelIntegrationTests` and only has to be fill ...@@ -700,7 +701,7 @@ Cookiecutter, called `BrandNewBertModelIntegrationTests` and only has to be fill
tests are passing, run tests are passing, run
```bash ```bash
RUN_SLOW=1 pytest -sv tests/test_modeling_brand_new_bert.py::BrandNewBertModelIntegrationTests RUN_SLOW=1 pytest -sv tests/models/brand_new_bert/test_modeling_brand_new_bert.py::BrandNewBertModelIntegrationTests
``` ```
<Tip> <Tip>
...@@ -758,7 +759,8 @@ contain a couple of hard-coded integration tests. ...@@ -758,7 +759,8 @@ contain a couple of hard-coded integration tests.
**10. Run End-to-end integration tests** **10. Run End-to-end integration tests**
Having added the tokenizer, you should also add a couple of end-to-end integration tests using both the model and the Having added the tokenizer, you should also add a couple of end-to-end integration tests using both the model and the
tokenizer to `tests/test_modeling_brand_new_bert.py` in 🤗 Transformers. Such a test should show on a meaningful tokenizer to `tests/models/brand_new_bert/test_modeling_brand_new_bert.py` in 🤗 Transformers.
Such a test should show on a meaningful
text-to-text sample that the 🤗 Transformers implementation works as expected. A meaningful text-to-text sample can text-to-text sample that the 🤗 Transformers implementation works as expected. A meaningful text-to-text sample can
include *e.g.* a source-to-target-translation pair, an article-to-summary pair, a question-to-answer pair, etc… If none include *e.g.* a source-to-target-translation pair, an article-to-summary pair, a question-to-answer pair, etc… If none
of the ported checkpoints has been fine-tuned on a downstream task it is enough to simply rely on the model tests. In a of the ported checkpoints has been fine-tuned on a downstream task it is enough to simply rely on the model tests. In a
......
This diff is collapsed.
...@@ -10,7 +10,7 @@ an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express o ...@@ -10,7 +10,7 @@ an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express o
specific language governing permissions and limitations under the License. specific language governing permissions and limitations under the License.
--> -->
# Converting Tensorflow Checkpoints # Converting From Tensorflow Checkpoints
A command-line interface is provided to convert original Bert/GPT/GPT-2/Transformer-XL/XLNet/XLM checkpoints to models A command-line interface is provided to convert original Bert/GPT/GPT-2/Transformer-XL/XLNet/XLM checkpoints to models
that can be loaded using the `from_pretrained` methods of the library. that can be loaded using the `from_pretrained` methods of the library.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment