Commit 4a8e45d1 authored by A. Unique TensorFlower's avatar A. Unique TensorFlower
Browse files

Internal change

PiperOrigin-RevId: 470722284
parent 1587d2db
# TF-NLP Model Garden # TF-NLP Model Garden
⚠️ Disclaimer: All datasets hyperlinked from this page are not owned or ⚠️ Disclaimer: All datasets hyperlinked from this page are not owned or
distributed by Google. The dataset is made available by third parties. distributed by Google. The dataset is made available by third parties. Please
Please review the terms and conditions made available by the third parties review the terms and conditions made available by the third parties before using
before using the data. the data.
This codebase provides a Natrual Language Processing modeling toolkit written in This codebase provides a Natural Language Processing modeling toolkit written in
[TF2](https://www.tensorflow.org/guide/effective_tf2). It allows researchers and [TF2](https://www.tensorflow.org/guide/effective_tf2). It allows researchers and
developers to reproduce state-of-the-art model results and train custom models developers to reproduce state-of-the-art model results and train custom models
to experiment new research ideas. to experiment new research ideas.
## Features ## Features
* Reusable and modularized modeling building blocks * Reusable and modularized modeling building blocks
* State-of-the-art reproducible * State-of-the-art reproducible
* Easy to customize and extend * Easy to customize and extend
* End-to-end training * End-to-end training
* Distributed trainable on both GPUs and TPUs * Distributed trainable on both GPUs and TPUs
## Major components ## Major components
### Libraries ### Libraries
We provide modeling library to allow users to train custom models for new We provide modeling library to allow users to train custom models for new
research ideas. Detailed intructions can be found in READMEs in each folder. research ideas. Detailed instructions can be found in READMEs in each folder.
* [modeling/](modeling): modeling library that provides building blocks * [modeling/](modeling): modeling library that provides building blocks
(e.g.,Layers, Networks, and Models) that can be assembled into (e.g.,Layers, Networks, and Models) that can be assembled into
transformer-based achitectures . transformer-based architectures.
* [data/](data): binaries and utils for input preprocessing, tokenization, * [data/](data): binaries and utils for input preprocessing, tokenization,
etc. etc.
...@@ -35,14 +35,14 @@ research ideas. Detailed intructions can be found in READMEs in each folder. ...@@ -35,14 +35,14 @@ research ideas. Detailed intructions can be found in READMEs in each folder.
We provide SoTA model implementations, pre-trained models, training and We provide SoTA model implementations, pre-trained models, training and
evaluation examples, and command lines. Detail instructions can be found in the evaluation examples, and command lines. Detail instructions can be found in the
READMEs for specific papers. Below are some papers implemented in the READMEs for specific papers. Below are some papers implemented in the repository
repository and more NLP projects can be found in the and more NLP projects can be found in the
[`projects`](https://github.com/tensorflow/models/tree/master/official/projects) [`projects`](https://github.com/tensorflow/models/tree/master/official/projects)
folder: folder:
1. [BERT](MODEL_GARDEN.md#available-model-configs): [BERT: Pre-training of Deep Bidirectional Transformers for 1. [BERT](MODEL_GARDEN.md#available-model-configs): [BERT: Pre-training of Deep
Language Understanding](https://arxiv.org/abs/1810.04805) by Devlin et al., Bidirectional Transformers for Language
2018 Understanding](https://arxiv.org/abs/1810.04805) by Devlin et al., 2018
2. [ALBERT](MODEL_GARDEN.md#available-model-configs): 2. [ALBERT](MODEL_GARDEN.md#available-model-configs):
[A Lite BERT for Self-supervised Learning of Language Representations](https://arxiv.org/abs/1909.11942) [A Lite BERT for Self-supervised Learning of Language Representations](https://arxiv.org/abs/1909.11942)
by Lan et al., 2019 by Lan et al., 2019
...@@ -56,9 +56,8 @@ folder: ...@@ -56,9 +56,8 @@ folder:
### Common Training Driver ### Common Training Driver
We provide a single common driver [train.py](train.py) to train above SoTA We provide a single common driver [train.py](train.py) to train above SoTA
models on popluar tasks. Please see [docs/train.md](docs/train.md) for models on popular tasks. Please see [docs/train.md](docs/train.md) for more
more details. details.
### Pre-trained models with checkpoints and TF-Hub ### Pre-trained models with checkpoints and TF-Hub
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment