" <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/models/blob/master/research/seq_flow_lite/demo/colab/emotion_colab.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://github.com/tensorflow/models/blob/master/research/seq_flow_lite/demo/colab/emotion_colab.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
" </td>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dhekoIrWiSsv"
},
"source": [
"In this tutorial, we will work through training a neural emotion prediction model, using the tensorflow-models PIP package, and Bazel.\n",
"\n",
"This tutorial is using GoEmotions, an emotion prediction dataset, available on [TensorFlow TFDS](https://www.tensorflow.org/datasets/catalog/goemotions). We will be training a sequence projection model architecture named PRADO, available on [TensorFlow Model Garden](https://github.com/tensorflow/models/blob/master/research/seq_flow_lite/models/prado.py). Finally, we will examine an application of emotion prediction to emoji suggestions from text."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "grmac7ZYj02a"
},
"source": [
"## Setup"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "D_mi4NZeeB1l"
},
"source": [
"### Install the TensorFlow Model Garden pip package"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tCk46-HdmIyD"
},
"source": [
"`tf-nightly` is the nightly Model Garden package created daily automatically. We install it with pip."
]
},
{
"cell_type": "code",
"metadata": {
"id": "mvO0_HcKx0_V"
},
"source": [
"!pip install tfds-nightly"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "p2wqyg-7mbfV"
},
"source": [
"### Install the Sequence Projection Models package"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1JRZS_aSeINK"
},
"source": [
"Install Bazel: This will allow us to build custom TensorFlow ops used by the PRADO architecture."
"* First, we load the GoEmotions data from TFDS.\n",
"* Next, we prepare the PRADO model for training. We set up the model configuration, including hyperparameters and labels. We also prepare the dataset, which involves projecting the inputs from the dataset, and passing the projections to the model. This is needed because a model training on TPU can not handle string inputs.\n",
"* Finally, we train and evaluate the model and produce model-level and per-label metrics."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YtvR40K8K0Bn"
},
"source": [
"***Start here on Runtime reset***, once the packages above are properly installed:\n",
"* Go to the `seq_flow_lite` directory."
]
},
{
"cell_type": "code",
"metadata": {
"id": "ImEejssVKvxR"
},
"source": [
"%cd models/research/seq_flow_lite"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "uwSPqHXAeQ6H"
},
"source": [
"* Import the Tensorflow and Tensorflow Dataset libraries."
]
},
{
"cell_type": "code",
"metadata": {
"id": "kc4y4n80eL_b"
},
"source": [
"import tensorflow as tf\n",
"import tensorflow_datasets as tfds"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "j-CtG3cagPgl"
},
"source": [
"### The data: GoEmotions\n",
"In this tutorial, we use the [GoEmotions dataset from TFDS](https://www.tensorflow.org/datasets/catalog/goemotions).\n",
"\n",
"GoEmotions is a corpus of comments extracted from Reddit, with human annotations to 27 emotion categories or Neutral.\n",
"\n",
"* Number of labels: 27.\n",
"* Size of training dataset: 43,410.\n",
"* Size of evaluation dataset: 5,427.\n",
"* Maximum sequence length in training and evaluation datasets: 30.\n",
"We train an Emotion Prediction model, based on the [PRADO architecture](https://github.com/tensorflow/models/blob/master/research/seq_flow_lite/models/prado.py) from the [Sequence Projection Models package](https://github.com/tensorflow/models/tree/master/research/seq_flow_lite).\n",
"\n",
"PRADO projects input sequences to fixed sized features. The idea behind this approach is to build embedding-free models that minimize the model size. Instead of using an embedding table to lookup embeddings, sequence projection models compute them on the fly, resulting in space-efficient models.\n",
"\n",
"In this section, we prepare the PRADO model for training.\n",
"\n",
"This GoEmotions dataset is not set up so that it can be directly fed into the PRADO model, so below, we also handle the necessary preprocessing by providing a dataset builder."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "e9uPSZYpgBqP"
},
"source": [
"Prepare the model configuration:\n",
"* Enumerate the labels expected to be found in the GoEmotions dataset.\n",
"* Prepare the `MODEL_CONFIG` dictionary which includes training parameters for the model. See sample configs for the PRADO model [here](https://github.com/tensorflow/models/tree/master/research/seq_flow_lite/configs)."
]
},
{
"cell_type": "code",
"metadata": {
"id": "DkQMnTcLyFeR"
},
"source": [
"LABELS = [\n",
" 'admiration',\n",
" 'amusement',\n",
" 'anger',\n",
" 'annoyance',\n",
" 'approval',\n",
" 'caring',\n",
" 'confusion',\n",
" 'curiosity',\n",
" 'desire',\n",
" 'disappointment',\n",
" 'disapproval',\n",
" 'disgust',\n",
" 'embarrassment',\n",
" 'excitement',\n",
" 'fear',\n",
" 'gratitude',\n",
" 'grief',\n",
" 'joy',\n",
" 'love',\n",
" 'nervousness',\n",
" 'optimism',\n",
" 'pride',\n",
" 'realization',\n",
" 'relief',\n",
" 'remorse',\n",
" 'sadness',\n",
" 'surprise',\n",
" 'neutral',\n",
"]\n",
"\n",
"# Model training parameters.\n",
"CONFIG = {\n",
" 'name': 'models.prado',\n",
" 'batch_size': 1024,\n",
" 'train_steps': 10000,\n",
" 'learning_rate': 0.0006,\n",
" 'learning_rate_decay_steps': 340,\n",
" 'learning_rate_decay_rate': 0.7,\n",
"}\n",
"\n",
"# Limits the amount of logging output produced by the training run, in order to\n",
"Print a batch of examples in model format. This will consist of:\n",
"* the projection tensors (projection and seq_length)\n",
"* the label tensor (second tuple value)\n",
"\n",
"The projection tensor is a **[batch size, max_seq_length, feature_size]** floating point tensor. The **[b, i]** vector is a feature vector of the **i**th token of the **b**th comment_text. The rest of the tensor is zero-padded, and the\n",
"seq_length tensor indicates the number of features vectors for each comment_text.\n",
"\n",
"The label tensor is an indicator tensor of the set of true labels for the example."
]
},
{
"cell_type": "code",
"metadata": {
"id": "1OyK7rjTvBjF"
},
"source": [
"example = next(iter(train_dataset))\n",
"print(\"inputs = {}\".format(example[0]))\n",
"print(\"labels = {}\".format(example[1]))"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "ytMQHT5Kd7A_"
},
"source": [
"In this version of the dataset, the original example has been added as the third element of the tuple."
]
},
{
"cell_type": "code",
"metadata": {
"id": "29EzRoCfI91r"
},
"source": [
"example = next(iter(inspect_dataset))\n",
"print(\"inputs = {}\".format(example[0]))\n",
"print(\"labels = {}\".format(example[1]))\n",
"print(\"original example = {}\".format(example[2]))"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "CLDbHTIvvX11"
},
"source": [
"### Train and Evaluate"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "QqUTa7wXsHoO"
},
"source": [
"First we define a function to build the model. We vary the model inputs depending on task. For training and evaluation, we'll take the projection and sequence length as inputs. Otherwise, we'll take strings as inputs."
]
},
{
"cell_type": "code",
"metadata": {
"id": "erEiNX3ToLZ1"
},
"source": [
"from models import prado\n",
"\n",
"def build_model(mode):\n",
" # First we define our inputs.\n",
" inputs = []\n",
" if mode == base_layers.TRAIN or mode == base_layers.EVAL:\n",
" # For TRAIN and EVAL, we'll be getting dataset examples,\n",
" # so we'll get projections and sequence_lengths.\n",
"## Suggest Emojis using an Emotion Prediction model\n",
"\n",
"In this section, we apply the Emotion Prediction model trained above to suggest emojis relevant to input text.\n",
"\n",
"Refer to our [GoEmotions Model Card](https://github.com/google-research/google-research/blob/master/goemotions/goemotions_model_card.pdf) for additional uses of the model and considerations and limitations for using the GoEmotions data."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "aybpGQV1qr8I"
},
"source": [
"Map each emotion label to a relevant emoji:\n",
"* Emotions are subtle and multi-faceted. In many cases, no one emoji can truely capture the full complexity of the human experience behind each emotion. \n",
"* For the purpose of this exercise, we will select an emoji that captures at least one facet that is conveyed by an emotion label."
]
},
{
"cell_type": "code",
"metadata": {
"id": "lgs12b90qmSQ"
},
"source": [
"EMOJI_MAP = {\n",
" 'admiration': '👏',\n",
" 'amusement': '😂',\n",
" 'anger': '😡',\n",
" 'annoyance': '😒',\n",
" 'approval': '👍',\n",
" 'caring': '🤗',\n",
" 'confusion': '😕',\n",
" 'curiosity': '🤔',\n",
" 'desire': '😍',\n",
" 'disappointment': '😞',\n",
" 'disapproval': '👎',\n",
" 'disgust': '🤮',\n",
" 'embarrassment': '😳',\n",
" 'excitement': '🤩',\n",
" 'fear': '😨',\n",
" 'gratitude': '🙏',\n",
" 'grief': '😢',\n",
" 'joy': '😃',\n",
" 'love': '❤️',\n",
" 'nervousness': '😬',\n",
" 'optimism': '🤞',\n",
" 'pride': '😌',\n",
" 'realization': '💡',\n",
" 'relief': '😅',\n",
" 'remorse': '',\n",
" 'sadness': '😞',\n",
" 'surprise': '😲',\n",
" 'neutral': '',\n",
"}"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "rh_3y7OL7JG_"
},
"source": [
"Select sample inputs:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "rdD6xPpn7Mjm"
},
"source": [
"PREDICT_TEXT = [\n",
" b'Good for you!',\n",
" b'Happy birthday!',\n",
" b'I love you.',\n",
"]"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "vavivya6hGw0"
},
"source": [
"Run inference for the selected examples:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "tJ6iyLlLo5-3"
},
"source": [
"import numpy as np\n",
"\n",
"model = build_model(base_layers.PREDICT)\n",
"model.load_weights('model/model_checkpoint')\n",
"\n",
"for text in PREDICT_TEXT:\n",
" results = model.predict(x=[text])\n",
" print('')\n",
" print('{}:'.format(text))\n",
" labels = np.flip(np.argsort(results[0]))\n",
" for x in range(3):\n",
" label = LABELS[labels[x]]\n",
" label = EMOJI_MAP[label] if EMOJI_MAP[label] else label\n",
