"Now that you've seen the basics, let's build some real model components with autograph.\n",
"\n",
"It's relatively simple to integrate `autograph` with `tf.keras`. But remember that batchng is essential for performance. So the best candidate code for conversion to autograph is code where the control flow is decided at the _batch_ level. If decisions are made at the individual _example_ level you will still need to index and batch your examples to maintain performance while appling the control flow logic. \n",
"\n",
"\n",
"### Stateless functions\n",
"\n",
"For stateless functions like `collatz`, below, the easiest way to include them in a keras model is to wrap them up as a layer uisng `tf.keras.layers.Lambda`."
"<!--TODO(markdaoust) link to full examples or these referenced models.-->\n",
"\n",
"The easiest way is to `@autograph.convert()` the `call` method. See the [keras guide](https://tensorflow.org/guide/keras#build_advanced_models) for details on how to build on these classes. \n",
"\n",
"Here is a simple example of the [stocastic network depth](https://arxiv.org/abs/1603.09382) technique :"
]
},
{
"metadata": {
"id": "DJi_RJkeeOju",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"# `K` is used to check if we're in train or test mode.\n",
"The [standard approach](https://www.tensorflow.org/api_docs/python/tf/keras/layers/RNN) to custom RNN cells has the same issues that are solved by autograph.\n",
"\n",
"Implementing RNN cells with `autograph` is not much different from implementing them [under eager execution](https://colab.sandbox.google.com/github/tensorflow/tensorflow/blob/master/tensorflow/contrib/eager/python/examples/nmt_with_attention/nmt_with_attention.ipynb).\n",
"\n",
"To implement the prediction step in a keras model you could say:\n",
"Since writing control flow in AutoGraph is easy, running a training loop in a TensorFlow graph should also be easy. \n",
"Since writing control flow in AutoGraph is easy, running a training loop in a TensorFlow graph should also be easy. \n",
"\n",
"\n",
"<!--TODO(markdaoust) link to examples showing autograph **in** keras models when ready-->\n",
"Important: While this example wraps a `tf.keras.Model` using AutoGraph, `tf.contrib.autograph` is compatible with `tf.keras` and can be used in [Keras custom layers and models](https://tensorflow.org/guide/keras#build_advanced_models). \n",
"\n",
"Important: While this example wraps a `tf.keras.Model` using AutoGraph, `tf.contrib.autograph` is compatible with `tf.keras` and can be used in [Keras custom layers and models](https://tensorflow.org/guide/keras#build_advanced_models). The easiest way is to `@autograph.convert()` the `call` method.\n",
"\n",
"\n",
"This example shows how to train a simple Keras model on MNIST with the entire training process—loading batches, calculating gradients, updating parameters, calculating validation accuracy, and repeating until convergence—is performed in-graph."
"This example shows how to train a simple Keras model on MNIST with the entire training process—loading batches, calculating gradients, updating parameters, calculating validation accuracy, and repeating until convergence—is performed in-graph."
