"[View in Colaboratory](https://colab.research.google.com/github/MarkDaoust/models/blob/autopgraph-guide/samples/core/guide/autograph_control_flow.ipynb)"
" <img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" /><span>Run in Google Colab</span></a> \n",
" <img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a> \n",
"</td><td>\n",
"</td><td>\n",
"<a target=\"_blank\" href=\"https://github.com/tensorflow/models/blob/master/samples/core/guide/autograph_control_flow.ipynb\"><img width=32px src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" /><span>View source on GitHub</span></a></td></table>"
"<a target=\"_blank\" href=\"https://github.com/tensorflow/models/blob/master/samples/core/guide/autograph.ipynb\"><img width=32px src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a></td></table>"
]
]
},
},
{
{
...
@@ -87,7 +83,7 @@
...
@@ -87,7 +83,7 @@
},
},
"cell_type": "markdown",
"cell_type": "markdown",
"source": [
"source": [
"[AutoGraph](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/autograph/README.md) helps you write complicated graph code using just plain Python -- behind the scenes, AutoGraph automatically transforms your code into the equivalent TF graph code. We support a large chunk of the Python language, which is growing. [Please see this document for what we currently support, and what we're working on](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/autograph/LIMITATIONS.md)."
"[AutoGraph](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/autograph/) helps you write complicated graph code using normal Python. Behind the scenes, AutoGraph automatically transforms your code into the equivalent [TensorFlow graph code](https://www.tensorflow.org/guide/graphs). AutoGraph already supports much of the Python language, and that coverage continues to grow. For a list of supported Python langauge features, see the [Autograph capabilities and limitations](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/autograph/LIMITATIONS.md)."
]
]
},
},
{
{
...
@@ -97,58 +93,86 @@
...
@@ -97,58 +93,86 @@
},
},
"cell_type": "markdown",
"cell_type": "markdown",
"source": [
"source": [
"### Setup"
"## Setup\n",
"\n",
"To use AutoGraph, install the latest version of TensorFlow:"
"Import TensorFlow and AutoGraph, any supporting modules, and enable [eager execution](https://www.tensorflow.org/guide/eager):"
"\n",
"AutoGraph can convert a large chunk of the Python language into equivalent graph-construction code, and we're adding new supported language features all the time. In this section, we'll give you a taste of some of the functionality in AutoGraph.\n",
"AutoGraph will automatically convert most Python control flow statements into their correct graph equivalent. "
"Here's a quick example of how it works. Autograph can convert functions like this:"
"## Automatically convert Python control flow\n",
"\n",
"AutoGraph will convert much of the Python language into the equivalent TensorFlow graph building code. It converts a function like:"
]
]
},
},
{
{
"metadata": {
"metadata": {
"id": "aA3gOodCBkOw",
"id": "aA3gOodCBkOw",
"colab_type": "code",
"colab_type": "code",
"colab": {}
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
},
"cell_type": "code",
"cell_type": "code",
"source": [
"source": [
...
@@ -169,14 +193,19 @@
...
@@ -169,14 +193,19 @@
},
},
"cell_type": "markdown",
"cell_type": "markdown",
"source": [
"source": [
"Into graph-building functions like this:"
"To a function that uses graphbuilding:"
]
]
},
},
{
{
"metadata": {
"metadata": {
"id": "_EMhGUjRZoKQ",
"id": "_EMhGUjRZoKQ",
"colab_type": "code",
"colab_type": "code",
"colab": {}
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
},
"cell_type": "code",
"cell_type": "code",
"source": [
"source": [
...
@@ -192,14 +221,19 @@
...
@@ -192,14 +221,19 @@
},
},
"cell_type": "markdown",
"cell_type": "markdown",
"source": [
"source": [
"You can take code written for eager execution and run it in a `tf.Graph`. You get the same results, but with all the benfits of graphs:"
"Code written for eager execution can run in a `tf.Graph` with the same results, but with the benfits of graph execution:"
]
]
},
},
{
{
"metadata": {
"metadata": {
"id": "I1RtBvoKBxq5",
"id": "I1RtBvoKBxq5",
"colab_type": "code",
"colab_type": "code",
"colab": {}
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
},
"cell_type": "code",
"cell_type": "code",
"source": [
"source": [
...
@@ -222,7 +256,12 @@
...
@@ -222,7 +256,12 @@
"metadata": {
"metadata": {
"id": "SGjSq0WQvwGs",
"id": "SGjSq0WQvwGs",
"colab_type": "code",
"colab_type": "code",
"colab": {}
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
},
"cell_type": "code",
"cell_type": "code",
"source": [
"source": [
...
@@ -246,14 +285,19 @@
...
@@ -246,14 +285,19 @@
},
},
"cell_type": "markdown",
"cell_type": "markdown",
"source": [
"source": [
"We support common statements like `while`, `for`, `if`, `break`, `return` and more. You can even nest them as much as you like. Imagine trying to write the graph version of this code by hand:\n"
"AutoGraph supports common Python statements like `while`, `for`, `if`, `break`, and `return`, with support for nesting. Compare this function with the complicated graph verson displayed in the following code blocks:"
]
]
},
},
{
{
"metadata": {
"metadata": {
"id": "toxKBOXbB1ro",
"id": "toxKBOXbB1ro",
"colab_type": "code",
"colab_type": "code",
"colab": {}
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
},
"cell_type": "code",
"cell_type": "code",
"source": [
"source": [
...
@@ -281,7 +325,12 @@
...
@@ -281,7 +325,12 @@
"metadata": {
"metadata": {
"id": "jlyQgxYsYSXr",
"id": "jlyQgxYsYSXr",
"colab_type": "code",
"colab_type": "code",
"colab": {}
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
},
"cell_type": "code",
"cell_type": "code",
"source": [
"source": [
...
@@ -297,18 +346,21 @@
...
@@ -297,18 +346,21 @@
},
},
"cell_type": "markdown",
"cell_type": "markdown",
"source": [
"source": [
"Try replacing the `continue` in the above code with `break` -- AutoGraph supports that as well! \n",
"\n",
"## Decorator\n",
"## Decorator\n",
"\n",
"\n",
"If you don't need easy access to the original python function use the `convert` decorator:"
"If you don't need easy access to the original Python function, use the `convert` decorator:"
]
]
},
},
{
{
"metadata": {
"metadata": {
"id": "BKhFNXDic4Mw",
"id": "BKhFNXDic4Mw",
"colab_type": "code",
"colab_type": "code",
"colab": {}
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
},
"cell_type": "code",
"cell_type": "code",
"source": [
"source": [
...
@@ -322,58 +374,55 @@
...
@@ -322,58 +374,55 @@
" print('Buzz')\n",
" print('Buzz')\n",
" else:\n",
" else:\n",
" print(num)\n",
" print(num)\n",
" return num"
" return num\n",
],
"\n",
"execution_count": 0,
"\n",
"outputs": []
"with tf.Graph().as_default():\n",
},
{
"metadata": {
"id": "TUqkNkaadDgy",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"with tf.Graph().as_default(): \n",
" # The result works like a regular op: takes tensors in, returns tensors.\n",
" # The result works like a regular op: takes tensors in, returns tensors.\n",
" # You can inspect the graph using tf.get_default_graph().as_graph_def()\n",
" # You can inspect the graph using tf.get_default_graph().as_graph_def()\n",
" num = tf.placeholder(tf.int32)\n",
" num = tf.placeholder(tf.int32)\n",
" result = fizzbuzz(num)\n",
" result = fizzbuzz(num)\n",
" with tf.Session() as sess:\n",
" with tf.Session() as sess:\n",
" for n in range(10,16):\n",
" for n in range(10,16):\n",
" sess.run(result, feed_dict={num:n})"
" sess.run(result, feed_dict={num:n})"
],
],
"execution_count": 0,
"execution_count": 0,
"outputs": []
"outputs": []
},
},
{
{
"metadata": {
"metadata": {
"id": "raExYNNZgibS",
"id": "-pkEH6OecW7h",
"colab_type": "text"
"colab_type": "text"
},
},
"cell_type": "markdown",
"cell_type": "markdown",
"source": [
"source": [
"## Examples"
"## Examples\n",
"\n",
"Let's demonstrate some useful Python language features."
]
]
},
},
{
{
"metadata": {
"metadata": {
"id": "-pkEH6OecW7h",
"id": "axoRAkWi0CQG",
"colab_type": "text"
"colab_type": "text"
},
},
"cell_type": "markdown",
"cell_type": "markdown",
"source": [
"source": [
"### Assert\n",
"### Assert\n",
"\n",
"\n",
"Let's try some other useful Python constructs, like `print` and `assert`. We automatically convert Python `assert` statements into the equivalent `tf.Assert` code. "
"AutoGraph automatically converts the Python `assert` statement into the equivalent `tf.Assert` code:"
]
]
},
},
{
{
"metadata": {
"metadata": {
"id": "IAOgh62zCPZ4",
"id": "IAOgh62zCPZ4",
"colab_type": "code",
"colab_type": "code",
"colab": {}
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
},
"cell_type": "code",
"cell_type": "code",
"source": [
"source": [
...
@@ -401,14 +450,19 @@
...
@@ -401,14 +450,19 @@
"source": [
"source": [
"### Print\n",
"### Print\n",
"\n",
"\n",
"You can also use plain Python `print` functions in in-graph"
"Use the Python `print` function in-graph:"
]
]
},
},
{
{
"metadata": {
"metadata": {
"id": "ySTsuxnqCTQi",
"id": "ySTsuxnqCTQi",
"colab_type": "code",
"colab_type": "code",
"colab": {}
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
},
"cell_type": "code",
"cell_type": "code",
"source": [
"source": [
...
@@ -436,14 +490,19 @@
...
@@ -436,14 +490,19 @@
"source": [
"source": [
"### Lists\n",
"### Lists\n",
"\n",
"\n",
"Appending to lists in loops also works (we create tensor list ops for you behind the scenes)."
"Append to lists in loops (tensor list ops are automatically created):"
]
]
},
},
{
{
"metadata": {
"metadata": {
"id": "ABX070KwCczR",
"id": "ABX070KwCczR",
"colab_type": "code",
"colab_type": "code",
"colab": {}
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
},
"cell_type": "code",
"cell_type": "code",
"source": [
"source": [
...
@@ -482,7 +541,12 @@
...
@@ -482,7 +541,12 @@
"metadata": {
"metadata": {
"id": "4yyNOf-Twr6s",
"id": "4yyNOf-Twr6s",
"colab_type": "code",
"colab_type": "code",
"colab": {}
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
},
"cell_type": "code",
"cell_type": "code",
"source": [
"source": [
...
@@ -517,7 +581,12 @@
...
@@ -517,7 +581,12 @@
"metadata": {
"metadata": {
"id": "ucmZyQVL03bF",
"id": "ucmZyQVL03bF",
"colab_type": "code",
"colab_type": "code",
"colab": {}
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
},
"cell_type": "code",
"cell_type": "code",
"source": [
"source": [
...
@@ -548,7 +617,12 @@
...
@@ -548,7 +617,12 @@
"metadata": {
"metadata": {
"id": "1sjaFcL717Ig",
"id": "1sjaFcL717Ig",
"colab_type": "code",
"colab_type": "code",
"colab": {}
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
},
"cell_type": "code",
"cell_type": "code",
"source": [
"source": [
...
@@ -572,16 +646,6 @@
...
@@ -572,16 +646,6 @@
"execution_count": 0,
"execution_count": 0,
"outputs": []
"outputs": []
},
},
{
"metadata": {
"id": "hy99pRWpMcuN",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
""
]
},
{
{
"metadata": {
"metadata": {
"id": "4LfnJjm0Bm0B",
"id": "4LfnJjm0Bm0B",
...
@@ -589,15 +653,15 @@
...
@@ -589,15 +653,15 @@
},
},
"cell_type": "markdown",
"cell_type": "markdown",
"source": [
"source": [
"## Advanced example: A training loop in-graph\n",
"## Advanced example: An in-graph training loop\n",
"\n",
"\n",
"Writing control flow in AutoGraph is easy, so running a training loop in a TensorFlow graph should be easy as well! \n",
"Since writing control flow in AutoGraph is easy, running a training loop in a TensorFlow graph should also be easy. \n",
"\n",
"\n",
"<!--TODO(markdaoust) link to examples showing autograph **in** keras models when ready-->\n",
"<!--TODO(markdaoust) link to examples showing autograph **in** keras models when ready-->\n",
"\n",
"\n",
"Important: While this example wraps up a `tf.keras.Model` using autograph, `tf.contrib.autograph` is fully compatible with `tf.keras` and can be used in the definitions [custom of keras layers and models](http://tensorflow.org/guide/keras#build_advanced_models). The easiest way is to `@autograph.convert()` the `call` method.\n",
"Important: While this example wraps a `tf.keras.Model` using AutoGraph, `tf.contrib.autograph` is compatible with `tf.keras` and can be used in [Keras custom layers and models](https://tensorflow.org/guide/keras#build_advanced_models). The easiest way is to `@autograph.convert()` the `call` method.\n",
"\n",
"\n",
"Here, we show an example of training a simple Keras model on MNIST, where the entire training process -- loading batches, calculating gradients, updating parameters, calculating validation accuracy, and repeating until convergence -- is done in-graph."
"This example shows how to train a simple Keras model on MNIST with the entire training process—loading batches, calculating gradients, updating parameters, calculating validation accuracy, and repeating until convergence—is performed in-graph."
