"[AutoGraph](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/autograph/README.md) helps you write complicated graph code using just plain Python -- behind the scenes, AutoGraph automatically transforms your code into the equivalent TF graph code. We support a large chunk of the Python language, which is growing. [Please see this document for what we currently support, and what we're working on](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/autograph/LIMITATIONS.md).\n",
"\n",
"Here's a quick example of how it works:\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
},
"colab_type": "code",
"id": "aA3gOodCBkOw"
},
"outputs": [],
"source": [
"# Autograph can convert functions like this...\n",
"def g(x):\n",
" if x \u003e 0:\n",
" x = x * x\n",
" else:\n",
" x = 0.0\n",
" return x\n",
"\n",
"# ...into graph-building functions like this:\n",
"def tf_g(x):\n",
" with tf.name_scope('g'):\n",
" \n",
" def if_true():\n",
" with tf.name_scope('if_true'):\n",
" x_1, = x,\n",
" x_1 = x_1 * x_1\n",
" return x_1,\n",
"\n",
" def if_false():\n",
" with tf.name_scope('if_false'):\n",
" x_1, = x,\n",
" x_1 = 0.0\n",
" return x_1,\n",
"\n",
" x = autograph_utils.run_cond(tf.greater(x, 0), if_true, if_false)\n",
" return x\n"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
},
"colab_type": "code",
"id": "I1RtBvoKBxq5"
},
"outputs": [],
"source": [
"# You can run your plain-Python code in graph mode,\n",
"# and get the same results out, but with all the benfits of graphs:\n",
"print('Original value: %2.2f' % g(9.0))\n",
"\n",
"# Generate a graph-version of g and call it:\n",
"tf_g = autograph.to_graph(g)\n",
"\n",
"with tf.Graph().as_default(): \n",
" # The result works like a regular op: takes tensors in, returns tensors.\n",
" # You can inspect the graph using tf.get_default_graph().as_graph_def()\n",
"# You can view, debug and tweak the generated code:\n",
"print(autograph.to_code(g))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "m-jWmsCmByyw"
},
"source": [
"#### Automatically converting complex control flow\n",
"\n",
"AutoGraph can convert a large chunk of the Python language into equivalent graph-construction code, and we're adding new supported language features all the time. In this section, we'll give you a taste of some of the functionality in AutoGraph.\n",
"AutoGraph will automatically convert most Python control flow statements into their correct graph equivalent. \n",
" \n",
"We support common statements like `while`, `for`, `if`, `break`, `return` and more. You can even nest them as much as you like. Imagine trying to write the graph version of this code by hand:\n"
"Try replacing the `continue` in the above code with `break` -- AutoGraph supports that as well! \n",
" \n",
"Let's try some other useful Python constructs, like `print` and `assert`. We automatically convert Python `assert` statements into the equivalent `tf.Assert` code. "
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
},
"colab_type": "code",
"id": "IAOgh62zCPZ4"
},
"outputs": [],
"source": [
"def f(x):\n",
" assert x != 0, 'Do not pass zero!'\n",
" return x * x\n",
"\n",
"tf_f = autograph.to_graph(f)\n",
"with tf.Graph().as_default(): \n",
" with tf.Session():\n",
" try:\n",
" print(tf_f(tf.constant(0)).eval())\n",
" except tf.errors.InvalidArgumentError as e:\n",
" print('Got error message:\\n%s' % e.message)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "KRu8iIPBCQr5"
},
"source": [
"You can also use plain Python `print` functions in in-graph"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
},
"colab_type": "code",
"id": "ySTsuxnqCTQi"
},
"outputs": [],
"source": [
"def f(n):\n",
" if n \u003e= 0:\n",
" while n \u003c 5:\n",
" n += 1\n",
" print(n)\n",
" return n\n",
" \n",
"tf_f = autograph.to_graph(f)\n",
"with tf.Graph().as_default():\n",
" with tf.Session():\n",
" tf_f(tf.constant(0)).eval()"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "NqF0GT-VCVFh"
},
"source": [
"Appending to lists in loops also works (we create a `TensorArray` for you behind the scenes)"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
},
"colab_type": "code",
"id": "ABX070KwCczR"
},
"outputs": [],
"source": [
"def f(n):\n",
" z = []\n",
" # We ask you to tell us the element dtype of the list\n",
" z = autograph.utils.set_element_type(z, tf.int32)\n",
" for i in range(n):\n",
" z.append(i)\n",
" # when you're done with the list, stack it\n",
" # (this is just like np.stack)\n",
" return autograph.stack(z) \n",
"\n",
"tf_f = autograph.to_graph(f)\n",
"with tf.Graph().as_default(): \n",
" with tf.Session():\n",
" print(tf_f(tf.constant(3)).eval())\n",
"\n",
"print('\\n\\n'+autograph.to_code(f))"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
},
"colab_type": "code",
"id": "iu5IF7n2Df7C"
},
"outputs": [],
"source": [
"def fizzbuzz(num):\n",
" if num % 3 == 0 and num % 5 == 0:\n",
" print('FizzBuzz')\n",
" elif num % 3 == 0:\n",
" print('Fizz')\n",
" elif num % 5 == 0:\n",
" print('Buzz')\n",
" else:\n",
" print(num)\n",
" return num"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
},
"colab_type": "code",
"id": "EExAjWuwDPpR"
},
"outputs": [],
"source": [
"tf_g = autograph.to_graph(fizzbuzz)\n",
"\n",
"with tf.Graph().as_default(): \n",
" # The result works like a regular op: takes tensors in, returns tensors.\n",
" # You can inspect the graph using tf.get_default_graph().as_graph_def()\n",
" g_ops = tf_g(tf.constant(15))\n",
" with tf.Session() as sess:\n",
" sess.run(g_ops) \n",
" \n",
"# You can view, debug and tweak the generated code:\n",
"print('\\n')\n",
"print(autograph.to_code(fizzbuzz))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "SzpKGzVpBkph"
},
"source": [
"# De-graphify Exercises\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "8k23dxcSmmXq"
},
"source": [
"#### Easy print statements"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
},
"colab_type": "code",
"id": "dE1Vsmp-mlpK"
},
"outputs": [],
"source": [
"# See what happens when you turn AutoGraph off.\n",
"# Do you see the type or the value of x when you print it?\n",
"\n",
"# @autograph.convert()\n",
"def square_log(x):\n",
" x = x * x\n",
" print('Squared value of x =', x)\n",
" return x\n",
"\n",
"\n",
"with tf.Graph().as_default(): \n",
" with tf.Session() as sess:\n",
" print(sess.run(square_log(tf.constant(4))))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "_R-Q7BbxmkBF"
},
"source": [
"#### Now some exercises. Convert the TensorFlow code into AutoGraph'd Python code."
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
},
"colab_type": "code",
"id": "SwA11tO-yCvg"
},
"outputs": [],
"source": [
"def square_if_positive(x):\n",
" x = tf.cond(tf.greater(x, 0), lambda: x * x, lambda: x)\n",
"Writing control flow in AutoGraph is easy, so running a training loop in a TensorFlow graph should be easy as well! \n",
"\n",
"Here, we show an example of training a simple Keras model on MNIST, where the entire training process -- loading batches, calculating gradients, updating parameters, calculating validation accuracy, and repeating until convergence -- is done in-graph."
