Skip to content

GitLab

"examples/vscode:/vscode.git/clone" did not exist on "c891330f7966fae4aec27962e243a1432d0dc6b5"
Unverified Commit 7f03af18 authored by Mark Daoust's avatar Mark Daoust Committed by GitHub
Browse files

Merge pull request #1 from lamberta/edit-get_started 

Copy edits following Mark's updates.
parents 9cf069c4 bff24ace
Hide whitespace changes
Inline Side-by-side
Showing with 249 additions and 126 deletions
samples/core/get_started/eager.ipynb
View file @ 7f03af18
...@@ -5,6 +5,8 @@ ...@@ -5,6 +5,8 @@
"colab": { "colab": {
"name": "eager.ipynb", "name": "eager.ipynb",
"version": "0.3.2", "version": "0.3.2",
"views": {},
"default_view": {},
"provenance": [], "provenance": [],
"private_outputs": true, "private_outputs": true,
"collapsed_sections": [], "collapsed_sections": [],
...@@ -32,7 +34,12 @@ ...@@ -32,7 +34,12 @@
"metadata": { "metadata": {
"id": "CPII1rGR2rF9", "id": "CPII1rGR2rF9",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
...@@ -76,17 +83,22 @@ ...@@ -76,17 +83,22 @@
}, },
"cell_type": "markdown", "cell_type": "markdown",
"source": [ "source": [
"This tutorial describes how to use machine learning to *categorize* Iris flowers by species. It uses [TensorFlow](https://www.tensorflow.org)'s eager execution to (1) build a *model*, (2) *train* the model on example data, and (3) use the model to make *predictions* on unknown data. Machine learning experience isn't required to follow this guide, but you'll need to read some Python code.\n", "This guide uses machine learning to *categorize* Iris flowers by species. It uses [TensorFlow](https://www.tensorflow.org)'s eager execution to:\n",
"1. Build a *model*,\n",
"2. *Train* this model on example data, and\n",
"3. Use the model to make *predictions* about unknown data.\n",
"\n",
"Machine learning experience isn't required, but you'll need to read some Python code.\n",
"\n", "\n",
"## TensorFlow programming\n", "## TensorFlow programming\n",
"\n", "\n",
"There many [TensorFlow APIs](https://www.tensorflow.org/api_docs/python/) available, but we recommend starting with these high-level TensorFlow concepts:\n", "There are many [TensorFlow APIs](https://www.tensorflow.org/api_docs/python/) available, but start with these high-level TensorFlow concepts:\n",
"\n", "\n",
"* Enable an [eager execution](https://www.tensorflow.org/programmers_guide/eager) development environment,\n", "* Enable an [eager execution](https://www.tensorflow.org/programmers_guide/eager) development environment,\n",
"* Import data with the [Datasets API](https://www.tensorflow.org/programmers_guide/datasets),\n", "* Import data with the [Datasets API](https://www.tensorflow.org/programmers_guide/datasets),\n",
"* Build models and layers with TensorFlow's [Keras API](https://keras.io/getting-started/sequential-model-guide/).\n", "* Build models and layers with TensorFlow's [Keras API](https://keras.io/getting-started/sequential-model-guide/).\n",
"\n", "\n",
"This tutorial shows these APIs and is structured like many other TensorFlow programs:\n", "This tutorial is structured like many TensorFlow programs:\n",
"\n", "\n",
"1. Import and parse the data sets.\n", "1. Import and parse the data sets.\n",
"2. Select the type of model.\n", "2. Select the type of model.\n",
...@@ -94,11 +106,13 @@ ...@@ -94,11 +106,13 @@
"4. Evaluate the model's effectiveness.\n", "4. Evaluate the model's effectiveness.\n",
"5. Use the trained model to make predictions.\n", "5. Use the trained model to make predictions.\n",
"\n", "\n",
"To learn more about using TensorFlow, see the [Getting Started guide](https://www.tensorflow.org/get_started/) and the [example tutorials](https://www.tensorflow.org/tutorials/). If you'd like to learn about the basics of machine learning, consider taking the [Machine Learning Crash Course](https://developers.google.com/machine-learning/crash-course/).\n", "For more TensorFlow examples, see the [Get Started](https://www.tensorflow.org/get_started/) and [Tutorials](https://www.tensorflow.org/tutorials/) sections. To learn machine learning basics, consider taking the [Machine Learning Crash Course](https://developers.google.com/machine-learning/crash-course/).\n",
"\n", "\n",
"## Run the notebook\n", "## Run the notebook\n",
"\n", "\n",
"This tutorial is available as an interactive [Colab notebook](https://colab.research.google.com) for you to run and change the Python code directly in the browser. The notebook handles setup and dependencies while you \"play\" cells to execute the code blocks. This is a fun way to explore the program and test ideas. If you are unfamiliar with Python notebook environments, there are a couple of things to keep in mind:\n", "This tutorial is available as an interactive [Colab notebook](https://colab.research.google.com) that can execute and modify Python code directly in the browser. The notebook handles setup and dependencies while you \"play\" cells to run the code blocks. This is a fun way to explore the program and test ideas.\n",
"\n",
"If you are unfamiliar with Python notebook environments, there are a couple of things to keep in mind:\n",
"\n", "\n",
"1. Executing code requires connecting to a runtime environment. In the Colab notebook menu, select *Runtime > Connect to runtime...*\n", "1. Executing code requires connecting to a runtime environment. In the Colab notebook menu, select *Runtime > Connect to runtime...*\n",
"2. Notebook cells are arranged sequentially to gradually build the program. Typically, later code cells depend on prior code cells, though you can always rerun a code block. To execute the entire notebook in order, select *Runtime > Run all*. To rerun a code cell, select the cell and click the *play icon* on the left." "2. Notebook cells are arranged sequentially to gradually build the program. Typically, later code cells depend on prior code cells, though you can always rerun a code block. To execute the entire notebook in order, select *Runtime > Run all*. To rerun a code cell, select the cell and click the *play icon* on the left."
...@@ -130,7 +144,12 @@ ...@@ -130,7 +144,12 @@
"metadata": { "metadata": {
"id": "jBmKxLVy9Uhg", "id": "jBmKxLVy9Uhg",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
...@@ -148,7 +167,7 @@ ...@@ -148,7 +167,7 @@
"source": [ "source": [
"### Configure imports and eager execution\n", "### Configure imports and eager execution\n",
"\n", "\n",
"Import the required Python modules, including TensorFlow, and enable eager execution for this program. Eager execution makes TensorFlow evaluate operations immediately, returning concrete values instead of creating a [computational graph](https://www.tensorflow.org/programmers_guide/graphs) that is executed later. If you are used to a REPL or the `python` interactive console, you'll feel at home.\n", "Import the required Python modules—including TensorFlow—and enable eager execution for this program. Eager execution makes TensorFlow evaluate operations immediately, returning concrete values instead of creating a [computational graph](https://www.tensorflow.org/programmers_guide/graphs) that is executed later. If you are used to a REPL or the `python` interactive console, this feels familiar.\n",
"\n", "\n",
"Once eager execution is enabled, it *cannot* be disabled within the same program. See the [eager execution guide](https://www.tensorflow.org/programmers_guide/eager) for more details." "Once eager execution is enabled, it *cannot* be disabled within the same program. See the [eager execution guide](https://www.tensorflow.org/programmers_guide/eager) for more details."
] ]
...@@ -157,7 +176,12 @@ ...@@ -157,7 +176,12 @@
"metadata": { "metadata": {
"id": "g4Wzg69bnwK2", "id": "g4Wzg69bnwK2",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
...@@ -188,7 +212,7 @@ ...@@ -188,7 +212,7 @@
"\n", "\n",
"Imagine you are a botanist seeking an automated way to categorize each Iris flower you find. Machine learning provides many algorithms to statistically classify flowers. For instance, a sophisticated machine learning program could classify flowers based on photographs. Our ambitions are more modest—we're going to classify Iris flowers based on the length and width measurements of their [sepals](https://en.wikipedia.org/wiki/Sepal) and [petals](https://en.wikipedia.org/wiki/Petal).\n", "Imagine you are a botanist seeking an automated way to categorize each Iris flower you find. Machine learning provides many algorithms to statistically classify flowers. For instance, a sophisticated machine learning program could classify flowers based on photographs. Our ambitions are more modest—we're going to classify Iris flowers based on the length and width measurements of their [sepals](https://en.wikipedia.org/wiki/Sepal) and [petals](https://en.wikipedia.org/wiki/Petal).\n",
"\n", "\n",
"The Iris genus entails about 300 species, but our program will classify only the following three:\n", "The Iris genus entails about 300 species, but our program will only classify the following three:\n",
"\n", "\n",
"* Iris setosa\n", "* Iris setosa\n",
"* Iris virginica\n", "* Iris virginica\n",
...@@ -216,7 +240,7 @@ ...@@ -216,7 +240,7 @@
"source": [ "source": [
"## Import and parse the training dataset\n", "## Import and parse the training dataset\n",
"\n", "\n",
"We need to download the dataset file and convert it to a structure that can be used by this Python program.\n", "Download the dataset file and convert it to a structure that can be used by this Python program.\n",
"\n", "\n",
"### Download the dataset\n", "### Download the dataset\n",
"\n", "\n",
...@@ -227,7 +251,12 @@ ...@@ -227,7 +251,12 @@
"metadata": { "metadata": {
"id": "J6c7uEU9rjRM", "id": "J6c7uEU9rjRM",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
...@@ -257,7 +286,12 @@ ...@@ -257,7 +286,12 @@
"metadata": { "metadata": {
"id": "FQvb_JYdrpPm", "id": "FQvb_JYdrpPm",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
...@@ -273,7 +307,7 @@ ...@@ -273,7 +307,7 @@
}, },
"cell_type": "markdown", "cell_type": "markdown",
"source": [ "source": [
"From this view of the dataset, we see the following:\n", "From this view of the dataset, notice the following:\n",
"\n", "\n",
"1. The first line is a header containing information about the dataset:\n", "1. The first line is a header containing information about the dataset:\n",
" * There are 120 total examples. Each example has four features and one of three possible label names. \n", " * There are 120 total examples. Each example has four features and one of three possible label names. \n",
...@@ -288,13 +322,23 @@ ...@@ -288,13 +322,23 @@
"metadata": { "metadata": {
"id": "9Edhevw7exl6", "id": "9Edhevw7exl6",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
"# column order in CSV file\n",
"column_names = ['sepal_length', 'sepal_width', 'petal_length', 'petal_width', 'species']\n", "column_names = ['sepal_length', 'sepal_width', 'petal_length', 'petal_width', 'species']\n",
"\n",
"feature_names = column_names[:-1]\n", "feature_names = column_names[:-1]\n",
"label_name = column_names[-1]" "label_name = column_names[-1]\n",
"\n",
"print(\"Features: {}\".format(feature_names))\n",
"print(\"Label: {}\".format(label_name))"
], ],
"execution_count": 0, "execution_count": 0,
"outputs": [] "outputs": []
...@@ -319,7 +363,12 @@ ...@@ -319,7 +363,12 @@
"metadata": { "metadata": {
"id": "sVNlJlUOhkoX", "id": "sVNlJlUOhkoX",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
...@@ -337,23 +386,30 @@ ...@@ -337,23 +386,30 @@
"source": [ "source": [
"### Create a `tf.data.Dataset`\n", "### Create a `tf.data.Dataset`\n",
"\n", "\n",
"TensorFlow's [Dataset API](https://www.tensorflow.org/programmers_guide/datasets) handles many common cases for feeding data into a model. This is a high-level API for reading data and transforming it into a form used for training. See the [Datasets Quick Start guide](https://www.tensorflow.org/get_started/datasets_quickstart) for more information.\n", "TensorFlow's [Dataset API](https://www.tensorflow.org/programmers_guide/datasets) handles many common cases for loading data into a model. This is a high-level API for reading data and transforming it into a form used for training. See the [Datasets Quick Start guide](https://www.tensorflow.org/get_started/datasets_quickstart) for more information.\n",
"\n", "\n",
"\n", "\n",
"Since our dataset is a CSV-formatted text file, we'll use the the [make_csv_dataset](https://www.tensorflow.org/api_docs/python/tf/contrib/data/make_csv_dataset) function to easily parse the data into a suitable format. This function is meant to generate data for training models so the default behavior is to shuffle the data (`shuffle=True, shuffle_buffer_size=10000`), and repeat the dataset forever (`num_epochs=None`). Also note the [batch_size](https://developers.google.com/machine-learning/glossary/#batch_size) parameter." "Since the dataset is a CSV-formatted text file, use the the [make_csv_dataset](https://www.tensorflow.org/api_docs/python/tf/contrib/data/make_csv_dataset) function to parse the data into a suitable format. Since this function generates data for training models, the default behavior is to shuffle the data (`shuffle=True, shuffle_buffer_size=10000`), and repeat the dataset forever (`num_epochs=None`). We also set the [batch_size](https://developers.google.com/machine-learning/glossary/#batch_size) parameter."
] ]
}, },
{ {
"metadata": { "metadata": {
"id": "WsxHnz1ebJ2S", "id": "WsxHnz1ebJ2S",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
"batch_size=32\n", "batch_size = 32\n",
"\n",
"train_dataset = tf.contrib.data.make_csv_dataset(\n", "train_dataset = tf.contrib.data.make_csv_dataset(\n",
" train_dataset_fp, batch_size, \n", " train_dataset_fp,\n",
" batch_size, \n",
" column_names=column_names,\n", " column_names=column_names,\n",
" label_name=label_name,\n", " label_name=label_name,\n",
" num_epochs=1)" " num_epochs=1)"
...@@ -368,49 +424,63 @@ ...@@ -368,49 +424,63 @@
}, },
"cell_type": "markdown", "cell_type": "markdown",
"source": [ "source": [
"This function returns a `tf.data.Dataset` of `(features, label)` pairs, where `features` is a `{'feature_name': value}` dictionary.\n", "The `make_csv_dataset` function returns a `tf.data.Dataset` of `(features, label)` pairs, where `features` is a dictionary: `{'feature_name': value}`\n",
"\n", "\n",
"With eager execution enabled, these `Dataset` objects are iterable. Let's look at a batch of features:" "With eager execution enabled, these `Dataset` objects are iterable. Let's look at a batch of features:"
] ]
}, },
{ {
"metadata": { "metadata": {
"id": "kRP72tP9C0Qw", "id": "iDuG94H-C122",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
"features, labels = next(iter(train_dataset))" "features, labels = next(iter(train_dataset))\n",
"\n",
"features"
], ],
"execution_count": 0, "execution_count": 0,
"outputs": [] "outputs": []
}, },
{ {
"metadata": { "metadata": {
"id": "iDuG94H-C122", "id": "E63mArnQaAGz",
"colab_type": "code", "colab_type": "text"
"colab": {}
}, },
"cell_type": "code", "cell_type": "markdown",
"source": [ "source": [
"features" "Notice that like-features are grouped together, or *batched*. Each example row's fields are appended to the corresponding feature array. Change the `batch_size` to set the number of examples stored in these feature arrays.\n",
], "\n",
"execution_count": 0, "You can start to see some clusters by plotting a few features from the batch:"
"outputs": [] ]
}, },
{ {
"metadata": { "metadata": {
"id": "me5Wn-9FcyyO", "id": "me5Wn-9FcyyO",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
"plt.scatter(features['petal_length'], features['sepal_length'], \n", "plt.scatter(features['petal_length'],\n",
" c=labels, cmap='viridis')\n", " features['sepal_length'],\n",
"plt.xlabel(\"Petal Length\")\n", " c=labels,\n",
"plt.ylabel(\"Sepal Length\")\n" " cmap='viridis')\n",
"\n",
"plt.xlabel(\"Petal length\")\n",
"plt.ylabel(\"Sepal length\");"
], ],
"execution_count": 0, "execution_count": 0,
"outputs": [] "outputs": []
...@@ -422,22 +492,28 @@ ...@@ -422,22 +492,28 @@
}, },
"cell_type": "markdown", "cell_type": "markdown",
"source": [ "source": [
"To simplify the model building, let's repackage the features dictionary into an array with shape `(batch_size, num_features)`.\n", "To simplify the model building step, create a function to repackage the features dictionary into a single array with shape: `(batch_size, num_features)`.\n",
"\n",
"To do this we'll write a simple function using the [tf.stack](https://www.tensorflow.org/api_docs/python/tf/stack) method to pack the features into a single array.\n",
"\n", "\n",
"Stack takes a list of tensors, and stacks them along a new axis, like this:\n" "This function uses the [tf.stack](https://www.tensorflow.org/api_docs/python/tf/stack) method which takes values from a list of tensors and creates a combined tensor at the specified dimension."
] ]
}, },
{ {
"metadata": { "metadata": {
"id": "lSI2KLB4CAtc", "id": "jm932WINcaGU",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
"tf.stack(list(features.values()), axis=1)[:10]" "def pack_features_vector(features, labels):\n",
" \"\"\"Pack the features into a single array.\"\"\"\n",
" features = tf.stack(list(features.values()), axis=1)\n",
" return features, labels"
], ],
"execution_count": 0, "execution_count": 0,
"outputs": [] "outputs": []
...@@ -449,21 +525,22 @@ ...@@ -449,21 +525,22 @@
}, },
"cell_type": "markdown", "cell_type": "markdown",
"source": [ "source": [
"Then we'll use the [tf.data.Dataset.map](https://www.tensorflow.org/api_docs/python/tf/data/dataset/map) method to stack the `features` in each `(features,label)` pair in the dataset. " "Then use the [tf.data.Dataset.map](https://www.tensorflow.org/api_docs/python/tf/data/dataset/map) method to pack the `features` of each `(features,label)` pair into the training dataset:"
] ]
}, },
{ {
"metadata": { "metadata": {
"id": "jm932WINcaGU", "id": "ZbDkzGZIkpXf",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
"def pack_features_vector(features,labels):\n",
" features = tf.stack(list(features.values()), axis=1)\n",
" return features, labels\n",
" \n",
"train_dataset = train_dataset.map(pack_features_vector)" "train_dataset = train_dataset.map(pack_features_vector)"
], ],
"execution_count": 0, "execution_count": 0,
...@@ -483,13 +560,18 @@ ...@@ -483,13 +560,18 @@
"metadata": { "metadata": {
"id": "kex9ibEek6Tr", "id": "kex9ibEek6Tr",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
"features, labels = next(iter(train_dataset))\n", "features, labels = next(iter(train_dataset))\n",
"\n", "\n",
"print(features[:10])" "print(features[:5])"
], ],
"execution_count": 0, "execution_count": 0,
"outputs": [] "outputs": []
...@@ -544,7 +626,12 @@ ...@@ -544,7 +626,12 @@
"metadata": { "metadata": {
"id": "2fZ6oL2ig3ZK", "id": "2fZ6oL2ig3ZK",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
...@@ -564,7 +651,7 @@ ...@@ -564,7 +651,7 @@
}, },
"cell_type": "markdown", "cell_type": "markdown",
"source": [ "source": [
"The *[activation function](https://developers.google.com/machine-learning/crash-course/glossary#activation_function)* determines the output shape of each node in the layer. These non-linearities are important as without them the model would be equivalent to a single layer. There are many [available activations](https://www.tensorflow.org/api_docs/python/tf/keras/activations), but [ReLU](https://developers.google.com/machine-learning/crash-course/glossary#ReLU) is common for hidden layers.\n", "The *[activation function](https://developers.google.com/machine-learning/crash-course/glossary#activation_function)* determines the output shape of each node in the layer. These non-linearities are important—without them the model would be equivalent to a single layer. There are many [available activations](https://www.tensorflow.org/api_docs/python/tf/keras/activations), but [ReLU](https://developers.google.com/machine-learning/crash-course/glossary#ReLU) is common for hidden layers.\n",
"\n", "\n",
"The ideal number of hidden layers and neurons depends on the problem and the dataset. Like many aspects of machine learning, picking the best shape of the neural network requires a mixture of knowledge and experimentation. As a rule of thumb, increasing the number of hidden layers and neurons typically creates a more powerful model, which requires more data to train effectively." "The ideal number of hidden layers and neurons depends on the problem and the dataset. Like many aspects of machine learning, picking the best shape of the neural network requires a mixture of knowledge and experimentation. As a rule of thumb, increasing the number of hidden layers and neurons typically creates a more powerful model, which requires more data to train effectively."
] ]
...@@ -585,7 +672,12 @@ ...@@ -585,7 +672,12 @@
"metadata": { "metadata": {
"id": "xe6SQ5NrpB-I", "id": "xe6SQ5NrpB-I",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
...@@ -602,16 +694,21 @@ ...@@ -602,16 +694,21 @@
}, },
"cell_type": "markdown", "cell_type": "markdown",
"source": [ "source": [
"For each example it returns a [logit](https://developers.google.com/machine-learning/crash-course/glossary#logit) for each class. \n", "Here, each example returns a [logit](https://developers.google.com/machine-learning/crash-course/glossary#logit) for each class. \n",
"\n", "\n",
"To convert to a probability for each class, for each example, we use the [softmax](https://developers.google.com/machine-learning/crash-course/glossary#softmax) function:" "To convert these logits to a probability for each class, use the [softmax](https://developers.google.com/machine-learning/crash-course/glossary#softmax) function:"
] ]
}, },
{ {
"metadata": { "metadata": {
"id": "_tRwHZmTNTX2", "id": "_tRwHZmTNTX2",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
...@@ -627,33 +724,24 @@ ...@@ -627,33 +724,24 @@
}, },
"cell_type": "markdown", "cell_type": "markdown",
"source": [ "source": [
"Taking the `tf.argmax` across the classes would give us the predicted class index.\n", "Taking the `tf.argmax` across classes gives us the predicted class index. But, the model hasn't been trained yet, so these aren't good predictions."
"\n",
"The model hasn't been trained yet, so these aren't very good predictions."
] ]
}, },
{ {
"metadata": { "metadata": {
"id": "-Jzm_GoErz8B", "id": "-Jzm_GoErz8B",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
"tf.argmax(predictions, axis=1)" "print(\"Prediction: {}\".format(tf.argmax(predictions, axis=1)))\n",
], "print(\" Labels: {}\".format(labels))"
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "8w3eDAp9o0G9",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"labels"
], ],
"execution_count": 0, "execution_count": 0,
"outputs": [] "outputs": []
...@@ -690,37 +778,22 @@ ...@@ -690,37 +778,22 @@
"metadata": { "metadata": {
"id": "tMAT4DcMPwI-", "id": "tMAT4DcMPwI-",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
"def loss(model, x, y):\n", "def loss(model, x, y):\n",
" y_ = model(x)\n", " y_ = model(x)\n",
" return tf.losses.sparse_softmax_cross_entropy(labels=y, logits=y_)\n", " return tf.losses.sparse_softmax_cross_entropy(labels=y, logits=y_)\n",
"\n" "\n",
], "\n",
"execution_count": 0, "l = loss(model, features, labels)\n",
"outputs": [] "print(\"Loss test: {}\".format(l))"
},
{
"metadata": {
"id": "xSFT90LsQRNV",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"Let's test out this function:"
]
},
{
"metadata": {
"id": "uDdxM3aeQQyx",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"loss(model, features, labels)"
], ],
"execution_count": 0, "execution_count": 0,
"outputs": [] "outputs": []
...@@ -732,14 +805,19 @@ ...@@ -732,14 +805,19 @@
}, },
"cell_type": "markdown", "cell_type": "markdown",
"source": [ "source": [
"To perform the optimization we will use the [tf.GradientTape](https://www.tensorflow.org/api_docs/python/tf/GradientTape) context to calculate the *[gradients](https://developers.google.com/machine-learning/crash-course/glossary#gradient)* used to optimize our model. For more examples of this, see the [eager execution guide](https://www.tensorflow.org/programmers_guide/eager)." "Use the [tf.GradientTape](https://www.tensorflow.org/api_docs/python/tf/GradientTape) context to calculate the *[gradients](https://developers.google.com/machine-learning/crash-course/glossary#gradient)* used to optimize our model. For more examples of this, see the [eager execution guide](https://www.tensorflow.org/programmers_guide/eager)."
] ]
}, },
{ {
"metadata": { "metadata": {
"id": "x57HcKWhKkei", "id": "x57HcKWhKkei",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
...@@ -782,20 +860,25 @@ ...@@ -782,20 +860,25 @@
}, },
"cell_type": "markdown", "cell_type": "markdown",
"source": [ "source": [
"Let's setup the optimizer, and the `global_step` counter:" "Let's setup the optimizer and the `global_step` counter:"
] ]
}, },
{ {
"metadata": { "metadata": {
"id": "8xxi2NNGKwG_", "id": "8xxi2NNGKwG_",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
"\n",
"optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)\n", "optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)\n",
"global_step=tf.train.get_or_create_global_step()" "\n",
"global_step = tf.train.get_or_create_global_step()"
], ],
"execution_count": 0, "execution_count": 0,
"outputs": [] "outputs": []
...@@ -807,27 +890,31 @@ ...@@ -807,27 +890,31 @@
}, },
"cell_type": "markdown", "cell_type": "markdown",
"source": [ "source": [
"Now let's take a single optimization step:" "We'll use this to calculate a single optimization step:"
] ]
}, },
{ {
"metadata": { "metadata": {
"id": "rxRNTFVe56RG", "id": "rxRNTFVe56RG",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
"loss_value, grads = grad(model, features, labels)\n", "loss_value, grads = grad(model, features, labels)\n",
"\n", "\n",
"print(\"Step: \", global_step.numpy())\n", "print(\"Step: {}, Initial Loss: {}\".format(global_step.numpy(),\n",
"print(\"Initial loss:\", loss_value.numpy())\n", " loss_value.numpy()))\n",
"\n", "\n",
"optimizer.apply_gradients(zip(grads, model.variables), global_step)\n", "optimizer.apply_gradients(zip(grads, model.variables), global_step)\n",
"\n", "\n",
"print()\n", "print(\"Step: {}, Loss: {}\".format(global_step.numpy(),\n",
"print(\"Step: \", global_step.numpy())\n", " loss(model, features, labels).numpy()))"
"print(\"Loss: \", loss(model, features, labels).numpy())"
], ],
"execution_count": 0, "execution_count": 0,
"outputs": [] "outputs": []
...@@ -843,7 +930,7 @@ ...@@ -843,7 +930,7 @@
"\n", "\n",
"With all the pieces in place, the model is ready for training! A training loop feeds the dataset examples into the model to help it make better predictions. The following code block sets up these training steps:\n", "With all the pieces in place, the model is ready for training! A training loop feeds the dataset examples into the model to help it make better predictions. The following code block sets up these training steps:\n",
"\n", "\n",
"1. Iterate each epoch. An epoch is one pass through the dataset.\n", "1. Iterate each *epoch*. An epoch is one pass through the dataset.\n",
"2. Within an epoch, iterate over each example in the training `Dataset` grabbing its *features* (`x`) and *label* (`y`).\n", "2. Within an epoch, iterate over each example in the training `Dataset` grabbing its *features* (`x`) and *label* (`y`).\n",
"3. Using the example's features, make a prediction and compare it with the label. Measure the inaccuracy of the prediction and use that to calculate the model's loss and gradients.\n", "3. Using the example's features, make a prediction and compare it with the label. Measure the inaccuracy of the prediction and use that to calculate the model's loss and gradients.\n",
"4. Use an `optimizer` to update the model's variables.\n", "4. Use an `optimizer` to update the model's variables.\n",
...@@ -857,7 +944,12 @@ ...@@ -857,7 +944,12 @@
"metadata": { "metadata": {
"id": "AIgulGRUhpto", "id": "AIgulGRUhpto",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
...@@ -923,7 +1015,12 @@ ...@@ -923,7 +1015,12 @@
"metadata": { "metadata": {
"id": "agjvNd2iUGFn", "id": "agjvNd2iUGFn",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
...@@ -1003,7 +1100,12 @@ ...@@ -1003,7 +1100,12 @@
"metadata": { "metadata": {
"id": "Ps3_9dJ3Lodk", "id": "Ps3_9dJ3Lodk",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
...@@ -1019,12 +1121,18 @@ ...@@ -1019,12 +1121,18 @@
"metadata": { "metadata": {
"id": "SRMWCu30bnxH", "id": "SRMWCu30bnxH",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
"test_dataset = tf.contrib.data.make_csv_dataset(\n", "test_dataset = tf.contrib.data.make_csv_dataset(\n",
" train_dataset_fp, batch_size, \n", " train_dataset_fp,\n",
" batch_size, \n",
" column_names=column_names,\n", " column_names=column_names,\n",
" label_name='species',\n", " label_name='species',\n",
" num_epochs=1,\n", " num_epochs=1,\n",
...@@ -1051,7 +1159,12 @@ ...@@ -1051,7 +1159,12 @@
"metadata": { "metadata": {
"id": "Tw03-MK1cYId", "id": "Tw03-MK1cYId",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
...@@ -1081,7 +1194,12 @@ ...@@ -1081,7 +1194,12 @@
"metadata": { "metadata": {
"id": "uNwt2eMeOane", "id": "uNwt2eMeOane",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
...@@ -1112,7 +1230,12 @@ ...@@ -1112,7 +1230,12 @@
"metadata": { "metadata": {
"id": "kesTS5Lzv-M2", "id": "kesTS5Lzv-M2",
"colab_type": "code", "colab_type": "code",
"colab": {} "colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
}, },
"cell_type": "code", "cell_type": "code",
"source": [ "source": [
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment