Skip to content

GitLab

Unverified Commit a50bbe58 authored by VoVAllen's avatar VoVAllen Committed by GitHub
Browse files

Merge branch 'master' into master

parents 20469802 4d7b3ba8
Hide whitespace changes
Inline Side-by-side
Showing with 18622 additions and 2 deletions
README.md
View file @ a50bbe58
...@@ -9,4 +9,4 @@ For how to install and how to play with DGL, please read our ...@@ -9,4 +9,4 @@ For how to install and how to play with DGL, please read our
## Contribution rules ## Contribution rules
No direct push to master branch. All changes need to be PRed and reviewed before merging to the No direct push to master branch. All changes need to be PRed and reviewed before merging to the
master branch. master branch.
\ No newline at end of file
docs/source/conf.py
View file @ a50bbe58
...@@ -193,4 +193,4 @@ sphinx_gallery_conf = { ...@@ -193,4 +193,4 @@ sphinx_gallery_conf = {
'gallery_dirs' : gallery_dirs, 'gallery_dirs' : gallery_dirs,
'within_subsection_order' : FileNameSortKey, 'within_subsection_order' : FileNameSortKey,
'filename_pattern' : '.py', 'filename_pattern' : '.py',
} }
\ No newline at end of file
tutorial/capsule/Capsule Tutorial(WIP).html 0 → 100644
View file @ a50bbe58
This source diff could not be displayed because it is too large. You can view the blob instead.
tutorial/capsule/Capsule Tutorial(WIP).ipynb 0 → 100644
View file @ a50bbe58
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Capsule Network\n",
"================\n",
"**Author**: `Jinjing Zhou`\n",
"\n",
"This tutorial explains how to use DGL library and its language to implement the [capsule network](http://arxiv.org/abs/1710.09829) proposed by Geoffrey Hinton and his team. The algorithm aims to provide a better alternative to current neural network structures. By using DGL library, users can implement the algorithm in a more intuitive way."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Model Overview\n",
"\n",
"### Introduction\n",
"Capsule Network were first introduced in 2011 by Geoffrey Hinton, et al., in a paper called [Transforming Autoencoders](https://www.cs.toronto.edu/~fritz/absps/transauto6.pdf), but it was only a few months ago, in November 2017, that Sara Sabour, Nicholas Frosst, and Geoffrey Hinton published a paper called Dynamic Routing between Capsules, where they introduced a CapsNet architecture that reached state-of-the-art performance on MNIST.\n",
"\n",
"### What's a capsule?\n",
"> A capsule is a group of neurons whose activity vector represents the instantiation parameters of a specific type of entity such as an object or an object part. \n",
"\n",
"Generally Speaking, the idea of capsule is to encode all the information about the features into a vector form, by substituting scalars in traditional neural network with vectors. And use the norm of the vector to represents the meaning of original scalars. \n",
"![figure_1](./capsule_f1.png)\n",
"\n",
"### Dynamic Routing Algorithm\n",
"Due to the different structure of network, capsules network has different operations to calculate results. This figure shows the comparison, drawn by [Max Pechyonkin](https://medium.com/ai%C2%B3-theory-practice-business/understanding-hintons-capsule-networks-part-ii-how-capsules-work-153b6ade9f66O). \n",
"<img src=\"./capsule_f2.png\" style=\"height:250px;\"/><br/>\n",
"\n",
"The key idea is that the output of each capsule is the sum of weighted input vectors. We will go into details in the later section with code implementations.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Model Implementations\n",
"\n",
"### 1. Consider capsule routing as a graph structure\n",
"\n",
"We can consider each capsule as a node in a graph, and connect all the nodes between layers.\n",
"<img src=\"./capsule_f3.png\" style=\"height:200px;\"/>"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def construct_graph(self):\n",
" g = dgl.DGLGraph()\n",
" g.add_nodes(self.input_capsule_num + self.output_capsule_num)\n",
" input_nodes = list(range(self.input_capsule_num))\n",
" output_nodes = list(range(self.input_capsule_num, self.input_capsule_num + self.output_capsule_num))\n",
" u, v = [], []\n",
" for i in input_nodes:\n",
" for j in output_nodes:\n",
" u.append(i)\n",
" v.append(j)\n",
" g.add_edges(u, v)\n",
" return g, input_nodes, output_nodes"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2. Initialization & Affine Transformation\n",
"- Pre-compute $\\hat{u}_{j|i}$, initialize $b_{ij}$ and store them as edge attribute\n",
"- Initialize node features as zero\n",
"<img src=\"./capsule_f4.png\" style=\"height:200px;\"/>"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# x is the input vextor with shape [batch_size, input_capsule_dim, input_num]\n",
"# Transpose x to [batch_size, input_num, input_capsule_dim] \n",
"x = x.transpose(1, 2)\n",
"# Expand x to [batch_size, input_num, output_num, input_capsule_dim, 1]\n",
"x = torch.stack([x] * self.output_capsule_num, dim=2).unsqueeze(4)\n",
"# Expand W from [input_num, output_num, input_capsule_dim, output_capsule_dim] \n",
"# to [batch_size, input_num, output_num, output_capsule_dim, input_capsule_dim] \n",
"W = self.weight.expand(self.batch_size, *self.weight.size())\n",
"# u_hat's shape is [input_num, output_num, batch_size, output_capsule_dim]\n",
"u_hat = torch.matmul(W, x).permute(1, 2, 0, 3, 4).squeeze().contiguous()\n",
"\n",
"b_ij = torch.zeros(self.input_capsule_num, self.output_capsule_num).to(self.device)\n",
"\n",
"self.g.set_e_repr({'b_ij': b_ij.view(-1)})\n",
"self.g.set_e_repr({'u_hat': u_hat.view(-1, self.batch_size, self.unit_size)})\n",
"\n",
"# Initialize all node features as zero\n",
"node_features = torch.zeros(self.input_capsule_num + self.output_capsule_num, self.batch_size,\n",
" self.output_capsule_dim).to(self.device)\n",
"self.g.set_n_repr({'h': node_features})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3. Write Message Passing functions and Squash function"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 3.1 Squash function\n",
"Squashing function is to ensure that short vectors get shrunk to almost zero length and long vectors get shrunk to a length slightly below 1.\n",
"![squash](./squash.png)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"def squash(s):\n",
" msg_sq = torch.sum(s ** 2, dim=2, keepdim=True)\n",
" msg = torch.sqrt(msg_sq)\n",
" s = (msg_sq / (1.0 + msg_sq)) * (s / msg)\n",
" return s"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 3.2 Message Functions\n",
"At first stage, we need to define a message function to get all the attributes we need in the further computations."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def capsule_msg(src, edge):\n",
" return {'b_ij': edge['b_ij'], 'h': src['h'], 'u_hat': edge['u_hat']}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 3.3 Reduce Functions\n",
"At this stage, we need to define a reduce function to aggregate all the information we get from message function into node features.\n",
"This step implements the line 4 and line 5 in routing algorithms, which softmax over $b_{ij}$ and calculate weighted sum of input features. Note that softmax operation is over dimension $j$ instead of $i$. \n",
"<img src=\"./capsule_f5.png\" style=\"height:300px\">"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"def capsule_reduce(node, msg):\n",
" b_ij_c, u_hat = msg['b_ij'], msg['u_hat']\n",
" # line 4\n",
" c_i = F.softmax(b_ij_c, dim=0)\n",
" # line 5\n",
" s_j = (c_i.unsqueeze(2).unsqueeze(3) * u_hat).sum(dim=1)\n",
" return {'h': s_j}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 3.4 Node Update Functions\n",
"Squash the intermidiate representations into node features $v_j$\n",
"![step6](./step6.png)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def capsule_update(msg):\n",
" # line 6\n",
" v_j = squash(msg['h'])\n",
" return {'h': v_j}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 3.5 Edge Update Functions\n",
"Update the routing parameters\n",
"![step7](./step7.png)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def update_edge(self, u, v, edge):\n",
" return {'b_ij': edge['b_ij'] + (v['h'] * edge['u_hat']).mean(dim=1).sum(dim=1)}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4. Executing algorithm\n",
"Call `update_all` and `update_edge` functions to execute the algorithms"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"for i in range(self.num_routing):\n",
" self.g.update_all(self.capsule_msg, self.capsule_reduce, self.capsule_update)\n",
" self.g.update_edge(edge_func=self.update_edge)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.6",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.1"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
tutorial/capsule/Simple Routing.ipynb 0 → 100644
View file @ a50bbe58
{
"cells": [
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"env: DGL_LIBRARY_PATH=/data/jinjing/dgl/build\n",
"Populating the interactive namespace from numpy and matplotlib\n"
]
}
],
"source": [
"%env DGL_LIBRARY_PATH=/data/jinjing/dgl/build\n",
"%pylab inline\n",
"%config InlineBackend.figure_format = 'svg'\n",
"\n",
"import torch as th\n",
"import numpy as np\n",
"import torch.nn as nn\n",
"import torch.nn.functional as F\n",
"import dgl\n",
"import networkx as nx\n",
"\n",
"\n",
"def update_edge(u, v, edge):\n",
" return {'b_ij': edge['b_ij'] + (v['h'] * edge['u_hat']).sum(dim=1)}\n",
"\n",
"\n",
"def v2_reduce(node, msg):\n",
" b_ij_c, u_hat = msg['b_ij'], msg['u_hat']\n",
" c_i = F.softmax(b_ij_c, dim=0)\n",
" s_j = (c_i.unsqueeze(2) * u_hat).sum(dim=1)\n",
" return {'s_j': s_j}\n",
"\n",
"\n",
"def capsule_msg(src, edge):\n",
" return {'b_ij': edge['b_ij'], 'h': src['h'], 'u_hat': edge['u_hat']}\n",
"\n",
"\n",
"def squash(s):\n",
" s = s['s_j']\n",
" msg_sq = th.sum(s**2, dim=1, keepdim=True)\n",
" msg = th.sqrt(msg_sq)\n",
" s = (msg_sq / (1.0 + msg_sq)) * (s / msg)\n",
" return {'h': s}\n",
"\n",
"\n",
"np.random.seed(10)\n",
"num_points = 10\n",
"num_points2 = 2\n",
"\n",
"theta_dist1 = np.pi * (np.random.rand(num_points) * 2)\n",
"\n",
"radius_1 = np.random.rand(num_points) * 2 + 3\n",
"points_1 = th.from_numpy(\n",
" np.array([np.cos(theta_dist1) * radius_1,\n",
" np.sin(theta_dist1) * radius_1])).float()"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [],
"source": [
"g = dgl.DGLGraph()\n",
"g.add_nodes(num_points + num_points2)\n",
"\n",
"for i in range(num_points):\n",
" for j in range(num_points2):\n",
" g.add_edge(i, j + num_points)\n",
"\n",
"nodes1 = list(range(num_points))\n",
"nodes2 = list(range(num_points, num_points + num_points2))"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [],
"source": [
"W = th.randn((num_points2, num_points, 2, 2))\n",
"u_hat = th.matmul(points_1.t().unsqueeze(1), W).squeeze(2).view(-1, 2)"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([10, 1, 2])"
]
},
"execution_count": 29,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"points_1.t().unsqueeze(1).shape"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([20, 2])"
]
},
"execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"u_hat.shape"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [],
"source": [
"g.set_n_repr({'h': th.cat([points_1, th.zeros((2, num_points2))], dim=1).t()})\n",
"g.set_e_repr({'b_ij': th.zeros(g.number_of_edges()).float()})\n",
"g.set_e_repr({'u_hat': u_hat})"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/jinjing/.pyenv/versions/3.6.1/lib/python3.6/site-packages/dgl-0.0.1-py3.6.egg/dgl/frame.py:256: UserWarning: Initializer is not set. Use zero initializer instead. To suppress this warning, use `set_initializer` to explicitly specify which initializer to use.\n",
" dgl_warning('Initializer is not set. Use zero initializer instead.'\n"
]
},
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Created with matplotlib (http://matplotlib.org/) -->\n",
"<svg height=\"263.63625pt\" version=\"1.1\" viewBox=\"0 0 374.442187 263.63625\" width=\"374.442187pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
" <defs>\n",
" <style type=\"text/css\">\n",
"*{stroke-linecap:butt;stroke-linejoin:round;}\n",
" </style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 263.63625 \n",
"L 374.442187 263.63625 \n",
"L 374.442187 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill:none;\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 28.942188 239.758125 \n",
"L 363.742188 239.758125 \n",
"L 363.742188 22.318125 \n",
"L 28.942188 22.318125 \n",
"z\n",
"\" style=\"fill:#ffffff;\"/>\n",
" </g>\n",
" <g id=\"PathCollection_1\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"m4bf6dda95f\" style=\"stroke:#0000ff;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#pb261dca43e)\">\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"213.047736\" xlink:href=\"#m4bf6dda95f\" y=\"112.371934\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"199.257259\" xlink:href=\"#m4bf6dda95f\" y=\"152.20422\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_2\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"m40ceb65a15\" style=\"stroke:#ff0000;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#pb261dca43e)\">\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"211.200217\" xlink:href=\"#m40ceb65a15\" y=\"229.260355\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"348.382418\" xlink:href=\"#m40ceb65a15\" y=\"115.159245\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"128.148112\" xlink:href=\"#m40ceb65a15\" y=\"181.221381\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"191.526559\" xlink:href=\"#m40ceb65a15\" y=\"222.204952\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"44.301957\" xlink:href=\"#m40ceb65a15\" y=\"128.811073\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"213.844178\" xlink:href=\"#m40ceb65a15\" y=\"34.013391\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"238.137448\" xlink:href=\"#m40ceb65a15\" y=\"33.168056\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"200.087941\" xlink:href=\"#m40ceb65a15\" y=\"211.910537\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"267.895332\" xlink:href=\"#m40ceb65a15\" y=\"32.815895\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"313.106955\" xlink:href=\"#m40ceb65a15\" y=\"76.215599\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" id=\"m7d2824b1a8\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"64.329202\" xlink:href=\"#m7d2824b1a8\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- −4 -->\n",
" <defs>\n",
" <path d=\"M 10.59375 35.5 \n",
"L 73.1875 35.5 \n",
"L 73.1875 27.203125 \n",
"L 10.59375 27.203125 \n",
"z\n",
"\" id=\"DejaVuSans-2212\"/>\n",
" <path d=\"M 37.796875 64.3125 \n",
"L 12.890625 25.390625 \n",
"L 37.796875 25.390625 \n",
"z\n",
"M 35.203125 72.90625 \n",
"L 47.609375 72.90625 \n",
"L 47.609375 25.390625 \n",
"L 58.015625 25.390625 \n",
"L 58.015625 17.1875 \n",
"L 47.609375 17.1875 \n",
"L 47.609375 0 \n",
"L 37.796875 0 \n",
"L 37.796875 17.1875 \n",
"L 4.890625 17.1875 \n",
"L 4.890625 26.703125 \n",
"z\n",
"\" id=\"DejaVuSans-34\"/>\n",
" </defs>\n",
" <g transform=\"translate(56.958108 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_2\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"128.412424\" xlink:href=\"#m7d2824b1a8\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- −2 -->\n",
" <defs>\n",
" <path d=\"M 19.1875 8.296875 \n",
"L 53.609375 8.296875 \n",
"L 53.609375 0 \n",
"L 7.328125 0 \n",
"L 7.328125 8.296875 \n",
"Q 12.9375 14.109375 22.625 23.890625 \n",
"Q 32.328125 33.6875 34.8125 36.53125 \n",
"Q 39.546875 41.84375 41.421875 45.53125 \n",
"Q 43.3125 49.21875 43.3125 52.78125 \n",
"Q 43.3125 58.59375 39.234375 62.25 \n",
"Q 35.15625 65.921875 28.609375 65.921875 \n",
"Q 23.96875 65.921875 18.8125 64.3125 \n",
"Q 13.671875 62.703125 7.8125 59.421875 \n",
"L 7.8125 69.390625 \n",
"Q 13.765625 71.78125 18.9375 73 \n",
"Q 24.125 74.21875 28.421875 74.21875 \n",
"Q 39.75 74.21875 46.484375 68.546875 \n",
"Q 53.21875 62.890625 53.21875 53.421875 \n",
"Q 53.21875 48.921875 51.53125 44.890625 \n",
"Q 49.859375 40.875 45.40625 35.40625 \n",
"Q 44.1875 33.984375 37.640625 27.21875 \n",
"Q 31.109375 20.453125 19.1875 8.296875 \n",
"z\n",
"\" id=\"DejaVuSans-32\"/>\n",
" </defs>\n",
" <g transform=\"translate(121.041331 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_3\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"192.495647\" xlink:href=\"#m7d2824b1a8\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 0 -->\n",
" <defs>\n",
" <path d=\"M 31.78125 66.40625 \n",
"Q 24.171875 66.40625 20.328125 58.90625 \n",
"Q 16.5 51.421875 16.5 36.375 \n",
"Q 16.5 21.390625 20.328125 13.890625 \n",
"Q 24.171875 6.390625 31.78125 6.390625 \n",
"Q 39.453125 6.390625 43.28125 13.890625 \n",
"Q 47.125 21.390625 47.125 36.375 \n",
"Q 47.125 51.421875 43.28125 58.90625 \n",
"Q 39.453125 66.40625 31.78125 66.40625 \n",
"z\n",
"M 31.78125 74.21875 \n",
"Q 44.046875 74.21875 50.515625 64.515625 \n",
"Q 56.984375 54.828125 56.984375 36.375 \n",
"Q 56.984375 17.96875 50.515625 8.265625 \n",
"Q 44.046875 -1.421875 31.78125 -1.421875 \n",
"Q 19.53125 -1.421875 13.0625 8.265625 \n",
"Q 6.59375 17.96875 6.59375 36.375 \n",
"Q 6.59375 54.828125 13.0625 64.515625 \n",
"Q 19.53125 74.21875 31.78125 74.21875 \n",
"z\n",
"\" id=\"DejaVuSans-30\"/>\n",
" </defs>\n",
" <g transform=\"translate(189.314397 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"256.57887\" xlink:href=\"#m7d2824b1a8\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(253.39762 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_5\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"320.662093\" xlink:href=\"#m7d2824b1a8\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(317.480843 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_6\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" id=\"m0efda133e9\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m0efda133e9\" y=\"221.647687\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- −4 -->\n",
" <g transform=\"translate(7.2 225.446906)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_7\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m0efda133e9\" y=\"175.72747\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- −2 -->\n",
" <g transform=\"translate(7.2 179.526689)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m0efda133e9\" y=\"129.807254\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(15.579688 133.606472)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_9\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m0efda133e9\" y=\"83.887037\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(15.579688 87.686255)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m0efda133e9\" y=\"37.96682\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(15.579688 41.766039)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 28.942188 239.758125 \n",
"L 28.942188 22.318125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 363.742188 239.758125 \n",
"L 363.742188 22.318125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 28.942187 239.758125 \n",
"L 363.742188 239.758125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 28.942187 22.318125 \n",
"L 363.742188 22.318125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- Routing Number: 1 -->\n",
" <defs>\n",
" <path d=\"M 44.390625 34.1875 \n",
"Q 47.5625 33.109375 50.5625 29.59375 \n",
"Q 53.5625 26.078125 56.59375 19.921875 \n",
"L 66.609375 0 \n",
"L 56 0 \n",
"L 46.6875 18.703125 \n",
"Q 43.0625 26.03125 39.671875 28.421875 \n",
"Q 36.28125 30.8125 30.421875 30.8125 \n",
"L 19.671875 30.8125 \n",
"L 19.671875 0 \n",
"L 9.8125 0 \n",
"L 9.8125 72.90625 \n",
"L 32.078125 72.90625 \n",
"Q 44.578125 72.90625 50.734375 67.671875 \n",
"Q 56.890625 62.453125 56.890625 51.90625 \n",
"Q 56.890625 45.015625 53.6875 40.46875 \n",
"Q 50.484375 35.9375 44.390625 34.1875 \n",
"z\n",
"M 19.671875 64.796875 \n",
"L 19.671875 38.921875 \n",
"L 32.078125 38.921875 \n",
"Q 39.203125 38.921875 42.84375 42.21875 \n",
"Q 46.484375 45.515625 46.484375 51.90625 \n",
"Q 46.484375 58.296875 42.84375 61.546875 \n",
"Q 39.203125 64.796875 32.078125 64.796875 \n",
"z\n",
"\" id=\"DejaVuSans-52\"/>\n",
" <path d=\"M 30.609375 48.390625 \n",
"Q 23.390625 48.390625 19.1875 42.75 \n",
"Q 14.984375 37.109375 14.984375 27.296875 \n",
"Q 14.984375 17.484375 19.15625 11.84375 \n",
"Q 23.34375 6.203125 30.609375 6.203125 \n",
"Q 37.796875 6.203125 41.984375 11.859375 \n",
"Q 46.1875 17.53125 46.1875 27.296875 \n",
"Q 46.1875 37.015625 41.984375 42.703125 \n",
"Q 37.796875 48.390625 30.609375 48.390625 \n",
"z\n",
"M 30.609375 56 \n",
"Q 42.328125 56 49.015625 48.375 \n",
"Q 55.71875 40.765625 55.71875 27.296875 \n",
"Q 55.71875 13.875 49.015625 6.21875 \n",
"Q 42.328125 -1.421875 30.609375 -1.421875 \n",
"Q 18.84375 -1.421875 12.171875 6.21875 \n",
"Q 5.515625 13.875 5.515625 27.296875 \n",
"Q 5.515625 40.765625 12.171875 48.375 \n",
"Q 18.84375 56 30.609375 56 \n",
"z\n",
"\" id=\"DejaVuSans-6f\"/>\n",
" <path d=\"M 8.5 21.578125 \n",
"L 8.5 54.6875 \n",
"L 17.484375 54.6875 \n",
"L 17.484375 21.921875 \n",
"Q 17.484375 14.15625 20.5 10.265625 \n",
"Q 23.53125 6.390625 29.59375 6.390625 \n",
"Q 36.859375 6.390625 41.078125 11.03125 \n",
"Q 45.3125 15.671875 45.3125 23.6875 \n",
"L 45.3125 54.6875 \n",
"L 54.296875 54.6875 \n",
"L 54.296875 0 \n",
"L 45.3125 0 \n",
"L 45.3125 8.40625 \n",
"Q 42.046875 3.421875 37.71875 1 \n",
"Q 33.40625 -1.421875 27.6875 -1.421875 \n",
"Q 18.265625 -1.421875 13.375 4.4375 \n",
"Q 8.5 10.296875 8.5 21.578125 \n",
"z\n",
"M 31.109375 56 \n",
"z\n",
"\" id=\"DejaVuSans-75\"/>\n",
" <path d=\"M 18.3125 70.21875 \n",
"L 18.3125 54.6875 \n",
"L 36.8125 54.6875 \n",
"L 36.8125 47.703125 \n",
"L 18.3125 47.703125 \n",
"L 18.3125 18.015625 \n",
"Q 18.3125 11.328125 20.140625 9.421875 \n",
"Q 21.96875 7.515625 27.59375 7.515625 \n",
"L 36.8125 7.515625 \n",
"L 36.8125 0 \n",
"L 27.59375 0 \n",
"Q 17.1875 0 13.234375 3.875 \n",
"Q 9.28125 7.765625 9.28125 18.015625 \n",
"L 9.28125 47.703125 \n",
"L 2.6875 47.703125 \n",
"L 2.6875 54.6875 \n",
"L 9.28125 54.6875 \n",
"L 9.28125 70.21875 \n",
"z\n",
"\" id=\"DejaVuSans-74\"/>\n",
" <path d=\"M 9.421875 54.6875 \n",
"L 18.40625 54.6875 \n",
"L 18.40625 0 \n",
"L 9.421875 0 \n",
"z\n",
"M 9.421875 75.984375 \n",
"L 18.40625 75.984375 \n",
"L 18.40625 64.59375 \n",
"L 9.421875 64.59375 \n",
"z\n",
"\" id=\"DejaVuSans-69\"/>\n",
" <path d=\"M 54.890625 33.015625 \n",
"L 54.890625 0 \n",
"L 45.90625 0 \n",
"L 45.90625 32.71875 \n",
"Q 45.90625 40.484375 42.875 44.328125 \n",
"Q 39.84375 48.1875 33.796875 48.1875 \n",
"Q 26.515625 48.1875 22.3125 43.546875 \n",
"Q 18.109375 38.921875 18.109375 30.90625 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 21.34375 51.125 25.703125 53.5625 \n",
"Q 30.078125 56 35.796875 56 \n",
"Q 45.21875 56 50.046875 50.171875 \n",
"Q 54.890625 44.34375 54.890625 33.015625 \n",
"z\n",
"\" id=\"DejaVuSans-6e\"/>\n",
" <path d=\"M 45.40625 27.984375 \n",
"Q 45.40625 37.75 41.375 43.109375 \n",
"Q 37.359375 48.484375 30.078125 48.484375 \n",
"Q 22.859375 48.484375 18.828125 43.109375 \n",
"Q 14.796875 37.75 14.796875 27.984375 \n",
"Q 14.796875 18.265625 18.828125 12.890625 \n",
"Q 22.859375 7.515625 30.078125 7.515625 \n",
"Q 37.359375 7.515625 41.375 12.890625 \n",
"Q 45.40625 18.265625 45.40625 27.984375 \n",
"z\n",
"M 54.390625 6.78125 \n",
"Q 54.390625 -7.171875 48.1875 -13.984375 \n",
"Q 42 -20.796875 29.203125 -20.796875 \n",
"Q 24.46875 -20.796875 20.265625 -20.09375 \n",
"Q 16.0625 -19.390625 12.109375 -17.921875 \n",
"L 12.109375 -9.1875 \n",
"Q 16.0625 -11.328125 19.921875 -12.34375 \n",
"Q 23.78125 -13.375 27.78125 -13.375 \n",
"Q 36.625 -13.375 41.015625 -8.765625 \n",
"Q 45.40625 -4.15625 45.40625 5.171875 \n",
"L 45.40625 9.625 \n",
"Q 42.625 4.78125 38.28125 2.390625 \n",
"Q 33.9375 0 27.875 0 \n",
"Q 17.828125 0 11.671875 7.65625 \n",
"Q 5.515625 15.328125 5.515625 27.984375 \n",
"Q 5.515625 40.671875 11.671875 48.328125 \n",
"Q 17.828125 56 27.875 56 \n",
"Q 33.9375 56 38.28125 53.609375 \n",
"Q 42.625 51.21875 45.40625 46.390625 \n",
"L 45.40625 54.6875 \n",
"L 54.390625 54.6875 \n",
"z\n",
"\" id=\"DejaVuSans-67\"/>\n",
" <path id=\"DejaVuSans-20\"/>\n",
" <path d=\"M 9.8125 72.90625 \n",
"L 23.09375 72.90625 \n",
"L 55.421875 11.921875 \n",
"L 55.421875 72.90625 \n",
"L 64.984375 72.90625 \n",
"L 64.984375 0 \n",
"L 51.703125 0 \n",
"L 19.390625 60.984375 \n",
"L 19.390625 0 \n",
"L 9.8125 0 \n",
"z\n",
"\" id=\"DejaVuSans-4e\"/>\n",
" <path d=\"M 52 44.1875 \n",
"Q 55.375 50.25 60.0625 53.125 \n",
"Q 64.75 56 71.09375 56 \n",
"Q 79.640625 56 84.28125 50.015625 \n",
"Q 88.921875 44.046875 88.921875 33.015625 \n",
"L 88.921875 0 \n",
"L 79.890625 0 \n",
"L 79.890625 32.71875 \n",
"Q 79.890625 40.578125 77.09375 44.375 \n",
"Q 74.3125 48.1875 68.609375 48.1875 \n",
"Q 61.625 48.1875 57.5625 43.546875 \n",
"Q 53.515625 38.921875 53.515625 30.90625 \n",
"L 53.515625 0 \n",
"L 44.484375 0 \n",
"L 44.484375 32.71875 \n",
"Q 44.484375 40.625 41.703125 44.40625 \n",
"Q 38.921875 48.1875 33.109375 48.1875 \n",
"Q 26.21875 48.1875 22.15625 43.53125 \n",
"Q 18.109375 38.875 18.109375 30.90625 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 21.1875 51.21875 25.484375 53.609375 \n",
"Q 29.78125 56 35.6875 56 \n",
"Q 41.65625 56 45.828125 52.96875 \n",
"Q 50 49.953125 52 44.1875 \n",
"z\n",
"\" id=\"DejaVuSans-6d\"/>\n",
" <path d=\"M 48.6875 27.296875 \n",
"Q 48.6875 37.203125 44.609375 42.84375 \n",
"Q 40.53125 48.484375 33.40625 48.484375 \n",
"Q 26.265625 48.484375 22.1875 42.84375 \n",
"Q 18.109375 37.203125 18.109375 27.296875 \n",
"Q 18.109375 17.390625 22.1875 11.75 \n",
"Q 26.265625 6.109375 33.40625 6.109375 \n",
"Q 40.53125 6.109375 44.609375 11.75 \n",
"Q 48.6875 17.390625 48.6875 27.296875 \n",
"z\n",
"M 18.109375 46.390625 \n",
"Q 20.953125 51.265625 25.265625 53.625 \n",
"Q 29.59375 56 35.59375 56 \n",
"Q 45.5625 56 51.78125 48.09375 \n",
"Q 58.015625 40.1875 58.015625 27.296875 \n",
"Q 58.015625 14.40625 51.78125 6.484375 \n",
"Q 45.5625 -1.421875 35.59375 -1.421875 \n",
"Q 29.59375 -1.421875 25.265625 0.953125 \n",
"Q 20.953125 3.328125 18.109375 8.203125 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 75.984375 \n",
"L 18.109375 75.984375 \n",
"z\n",
"\" id=\"DejaVuSans-62\"/>\n",
" <path d=\"M 56.203125 29.59375 \n",
"L 56.203125 25.203125 \n",
"L 14.890625 25.203125 \n",
"Q 15.484375 15.921875 20.484375 11.0625 \n",
"Q 25.484375 6.203125 34.421875 6.203125 \n",
"Q 39.59375 6.203125 44.453125 7.46875 \n",
"Q 49.3125 8.734375 54.109375 11.28125 \n",
"L 54.109375 2.78125 \n",
"Q 49.265625 0.734375 44.1875 -0.34375 \n",
"Q 39.109375 -1.421875 33.890625 -1.421875 \n",
"Q 20.796875 -1.421875 13.15625 6.1875 \n",
"Q 5.515625 13.8125 5.515625 26.8125 \n",
"Q 5.515625 40.234375 12.765625 48.109375 \n",
"Q 20.015625 56 32.328125 56 \n",
"Q 43.359375 56 49.78125 48.890625 \n",
"Q 56.203125 41.796875 56.203125 29.59375 \n",
"z\n",
"M 47.21875 32.234375 \n",
"Q 47.125 39.59375 43.09375 43.984375 \n",
"Q 39.0625 48.390625 32.421875 48.390625 \n",
"Q 24.90625 48.390625 20.390625 44.140625 \n",
"Q 15.875 39.890625 15.1875 32.171875 \n",
"z\n",
"\" id=\"DejaVuSans-65\"/>\n",
" <path d=\"M 41.109375 46.296875 \n",
"Q 39.59375 47.171875 37.8125 47.578125 \n",
"Q 36.03125 48 33.890625 48 \n",
"Q 26.265625 48 22.1875 43.046875 \n",
"Q 18.109375 38.09375 18.109375 28.8125 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 20.953125 51.171875 25.484375 53.578125 \n",
"Q 30.03125 56 36.53125 56 \n",
"Q 37.453125 56 38.578125 55.875 \n",
"Q 39.703125 55.765625 41.0625 55.515625 \n",
"z\n",
"\" id=\"DejaVuSans-72\"/>\n",
" <path d=\"M 11.71875 12.40625 \n",
"L 22.015625 12.40625 \n",
"L 22.015625 0 \n",
"L 11.71875 0 \n",
"z\n",
"M 11.71875 51.703125 \n",
"L 22.015625 51.703125 \n",
"L 22.015625 39.3125 \n",
"L 11.71875 39.3125 \n",
"z\n",
"\" id=\"DejaVuSans-3a\"/>\n",
" <path d=\"M 12.40625 8.296875 \n",
"L 28.515625 8.296875 \n",
"L 28.515625 63.921875 \n",
"L 10.984375 60.40625 \n",
"L 10.984375 69.390625 \n",
"L 28.421875 72.90625 \n",
"L 38.28125 72.90625 \n",
"L 38.28125 8.296875 \n",
"L 54.390625 8.296875 \n",
"L 54.390625 0 \n",
"L 12.40625 0 \n",
"z\n",
"\" id=\"DejaVuSans-31\"/>\n",
" </defs>\n",
" <g transform=\"translate(139.317813 16.318125)scale(0.12 -0.12)\">\n",
" <use xlink:href=\"#DejaVuSans-52\"/>\n",
" <use x=\"69.419922\" xlink:href=\"#DejaVuSans-6f\"/>\n",
" <use x=\"130.601562\" xlink:href=\"#DejaVuSans-75\"/>\n",
" <use x=\"193.980469\" xlink:href=\"#DejaVuSans-74\"/>\n",
" <use x=\"233.189453\" xlink:href=\"#DejaVuSans-69\"/>\n",
" <use x=\"260.972656\" xlink:href=\"#DejaVuSans-6e\"/>\n",
" <use x=\"324.351562\" xlink:href=\"#DejaVuSans-67\"/>\n",
" <use x=\"387.828125\" xlink:href=\"#DejaVuSans-20\"/>\n",
" <use x=\"419.615234\" xlink:href=\"#DejaVuSans-4e\"/>\n",
" <use x=\"494.419922\" xlink:href=\"#DejaVuSans-75\"/>\n",
" <use x=\"557.798828\" xlink:href=\"#DejaVuSans-6d\"/>\n",
" <use x=\"655.210938\" xlink:href=\"#DejaVuSans-62\"/>\n",
" <use x=\"718.6875\" xlink:href=\"#DejaVuSans-65\"/>\n",
" <use x=\"780.210938\" xlink:href=\"#DejaVuSans-72\"/>\n",
" <use x=\"821.308594\" xlink:href=\"#DejaVuSans-3a\"/>\n",
" <use x=\"855\" xlink:href=\"#DejaVuSans-20\"/>\n",
" <use x=\"886.787109\" xlink:href=\"#DejaVuSans-31\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"pb261dca43e\">\n",
" <rect height=\"217.44\" width=\"334.8\" x=\"28.942188\" y=\"22.318125\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Created with matplotlib (http://matplotlib.org/) -->\n",
"<svg height=\"263.63625pt\" version=\"1.1\" viewBox=\"0 0 374.442187 263.63625\" width=\"374.442187pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
" <defs>\n",
" <style type=\"text/css\">\n",
"*{stroke-linecap:butt;stroke-linejoin:round;}\n",
" </style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 263.63625 \n",
"L 374.442187 263.63625 \n",
"L 374.442187 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill:none;\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 28.942188 239.758125 \n",
"L 363.742188 239.758125 \n",
"L 363.742188 22.318125 \n",
"L 28.942188 22.318125 \n",
"z\n",
"\" style=\"fill:#ffffff;\"/>\n",
" </g>\n",
" <g id=\"PathCollection_1\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"m96a9154b71\" style=\"stroke:#0000ff;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#pab9ca7c542)\">\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"211.249207\" xlink:href=\"#m96a9154b71\" y=\"111.341922\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"203.137797\" xlink:href=\"#m96a9154b71\" y=\"151.423386\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_2\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"m0f1303c5f4\" style=\"stroke:#ff0000;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#pab9ca7c542)\">\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"211.205687\" xlink:href=\"#m0f1303c5f4\" y=\"229.256573\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"348.438393\" xlink:href=\"#m0f1303c5f4\" y=\"115.159856\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"128.123006\" xlink:href=\"#m0f1303c5f4\" y=\"181.219449\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"191.524786\" xlink:href=\"#m0f1303c5f4\" y=\"222.201442\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"44.245982\" xlink:href=\"#m0f1303c5f4\" y=\"128.811159\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"213.850622\" xlink:href=\"#m0f1303c5f4\" y=\"34.017126\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"238.152835\" xlink:href=\"#m0f1303c5f4\" y=\"33.171824\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"200.08932\" xlink:href=\"#m0f1303c5f4\" y=\"211.907423\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"267.921674\" xlink:href=\"#m0f1303c5f4\" y=\"32.819677\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"313.149942\" xlink:href=\"#m0f1303c5f4\" y=\"76.21771\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" id=\"m3571746d55\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"64.2806\" xlink:href=\"#m3571746d55\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- −4 -->\n",
" <defs>\n",
" <path d=\"M 10.59375 35.5 \n",
"L 73.1875 35.5 \n",
"L 73.1875 27.203125 \n",
"L 10.59375 27.203125 \n",
"z\n",
"\" id=\"DejaVuSans-2212\"/>\n",
" <path d=\"M 37.796875 64.3125 \n",
"L 12.890625 25.390625 \n",
"L 37.796875 25.390625 \n",
"z\n",
"M 35.203125 72.90625 \n",
"L 47.609375 72.90625 \n",
"L 47.609375 25.390625 \n",
"L 58.015625 25.390625 \n",
"L 58.015625 17.1875 \n",
"L 47.609375 17.1875 \n",
"L 47.609375 0 \n",
"L 37.796875 0 \n",
"L 37.796875 17.1875 \n",
"L 4.890625 17.1875 \n",
"L 4.890625 26.703125 \n",
"z\n",
"\" id=\"DejaVuSans-34\"/>\n",
" </defs>\n",
" <g transform=\"translate(56.909507 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_2\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"128.387416\" xlink:href=\"#m3571746d55\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- −2 -->\n",
" <defs>\n",
" <path d=\"M 19.1875 8.296875 \n",
"L 53.609375 8.296875 \n",
"L 53.609375 0 \n",
"L 7.328125 0 \n",
"L 7.328125 8.296875 \n",
"Q 12.9375 14.109375 22.625 23.890625 \n",
"Q 32.328125 33.6875 34.8125 36.53125 \n",
"Q 39.546875 41.84375 41.421875 45.53125 \n",
"Q 43.3125 49.21875 43.3125 52.78125 \n",
"Q 43.3125 58.59375 39.234375 62.25 \n",
"Q 35.15625 65.921875 28.609375 65.921875 \n",
"Q 23.96875 65.921875 18.8125 64.3125 \n",
"Q 13.671875 62.703125 7.8125 59.421875 \n",
"L 7.8125 69.390625 \n",
"Q 13.765625 71.78125 18.9375 73 \n",
"Q 24.125 74.21875 28.421875 74.21875 \n",
"Q 39.75 74.21875 46.484375 68.546875 \n",
"Q 53.21875 62.890625 53.21875 53.421875 \n",
"Q 53.21875 48.921875 51.53125 44.890625 \n",
"Q 49.859375 40.875 45.40625 35.40625 \n",
"Q 44.1875 33.984375 37.640625 27.21875 \n",
"Q 31.109375 20.453125 19.1875 8.296875 \n",
"z\n",
"\" id=\"DejaVuSans-32\"/>\n",
" </defs>\n",
" <g transform=\"translate(121.016322 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_3\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"192.494231\" xlink:href=\"#m3571746d55\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 0 -->\n",
" <defs>\n",
" <path d=\"M 31.78125 66.40625 \n",
"Q 24.171875 66.40625 20.328125 58.90625 \n",
"Q 16.5 51.421875 16.5 36.375 \n",
"Q 16.5 21.390625 20.328125 13.890625 \n",
"Q 24.171875 6.390625 31.78125 6.390625 \n",
"Q 39.453125 6.390625 43.28125 13.890625 \n",
"Q 47.125 21.390625 47.125 36.375 \n",
"Q 47.125 51.421875 43.28125 58.90625 \n",
"Q 39.453125 66.40625 31.78125 66.40625 \n",
"z\n",
"M 31.78125 74.21875 \n",
"Q 44.046875 74.21875 50.515625 64.515625 \n",
"Q 56.984375 54.828125 56.984375 36.375 \n",
"Q 56.984375 17.96875 50.515625 8.265625 \n",
"Q 44.046875 -1.421875 31.78125 -1.421875 \n",
"Q 19.53125 -1.421875 13.0625 8.265625 \n",
"Q 6.59375 17.96875 6.59375 36.375 \n",
"Q 6.59375 54.828125 13.0625 64.515625 \n",
"Q 19.53125 74.21875 31.78125 74.21875 \n",
"z\n",
"\" id=\"DejaVuSans-30\"/>\n",
" </defs>\n",
" <g transform=\"translate(189.312981 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"256.601046\" xlink:href=\"#m3571746d55\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(253.419796 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_5\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"320.707862\" xlink:href=\"#m3571746d55\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(317.526612 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_6\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" id=\"md87533612d\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#md87533612d\" y=\"221.644199\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- −4 -->\n",
" <g transform=\"translate(7.2 225.443418)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_7\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#md87533612d\" y=\"175.72575\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- −2 -->\n",
" <g transform=\"translate(7.2 179.524969)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#md87533612d\" y=\"129.807301\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(15.579688 133.60652)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_9\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#md87533612d\" y=\"83.888852\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(15.579688 87.688071)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#md87533612d\" y=\"37.970403\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(15.579688 41.769622)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 28.942188 239.758125 \n",
"L 28.942188 22.318125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 363.742188 239.758125 \n",
"L 363.742188 22.318125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 28.942187 239.758125 \n",
"L 363.742188 239.758125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 28.942187 22.318125 \n",
"L 363.742188 22.318125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- Routing Number: 3 -->\n",
" <defs>\n",
" <path d=\"M 44.390625 34.1875 \n",
"Q 47.5625 33.109375 50.5625 29.59375 \n",
"Q 53.5625 26.078125 56.59375 19.921875 \n",
"L 66.609375 0 \n",
"L 56 0 \n",
"L 46.6875 18.703125 \n",
"Q 43.0625 26.03125 39.671875 28.421875 \n",
"Q 36.28125 30.8125 30.421875 30.8125 \n",
"L 19.671875 30.8125 \n",
"L 19.671875 0 \n",
"L 9.8125 0 \n",
"L 9.8125 72.90625 \n",
"L 32.078125 72.90625 \n",
"Q 44.578125 72.90625 50.734375 67.671875 \n",
"Q 56.890625 62.453125 56.890625 51.90625 \n",
"Q 56.890625 45.015625 53.6875 40.46875 \n",
"Q 50.484375 35.9375 44.390625 34.1875 \n",
"z\n",
"M 19.671875 64.796875 \n",
"L 19.671875 38.921875 \n",
"L 32.078125 38.921875 \n",
"Q 39.203125 38.921875 42.84375 42.21875 \n",
"Q 46.484375 45.515625 46.484375 51.90625 \n",
"Q 46.484375 58.296875 42.84375 61.546875 \n",
"Q 39.203125 64.796875 32.078125 64.796875 \n",
"z\n",
"\" id=\"DejaVuSans-52\"/>\n",
" <path d=\"M 30.609375 48.390625 \n",
"Q 23.390625 48.390625 19.1875 42.75 \n",
"Q 14.984375 37.109375 14.984375 27.296875 \n",
"Q 14.984375 17.484375 19.15625 11.84375 \n",
"Q 23.34375 6.203125 30.609375 6.203125 \n",
"Q 37.796875 6.203125 41.984375 11.859375 \n",
"Q 46.1875 17.53125 46.1875 27.296875 \n",
"Q 46.1875 37.015625 41.984375 42.703125 \n",
"Q 37.796875 48.390625 30.609375 48.390625 \n",
"z\n",
"M 30.609375 56 \n",
"Q 42.328125 56 49.015625 48.375 \n",
"Q 55.71875 40.765625 55.71875 27.296875 \n",
"Q 55.71875 13.875 49.015625 6.21875 \n",
"Q 42.328125 -1.421875 30.609375 -1.421875 \n",
"Q 18.84375 -1.421875 12.171875 6.21875 \n",
"Q 5.515625 13.875 5.515625 27.296875 \n",
"Q 5.515625 40.765625 12.171875 48.375 \n",
"Q 18.84375 56 30.609375 56 \n",
"z\n",
"\" id=\"DejaVuSans-6f\"/>\n",
" <path d=\"M 8.5 21.578125 \n",
"L 8.5 54.6875 \n",
"L 17.484375 54.6875 \n",
"L 17.484375 21.921875 \n",
"Q 17.484375 14.15625 20.5 10.265625 \n",
"Q 23.53125 6.390625 29.59375 6.390625 \n",
"Q 36.859375 6.390625 41.078125 11.03125 \n",
"Q 45.3125 15.671875 45.3125 23.6875 \n",
"L 45.3125 54.6875 \n",
"L 54.296875 54.6875 \n",
"L 54.296875 0 \n",
"L 45.3125 0 \n",
"L 45.3125 8.40625 \n",
"Q 42.046875 3.421875 37.71875 1 \n",
"Q 33.40625 -1.421875 27.6875 -1.421875 \n",
"Q 18.265625 -1.421875 13.375 4.4375 \n",
"Q 8.5 10.296875 8.5 21.578125 \n",
"z\n",
"M 31.109375 56 \n",
"z\n",
"\" id=\"DejaVuSans-75\"/>\n",
" <path d=\"M 18.3125 70.21875 \n",
"L 18.3125 54.6875 \n",
"L 36.8125 54.6875 \n",
"L 36.8125 47.703125 \n",
"L 18.3125 47.703125 \n",
"L 18.3125 18.015625 \n",
"Q 18.3125 11.328125 20.140625 9.421875 \n",
"Q 21.96875 7.515625 27.59375 7.515625 \n",
"L 36.8125 7.515625 \n",
"L 36.8125 0 \n",
"L 27.59375 0 \n",
"Q 17.1875 0 13.234375 3.875 \n",
"Q 9.28125 7.765625 9.28125 18.015625 \n",
"L 9.28125 47.703125 \n",
"L 2.6875 47.703125 \n",
"L 2.6875 54.6875 \n",
"L 9.28125 54.6875 \n",
"L 9.28125 70.21875 \n",
"z\n",
"\" id=\"DejaVuSans-74\"/>\n",
" <path d=\"M 9.421875 54.6875 \n",
"L 18.40625 54.6875 \n",
"L 18.40625 0 \n",
"L 9.421875 0 \n",
"z\n",
"M 9.421875 75.984375 \n",
"L 18.40625 75.984375 \n",
"L 18.40625 64.59375 \n",
"L 9.421875 64.59375 \n",
"z\n",
"\" id=\"DejaVuSans-69\"/>\n",
" <path d=\"M 54.890625 33.015625 \n",
"L 54.890625 0 \n",
"L 45.90625 0 \n",
"L 45.90625 32.71875 \n",
"Q 45.90625 40.484375 42.875 44.328125 \n",
"Q 39.84375 48.1875 33.796875 48.1875 \n",
"Q 26.515625 48.1875 22.3125 43.546875 \n",
"Q 18.109375 38.921875 18.109375 30.90625 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 21.34375 51.125 25.703125 53.5625 \n",
"Q 30.078125 56 35.796875 56 \n",
"Q 45.21875 56 50.046875 50.171875 \n",
"Q 54.890625 44.34375 54.890625 33.015625 \n",
"z\n",
"\" id=\"DejaVuSans-6e\"/>\n",
" <path d=\"M 45.40625 27.984375 \n",
"Q 45.40625 37.75 41.375 43.109375 \n",
"Q 37.359375 48.484375 30.078125 48.484375 \n",
"Q 22.859375 48.484375 18.828125 43.109375 \n",
"Q 14.796875 37.75 14.796875 27.984375 \n",
"Q 14.796875 18.265625 18.828125 12.890625 \n",
"Q 22.859375 7.515625 30.078125 7.515625 \n",
"Q 37.359375 7.515625 41.375 12.890625 \n",
"Q 45.40625 18.265625 45.40625 27.984375 \n",
"z\n",
"M 54.390625 6.78125 \n",
"Q 54.390625 -7.171875 48.1875 -13.984375 \n",
"Q 42 -20.796875 29.203125 -20.796875 \n",
"Q 24.46875 -20.796875 20.265625 -20.09375 \n",
"Q 16.0625 -19.390625 12.109375 -17.921875 \n",
"L 12.109375 -9.1875 \n",
"Q 16.0625 -11.328125 19.921875 -12.34375 \n",
"Q 23.78125 -13.375 27.78125 -13.375 \n",
"Q 36.625 -13.375 41.015625 -8.765625 \n",
"Q 45.40625 -4.15625 45.40625 5.171875 \n",
"L 45.40625 9.625 \n",
"Q 42.625 4.78125 38.28125 2.390625 \n",
"Q 33.9375 0 27.875 0 \n",
"Q 17.828125 0 11.671875 7.65625 \n",
"Q 5.515625 15.328125 5.515625 27.984375 \n",
"Q 5.515625 40.671875 11.671875 48.328125 \n",
"Q 17.828125 56 27.875 56 \n",
"Q 33.9375 56 38.28125 53.609375 \n",
"Q 42.625 51.21875 45.40625 46.390625 \n",
"L 45.40625 54.6875 \n",
"L 54.390625 54.6875 \n",
"z\n",
"\" id=\"DejaVuSans-67\"/>\n",
" <path id=\"DejaVuSans-20\"/>\n",
" <path d=\"M 9.8125 72.90625 \n",
"L 23.09375 72.90625 \n",
"L 55.421875 11.921875 \n",
"L 55.421875 72.90625 \n",
"L 64.984375 72.90625 \n",
"L 64.984375 0 \n",
"L 51.703125 0 \n",
"L 19.390625 60.984375 \n",
"L 19.390625 0 \n",
"L 9.8125 0 \n",
"z\n",
"\" id=\"DejaVuSans-4e\"/>\n",
" <path d=\"M 52 44.1875 \n",
"Q 55.375 50.25 60.0625 53.125 \n",
"Q 64.75 56 71.09375 56 \n",
"Q 79.640625 56 84.28125 50.015625 \n",
"Q 88.921875 44.046875 88.921875 33.015625 \n",
"L 88.921875 0 \n",
"L 79.890625 0 \n",
"L 79.890625 32.71875 \n",
"Q 79.890625 40.578125 77.09375 44.375 \n",
"Q 74.3125 48.1875 68.609375 48.1875 \n",
"Q 61.625 48.1875 57.5625 43.546875 \n",
"Q 53.515625 38.921875 53.515625 30.90625 \n",
"L 53.515625 0 \n",
"L 44.484375 0 \n",
"L 44.484375 32.71875 \n",
"Q 44.484375 40.625 41.703125 44.40625 \n",
"Q 38.921875 48.1875 33.109375 48.1875 \n",
"Q 26.21875 48.1875 22.15625 43.53125 \n",
"Q 18.109375 38.875 18.109375 30.90625 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 21.1875 51.21875 25.484375 53.609375 \n",
"Q 29.78125 56 35.6875 56 \n",
"Q 41.65625 56 45.828125 52.96875 \n",
"Q 50 49.953125 52 44.1875 \n",
"z\n",
"\" id=\"DejaVuSans-6d\"/>\n",
" <path d=\"M 48.6875 27.296875 \n",
"Q 48.6875 37.203125 44.609375 42.84375 \n",
"Q 40.53125 48.484375 33.40625 48.484375 \n",
"Q 26.265625 48.484375 22.1875 42.84375 \n",
"Q 18.109375 37.203125 18.109375 27.296875 \n",
"Q 18.109375 17.390625 22.1875 11.75 \n",
"Q 26.265625 6.109375 33.40625 6.109375 \n",
"Q 40.53125 6.109375 44.609375 11.75 \n",
"Q 48.6875 17.390625 48.6875 27.296875 \n",
"z\n",
"M 18.109375 46.390625 \n",
"Q 20.953125 51.265625 25.265625 53.625 \n",
"Q 29.59375 56 35.59375 56 \n",
"Q 45.5625 56 51.78125 48.09375 \n",
"Q 58.015625 40.1875 58.015625 27.296875 \n",
"Q 58.015625 14.40625 51.78125 6.484375 \n",
"Q 45.5625 -1.421875 35.59375 -1.421875 \n",
"Q 29.59375 -1.421875 25.265625 0.953125 \n",
"Q 20.953125 3.328125 18.109375 8.203125 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 75.984375 \n",
"L 18.109375 75.984375 \n",
"z\n",
"\" id=\"DejaVuSans-62\"/>\n",
" <path d=\"M 56.203125 29.59375 \n",
"L 56.203125 25.203125 \n",
"L 14.890625 25.203125 \n",
"Q 15.484375 15.921875 20.484375 11.0625 \n",
"Q 25.484375 6.203125 34.421875 6.203125 \n",
"Q 39.59375 6.203125 44.453125 7.46875 \n",
"Q 49.3125 8.734375 54.109375 11.28125 \n",
"L 54.109375 2.78125 \n",
"Q 49.265625 0.734375 44.1875 -0.34375 \n",
"Q 39.109375 -1.421875 33.890625 -1.421875 \n",
"Q 20.796875 -1.421875 13.15625 6.1875 \n",
"Q 5.515625 13.8125 5.515625 26.8125 \n",
"Q 5.515625 40.234375 12.765625 48.109375 \n",
"Q 20.015625 56 32.328125 56 \n",
"Q 43.359375 56 49.78125 48.890625 \n",
"Q 56.203125 41.796875 56.203125 29.59375 \n",
"z\n",
"M 47.21875 32.234375 \n",
"Q 47.125 39.59375 43.09375 43.984375 \n",
"Q 39.0625 48.390625 32.421875 48.390625 \n",
"Q 24.90625 48.390625 20.390625 44.140625 \n",
"Q 15.875 39.890625 15.1875 32.171875 \n",
"z\n",
"\" id=\"DejaVuSans-65\"/>\n",
" <path d=\"M 41.109375 46.296875 \n",
"Q 39.59375 47.171875 37.8125 47.578125 \n",
"Q 36.03125 48 33.890625 48 \n",
"Q 26.265625 48 22.1875 43.046875 \n",
"Q 18.109375 38.09375 18.109375 28.8125 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 20.953125 51.171875 25.484375 53.578125 \n",
"Q 30.03125 56 36.53125 56 \n",
"Q 37.453125 56 38.578125 55.875 \n",
"Q 39.703125 55.765625 41.0625 55.515625 \n",
"z\n",
"\" id=\"DejaVuSans-72\"/>\n",
" <path d=\"M 11.71875 12.40625 \n",
"L 22.015625 12.40625 \n",
"L 22.015625 0 \n",
"L 11.71875 0 \n",
"z\n",
"M 11.71875 51.703125 \n",
"L 22.015625 51.703125 \n",
"L 22.015625 39.3125 \n",
"L 11.71875 39.3125 \n",
"z\n",
"\" id=\"DejaVuSans-3a\"/>\n",
" <path d=\"M 40.578125 39.3125 \n",
"Q 47.65625 37.796875 51.625 33 \n",
"Q 55.609375 28.21875 55.609375 21.1875 \n",
"Q 55.609375 10.40625 48.1875 4.484375 \n",
"Q 40.765625 -1.421875 27.09375 -1.421875 \n",
"Q 22.515625 -1.421875 17.65625 -0.515625 \n",
"Q 12.796875 0.390625 7.625 2.203125 \n",
"L 7.625 11.71875 \n",
"Q 11.71875 9.328125 16.59375 8.109375 \n",
"Q 21.484375 6.890625 26.8125 6.890625 \n",
"Q 36.078125 6.890625 40.9375 10.546875 \n",
"Q 45.796875 14.203125 45.796875 21.1875 \n",
"Q 45.796875 27.640625 41.28125 31.265625 \n",
"Q 36.765625 34.90625 28.71875 34.90625 \n",
"L 20.21875 34.90625 \n",
"L 20.21875 43.015625 \n",
"L 29.109375 43.015625 \n",
"Q 36.375 43.015625 40.234375 45.921875 \n",
"Q 44.09375 48.828125 44.09375 54.296875 \n",
"Q 44.09375 59.90625 40.109375 62.90625 \n",
"Q 36.140625 65.921875 28.71875 65.921875 \n",
"Q 24.65625 65.921875 20.015625 65.03125 \n",
"Q 15.375 64.15625 9.8125 62.3125 \n",
"L 9.8125 71.09375 \n",
"Q 15.4375 72.65625 20.34375 73.4375 \n",
"Q 25.25 74.21875 29.59375 74.21875 \n",
"Q 40.828125 74.21875 47.359375 69.109375 \n",
"Q 53.90625 64.015625 53.90625 55.328125 \n",
"Q 53.90625 49.265625 50.4375 45.09375 \n",
"Q 46.96875 40.921875 40.578125 39.3125 \n",
"z\n",
"\" id=\"DejaVuSans-33\"/>\n",
" </defs>\n",
" <g transform=\"translate(139.317813 16.318125)scale(0.12 -0.12)\">\n",
" <use xlink:href=\"#DejaVuSans-52\"/>\n",
" <use x=\"69.419922\" xlink:href=\"#DejaVuSans-6f\"/>\n",
" <use x=\"130.601562\" xlink:href=\"#DejaVuSans-75\"/>\n",
" <use x=\"193.980469\" xlink:href=\"#DejaVuSans-74\"/>\n",
" <use x=\"233.189453\" xlink:href=\"#DejaVuSans-69\"/>\n",
" <use x=\"260.972656\" xlink:href=\"#DejaVuSans-6e\"/>\n",
" <use x=\"324.351562\" xlink:href=\"#DejaVuSans-67\"/>\n",
" <use x=\"387.828125\" xlink:href=\"#DejaVuSans-20\"/>\n",
" <use x=\"419.615234\" xlink:href=\"#DejaVuSans-4e\"/>\n",
" <use x=\"494.419922\" xlink:href=\"#DejaVuSans-75\"/>\n",
" <use x=\"557.798828\" xlink:href=\"#DejaVuSans-6d\"/>\n",
" <use x=\"655.210938\" xlink:href=\"#DejaVuSans-62\"/>\n",
" <use x=\"718.6875\" xlink:href=\"#DejaVuSans-65\"/>\n",
" <use x=\"780.210938\" xlink:href=\"#DejaVuSans-72\"/>\n",
" <use x=\"821.308594\" xlink:href=\"#DejaVuSans-3a\"/>\n",
" <use x=\"855\" xlink:href=\"#DejaVuSans-20\"/>\n",
" <use x=\"886.787109\" xlink:href=\"#DejaVuSans-33\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"pab9ca7c542\">\n",
" <rect height=\"217.44\" width=\"334.8\" x=\"28.942188\" y=\"22.318125\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Created with matplotlib (http://matplotlib.org/) -->\n",
"<svg height=\"263.63625pt\" version=\"1.1\" viewBox=\"0 0 374.442187 263.63625\" width=\"374.442187pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
" <defs>\n",
" <style type=\"text/css\">\n",
"*{stroke-linecap:butt;stroke-linejoin:round;}\n",
" </style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 263.63625 \n",
"L 374.442187 263.63625 \n",
"L 374.442187 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill:none;\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 28.942188 239.758125 \n",
"L 363.742188 239.758125 \n",
"L 363.742188 22.318125 \n",
"L 28.942188 22.318125 \n",
"z\n",
"\" style=\"fill:#ffffff;\"/>\n",
" </g>\n",
" <g id=\"PathCollection_1\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"m1fa6a5adb3\" style=\"stroke:#0000ff;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p16aab0a91b)\">\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"211.28696\" xlink:href=\"#m1fa6a5adb3\" y=\"111.361365\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"203.424703\" xlink:href=\"#m1fa6a5adb3\" y=\"151.350467\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_2\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"md3e1d8e9e6\" style=\"stroke:#ff0000;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p16aab0a91b)\">\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"211.205927\" xlink:href=\"#md3e1d8e9e6\" y=\"229.257975\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"348.440848\" xlink:href=\"#md3e1d8e9e6\" y=\"115.159629\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"128.121904\" xlink:href=\"#md3e1d8e9e6\" y=\"181.220165\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"191.524708\" xlink:href=\"#md3e1d8e9e6\" y=\"222.202743\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"44.243527\" xlink:href=\"#md3e1d8e9e6\" y=\"128.811127\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"213.850904\" xlink:href=\"#md3e1d8e9e6\" y=\"34.015742\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"238.15351\" xlink:href=\"#md3e1d8e9e6\" y=\"33.170427\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"200.08938\" xlink:href=\"#md3e1d8e9e6\" y=\"211.908577\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"267.92283\" xlink:href=\"#md3e1d8e9e6\" y=\"32.818275\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"313.151828\" xlink:href=\"#md3e1d8e9e6\" y=\"76.216927\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" id=\"mb6a25eb0a6\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"64.278468\" xlink:href=\"#mb6a25eb0a6\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- −4 -->\n",
" <defs>\n",
" <path d=\"M 10.59375 35.5 \n",
"L 73.1875 35.5 \n",
"L 73.1875 27.203125 \n",
"L 10.59375 27.203125 \n",
"z\n",
"\" id=\"DejaVuSans-2212\"/>\n",
" <path d=\"M 37.796875 64.3125 \n",
"L 12.890625 25.390625 \n",
"L 37.796875 25.390625 \n",
"z\n",
"M 35.203125 72.90625 \n",
"L 47.609375 72.90625 \n",
"L 47.609375 25.390625 \n",
"L 58.015625 25.390625 \n",
"L 58.015625 17.1875 \n",
"L 47.609375 17.1875 \n",
"L 47.609375 0 \n",
"L 37.796875 0 \n",
"L 37.796875 17.1875 \n",
"L 4.890625 17.1875 \n",
"L 4.890625 26.703125 \n",
"z\n",
"\" id=\"DejaVuSans-34\"/>\n",
" </defs>\n",
" <g transform=\"translate(56.907374 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_2\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"128.386318\" xlink:href=\"#mb6a25eb0a6\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- −2 -->\n",
" <defs>\n",
" <path d=\"M 19.1875 8.296875 \n",
"L 53.609375 8.296875 \n",
"L 53.609375 0 \n",
"L 7.328125 0 \n",
"L 7.328125 8.296875 \n",
"Q 12.9375 14.109375 22.625 23.890625 \n",
"Q 32.328125 33.6875 34.8125 36.53125 \n",
"Q 39.546875 41.84375 41.421875 45.53125 \n",
"Q 43.3125 49.21875 43.3125 52.78125 \n",
"Q 43.3125 58.59375 39.234375 62.25 \n",
"Q 35.15625 65.921875 28.609375 65.921875 \n",
"Q 23.96875 65.921875 18.8125 64.3125 \n",
"Q 13.671875 62.703125 7.8125 59.421875 \n",
"L 7.8125 69.390625 \n",
"Q 13.765625 71.78125 18.9375 73 \n",
"Q 24.125 74.21875 28.421875 74.21875 \n",
"Q 39.75 74.21875 46.484375 68.546875 \n",
"Q 53.21875 62.890625 53.21875 53.421875 \n",
"Q 53.21875 48.921875 51.53125 44.890625 \n",
"Q 49.859375 40.875 45.40625 35.40625 \n",
"Q 44.1875 33.984375 37.640625 27.21875 \n",
"Q 31.109375 20.453125 19.1875 8.296875 \n",
"z\n",
"\" id=\"DejaVuSans-32\"/>\n",
" </defs>\n",
" <g transform=\"translate(121.015225 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_3\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"192.494169\" xlink:href=\"#mb6a25eb0a6\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 0 -->\n",
" <defs>\n",
" <path d=\"M 31.78125 66.40625 \n",
"Q 24.171875 66.40625 20.328125 58.90625 \n",
"Q 16.5 51.421875 16.5 36.375 \n",
"Q 16.5 21.390625 20.328125 13.890625 \n",
"Q 24.171875 6.390625 31.78125 6.390625 \n",
"Q 39.453125 6.390625 43.28125 13.890625 \n",
"Q 47.125 21.390625 47.125 36.375 \n",
"Q 47.125 51.421875 43.28125 58.90625 \n",
"Q 39.453125 66.40625 31.78125 66.40625 \n",
"z\n",
"M 31.78125 74.21875 \n",
"Q 44.046875 74.21875 50.515625 64.515625 \n",
"Q 56.984375 54.828125 56.984375 36.375 \n",
"Q 56.984375 17.96875 50.515625 8.265625 \n",
"Q 44.046875 -1.421875 31.78125 -1.421875 \n",
"Q 19.53125 -1.421875 13.0625 8.265625 \n",
"Q 6.59375 17.96875 6.59375 36.375 \n",
"Q 6.59375 54.828125 13.0625 64.515625 \n",
"Q 19.53125 74.21875 31.78125 74.21875 \n",
"z\n",
"\" id=\"DejaVuSans-30\"/>\n",
" </defs>\n",
" <g transform=\"translate(189.312919 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"256.602019\" xlink:href=\"#mb6a25eb0a6\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(253.420769 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_5\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"320.70987\" xlink:href=\"#mb6a25eb0a6\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(317.52862 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_6\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" id=\"m11d0d864f3\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m11d0d864f3\" y=\"221.645492\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- −4 -->\n",
" <g transform=\"translate(7.2 225.444711)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_7\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m11d0d864f3\" y=\"175.726388\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- −2 -->\n",
" <g transform=\"translate(7.2 179.525606)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m11d0d864f3\" y=\"129.807283\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(15.579688 133.606502)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_9\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m11d0d864f3\" y=\"83.888179\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(15.579688 87.687398)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m11d0d864f3\" y=\"37.969075\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(15.579688 41.768294)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 28.942188 239.758125 \n",
"L 28.942188 22.318125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 363.742188 239.758125 \n",
"L 363.742188 22.318125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 28.942188 239.758125 \n",
"L 363.742188 239.758125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 28.942188 22.318125 \n",
"L 363.742188 22.318125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- Routing Number: 7 -->\n",
" <defs>\n",
" <path d=\"M 44.390625 34.1875 \n",
"Q 47.5625 33.109375 50.5625 29.59375 \n",
"Q 53.5625 26.078125 56.59375 19.921875 \n",
"L 66.609375 0 \n",
"L 56 0 \n",
"L 46.6875 18.703125 \n",
"Q 43.0625 26.03125 39.671875 28.421875 \n",
"Q 36.28125 30.8125 30.421875 30.8125 \n",
"L 19.671875 30.8125 \n",
"L 19.671875 0 \n",
"L 9.8125 0 \n",
"L 9.8125 72.90625 \n",
"L 32.078125 72.90625 \n",
"Q 44.578125 72.90625 50.734375 67.671875 \n",
"Q 56.890625 62.453125 56.890625 51.90625 \n",
"Q 56.890625 45.015625 53.6875 40.46875 \n",
"Q 50.484375 35.9375 44.390625 34.1875 \n",
"z\n",
"M 19.671875 64.796875 \n",
"L 19.671875 38.921875 \n",
"L 32.078125 38.921875 \n",
"Q 39.203125 38.921875 42.84375 42.21875 \n",
"Q 46.484375 45.515625 46.484375 51.90625 \n",
"Q 46.484375 58.296875 42.84375 61.546875 \n",
"Q 39.203125 64.796875 32.078125 64.796875 \n",
"z\n",
"\" id=\"DejaVuSans-52\"/>\n",
" <path d=\"M 30.609375 48.390625 \n",
"Q 23.390625 48.390625 19.1875 42.75 \n",
"Q 14.984375 37.109375 14.984375 27.296875 \n",
"Q 14.984375 17.484375 19.15625 11.84375 \n",
"Q 23.34375 6.203125 30.609375 6.203125 \n",
"Q 37.796875 6.203125 41.984375 11.859375 \n",
"Q 46.1875 17.53125 46.1875 27.296875 \n",
"Q 46.1875 37.015625 41.984375 42.703125 \n",
"Q 37.796875 48.390625 30.609375 48.390625 \n",
"z\n",
"M 30.609375 56 \n",
"Q 42.328125 56 49.015625 48.375 \n",
"Q 55.71875 40.765625 55.71875 27.296875 \n",
"Q 55.71875 13.875 49.015625 6.21875 \n",
"Q 42.328125 -1.421875 30.609375 -1.421875 \n",
"Q 18.84375 -1.421875 12.171875 6.21875 \n",
"Q 5.515625 13.875 5.515625 27.296875 \n",
"Q 5.515625 40.765625 12.171875 48.375 \n",
"Q 18.84375 56 30.609375 56 \n",
"z\n",
"\" id=\"DejaVuSans-6f\"/>\n",
" <path d=\"M 8.5 21.578125 \n",
"L 8.5 54.6875 \n",
"L 17.484375 54.6875 \n",
"L 17.484375 21.921875 \n",
"Q 17.484375 14.15625 20.5 10.265625 \n",
"Q 23.53125 6.390625 29.59375 6.390625 \n",
"Q 36.859375 6.390625 41.078125 11.03125 \n",
"Q 45.3125 15.671875 45.3125 23.6875 \n",
"L 45.3125 54.6875 \n",
"L 54.296875 54.6875 \n",
"L 54.296875 0 \n",
"L 45.3125 0 \n",
"L 45.3125 8.40625 \n",
"Q 42.046875 3.421875 37.71875 1 \n",
"Q 33.40625 -1.421875 27.6875 -1.421875 \n",
"Q 18.265625 -1.421875 13.375 4.4375 \n",
"Q 8.5 10.296875 8.5 21.578125 \n",
"z\n",
"M 31.109375 56 \n",
"z\n",
"\" id=\"DejaVuSans-75\"/>\n",
" <path d=\"M 18.3125 70.21875 \n",
"L 18.3125 54.6875 \n",
"L 36.8125 54.6875 \n",
"L 36.8125 47.703125 \n",
"L 18.3125 47.703125 \n",
"L 18.3125 18.015625 \n",
"Q 18.3125 11.328125 20.140625 9.421875 \n",
"Q 21.96875 7.515625 27.59375 7.515625 \n",
"L 36.8125 7.515625 \n",
"L 36.8125 0 \n",
"L 27.59375 0 \n",
"Q 17.1875 0 13.234375 3.875 \n",
"Q 9.28125 7.765625 9.28125 18.015625 \n",
"L 9.28125 47.703125 \n",
"L 2.6875 47.703125 \n",
"L 2.6875 54.6875 \n",
"L 9.28125 54.6875 \n",
"L 9.28125 70.21875 \n",
"z\n",
"\" id=\"DejaVuSans-74\"/>\n",
" <path d=\"M 9.421875 54.6875 \n",
"L 18.40625 54.6875 \n",
"L 18.40625 0 \n",
"L 9.421875 0 \n",
"z\n",
"M 9.421875 75.984375 \n",
"L 18.40625 75.984375 \n",
"L 18.40625 64.59375 \n",
"L 9.421875 64.59375 \n",
"z\n",
"\" id=\"DejaVuSans-69\"/>\n",
" <path d=\"M 54.890625 33.015625 \n",
"L 54.890625 0 \n",
"L 45.90625 0 \n",
"L 45.90625 32.71875 \n",
"Q 45.90625 40.484375 42.875 44.328125 \n",
"Q 39.84375 48.1875 33.796875 48.1875 \n",
"Q 26.515625 48.1875 22.3125 43.546875 \n",
"Q 18.109375 38.921875 18.109375 30.90625 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 21.34375 51.125 25.703125 53.5625 \n",
"Q 30.078125 56 35.796875 56 \n",
"Q 45.21875 56 50.046875 50.171875 \n",
"Q 54.890625 44.34375 54.890625 33.015625 \n",
"z\n",
"\" id=\"DejaVuSans-6e\"/>\n",
" <path d=\"M 45.40625 27.984375 \n",
"Q 45.40625 37.75 41.375 43.109375 \n",
"Q 37.359375 48.484375 30.078125 48.484375 \n",
"Q 22.859375 48.484375 18.828125 43.109375 \n",
"Q 14.796875 37.75 14.796875 27.984375 \n",
"Q 14.796875 18.265625 18.828125 12.890625 \n",
"Q 22.859375 7.515625 30.078125 7.515625 \n",
"Q 37.359375 7.515625 41.375 12.890625 \n",
"Q 45.40625 18.265625 45.40625 27.984375 \n",
"z\n",
"M 54.390625 6.78125 \n",
"Q 54.390625 -7.171875 48.1875 -13.984375 \n",
"Q 42 -20.796875 29.203125 -20.796875 \n",
"Q 24.46875 -20.796875 20.265625 -20.09375 \n",
"Q 16.0625 -19.390625 12.109375 -17.921875 \n",
"L 12.109375 -9.1875 \n",
"Q 16.0625 -11.328125 19.921875 -12.34375 \n",
"Q 23.78125 -13.375 27.78125 -13.375 \n",
"Q 36.625 -13.375 41.015625 -8.765625 \n",
"Q 45.40625 -4.15625 45.40625 5.171875 \n",
"L 45.40625 9.625 \n",
"Q 42.625 4.78125 38.28125 2.390625 \n",
"Q 33.9375 0 27.875 0 \n",
"Q 17.828125 0 11.671875 7.65625 \n",
"Q 5.515625 15.328125 5.515625 27.984375 \n",
"Q 5.515625 40.671875 11.671875 48.328125 \n",
"Q 17.828125 56 27.875 56 \n",
"Q 33.9375 56 38.28125 53.609375 \n",
"Q 42.625 51.21875 45.40625 46.390625 \n",
"L 45.40625 54.6875 \n",
"L 54.390625 54.6875 \n",
"z\n",
"\" id=\"DejaVuSans-67\"/>\n",
" <path id=\"DejaVuSans-20\"/>\n",
" <path d=\"M 9.8125 72.90625 \n",
"L 23.09375 72.90625 \n",
"L 55.421875 11.921875 \n",
"L 55.421875 72.90625 \n",
"L 64.984375 72.90625 \n",
"L 64.984375 0 \n",
"L 51.703125 0 \n",
"L 19.390625 60.984375 \n",
"L 19.390625 0 \n",
"L 9.8125 0 \n",
"z\n",
"\" id=\"DejaVuSans-4e\"/>\n",
" <path d=\"M 52 44.1875 \n",
"Q 55.375 50.25 60.0625 53.125 \n",
"Q 64.75 56 71.09375 56 \n",
"Q 79.640625 56 84.28125 50.015625 \n",
"Q 88.921875 44.046875 88.921875 33.015625 \n",
"L 88.921875 0 \n",
"L 79.890625 0 \n",
"L 79.890625 32.71875 \n",
"Q 79.890625 40.578125 77.09375 44.375 \n",
"Q 74.3125 48.1875 68.609375 48.1875 \n",
"Q 61.625 48.1875 57.5625 43.546875 \n",
"Q 53.515625 38.921875 53.515625 30.90625 \n",
"L 53.515625 0 \n",
"L 44.484375 0 \n",
"L 44.484375 32.71875 \n",
"Q 44.484375 40.625 41.703125 44.40625 \n",
"Q 38.921875 48.1875 33.109375 48.1875 \n",
"Q 26.21875 48.1875 22.15625 43.53125 \n",
"Q 18.109375 38.875 18.109375 30.90625 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 21.1875 51.21875 25.484375 53.609375 \n",
"Q 29.78125 56 35.6875 56 \n",
"Q 41.65625 56 45.828125 52.96875 \n",
"Q 50 49.953125 52 44.1875 \n",
"z\n",
"\" id=\"DejaVuSans-6d\"/>\n",
" <path d=\"M 48.6875 27.296875 \n",
"Q 48.6875 37.203125 44.609375 42.84375 \n",
"Q 40.53125 48.484375 33.40625 48.484375 \n",
"Q 26.265625 48.484375 22.1875 42.84375 \n",
"Q 18.109375 37.203125 18.109375 27.296875 \n",
"Q 18.109375 17.390625 22.1875 11.75 \n",
"Q 26.265625 6.109375 33.40625 6.109375 \n",
"Q 40.53125 6.109375 44.609375 11.75 \n",
"Q 48.6875 17.390625 48.6875 27.296875 \n",
"z\n",
"M 18.109375 46.390625 \n",
"Q 20.953125 51.265625 25.265625 53.625 \n",
"Q 29.59375 56 35.59375 56 \n",
"Q 45.5625 56 51.78125 48.09375 \n",
"Q 58.015625 40.1875 58.015625 27.296875 \n",
"Q 58.015625 14.40625 51.78125 6.484375 \n",
"Q 45.5625 -1.421875 35.59375 -1.421875 \n",
"Q 29.59375 -1.421875 25.265625 0.953125 \n",
"Q 20.953125 3.328125 18.109375 8.203125 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 75.984375 \n",
"L 18.109375 75.984375 \n",
"z\n",
"\" id=\"DejaVuSans-62\"/>\n",
" <path d=\"M 56.203125 29.59375 \n",
"L 56.203125 25.203125 \n",
"L 14.890625 25.203125 \n",
"Q 15.484375 15.921875 20.484375 11.0625 \n",
"Q 25.484375 6.203125 34.421875 6.203125 \n",
"Q 39.59375 6.203125 44.453125 7.46875 \n",
"Q 49.3125 8.734375 54.109375 11.28125 \n",
"L 54.109375 2.78125 \n",
"Q 49.265625 0.734375 44.1875 -0.34375 \n",
"Q 39.109375 -1.421875 33.890625 -1.421875 \n",
"Q 20.796875 -1.421875 13.15625 6.1875 \n",
"Q 5.515625 13.8125 5.515625 26.8125 \n",
"Q 5.515625 40.234375 12.765625 48.109375 \n",
"Q 20.015625 56 32.328125 56 \n",
"Q 43.359375 56 49.78125 48.890625 \n",
"Q 56.203125 41.796875 56.203125 29.59375 \n",
"z\n",
"M 47.21875 32.234375 \n",
"Q 47.125 39.59375 43.09375 43.984375 \n",
"Q 39.0625 48.390625 32.421875 48.390625 \n",
"Q 24.90625 48.390625 20.390625 44.140625 \n",
"Q 15.875 39.890625 15.1875 32.171875 \n",
"z\n",
"\" id=\"DejaVuSans-65\"/>\n",
" <path d=\"M 41.109375 46.296875 \n",
"Q 39.59375 47.171875 37.8125 47.578125 \n",
"Q 36.03125 48 33.890625 48 \n",
"Q 26.265625 48 22.1875 43.046875 \n",
"Q 18.109375 38.09375 18.109375 28.8125 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 20.953125 51.171875 25.484375 53.578125 \n",
"Q 30.03125 56 36.53125 56 \n",
"Q 37.453125 56 38.578125 55.875 \n",
"Q 39.703125 55.765625 41.0625 55.515625 \n",
"z\n",
"\" id=\"DejaVuSans-72\"/>\n",
" <path d=\"M 11.71875 12.40625 \n",
"L 22.015625 12.40625 \n",
"L 22.015625 0 \n",
"L 11.71875 0 \n",
"z\n",
"M 11.71875 51.703125 \n",
"L 22.015625 51.703125 \n",
"L 22.015625 39.3125 \n",
"L 11.71875 39.3125 \n",
"z\n",
"\" id=\"DejaVuSans-3a\"/>\n",
" <path d=\"M 8.203125 72.90625 \n",
"L 55.078125 72.90625 \n",
"L 55.078125 68.703125 \n",
"L 28.609375 0 \n",
"L 18.3125 0 \n",
"L 43.21875 64.59375 \n",
"L 8.203125 64.59375 \n",
"z\n",
"\" id=\"DejaVuSans-37\"/>\n",
" </defs>\n",
" <g transform=\"translate(139.317813 16.318125)scale(0.12 -0.12)\">\n",
" <use xlink:href=\"#DejaVuSans-52\"/>\n",
" <use x=\"69.419922\" xlink:href=\"#DejaVuSans-6f\"/>\n",
" <use x=\"130.601562\" xlink:href=\"#DejaVuSans-75\"/>\n",
" <use x=\"193.980469\" xlink:href=\"#DejaVuSans-74\"/>\n",
" <use x=\"233.189453\" xlink:href=\"#DejaVuSans-69\"/>\n",
" <use x=\"260.972656\" xlink:href=\"#DejaVuSans-6e\"/>\n",
" <use x=\"324.351562\" xlink:href=\"#DejaVuSans-67\"/>\n",
" <use x=\"387.828125\" xlink:href=\"#DejaVuSans-20\"/>\n",
" <use x=\"419.615234\" xlink:href=\"#DejaVuSans-4e\"/>\n",
" <use x=\"494.419922\" xlink:href=\"#DejaVuSans-75\"/>\n",
" <use x=\"557.798828\" xlink:href=\"#DejaVuSans-6d\"/>\n",
" <use x=\"655.210938\" xlink:href=\"#DejaVuSans-62\"/>\n",
" <use x=\"718.6875\" xlink:href=\"#DejaVuSans-65\"/>\n",
" <use x=\"780.210938\" xlink:href=\"#DejaVuSans-72\"/>\n",
" <use x=\"821.308594\" xlink:href=\"#DejaVuSans-3a\"/>\n",
" <use x=\"855\" xlink:href=\"#DejaVuSans-20\"/>\n",
" <use x=\"886.787109\" xlink:href=\"#DejaVuSans-37\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p16aab0a91b\">\n",
" <rect height=\"217.44\" width=\"334.8\" x=\"28.942188\" y=\"22.318125\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Created with matplotlib (http://matplotlib.org/) -->\n",
"<svg height=\"263.63625pt\" version=\"1.1\" viewBox=\"0 0 374.442187 263.63625\" width=\"374.442187pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
" <defs>\n",
" <style type=\"text/css\">\n",
"*{stroke-linecap:butt;stroke-linejoin:round;}\n",
" </style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 263.63625 \n",
"L 374.442187 263.63625 \n",
"L 374.442187 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill:none;\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 28.942188 239.758125 \n",
"L 363.742188 239.758125 \n",
"L 363.742188 22.318125 \n",
"L 28.942188 22.318125 \n",
"z\n",
"\" style=\"fill:#ffffff;\"/>\n",
" </g>\n",
" <g id=\"PathCollection_1\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"mb9ceaaf2ed\" style=\"stroke:#0000ff;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p781199e933)\">\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"211.287035\" xlink:href=\"#mb9ceaaf2ed\" y=\"111.361401\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"203.42531\" xlink:href=\"#mb9ceaaf2ed\" y=\"151.350309\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_2\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"m85cae867d0\" style=\"stroke:#ff0000;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p781199e933)\">\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"211.205927\" xlink:href=\"#m85cae867d0\" y=\"229.257978\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"348.440853\" xlink:href=\"#m85cae867d0\" y=\"115.159629\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"128.121902\" xlink:href=\"#m85cae867d0\" y=\"181.220167\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"191.524708\" xlink:href=\"#m85cae867d0\" y=\"222.202746\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"44.243522\" xlink:href=\"#m85cae867d0\" y=\"128.811127\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"213.850905\" xlink:href=\"#m85cae867d0\" y=\"34.015739\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"238.153511\" xlink:href=\"#m85cae867d0\" y=\"33.170424\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"200.08938\" xlink:href=\"#m85cae867d0\" y=\"211.90858\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"267.922832\" xlink:href=\"#m85cae867d0\" y=\"32.818272\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"313.151832\" xlink:href=\"#m85cae867d0\" y=\"76.216926\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" id=\"m9d56fe06ea\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"64.278463\" xlink:href=\"#m9d56fe06ea\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- −4 -->\n",
" <defs>\n",
" <path d=\"M 10.59375 35.5 \n",
"L 73.1875 35.5 \n",
"L 73.1875 27.203125 \n",
"L 10.59375 27.203125 \n",
"z\n",
"\" id=\"DejaVuSans-2212\"/>\n",
" <path d=\"M 37.796875 64.3125 \n",
"L 12.890625 25.390625 \n",
"L 37.796875 25.390625 \n",
"z\n",
"M 35.203125 72.90625 \n",
"L 47.609375 72.90625 \n",
"L 47.609375 25.390625 \n",
"L 58.015625 25.390625 \n",
"L 58.015625 17.1875 \n",
"L 47.609375 17.1875 \n",
"L 47.609375 0 \n",
"L 37.796875 0 \n",
"L 37.796875 17.1875 \n",
"L 4.890625 17.1875 \n",
"L 4.890625 26.703125 \n",
"z\n",
"\" id=\"DejaVuSans-34\"/>\n",
" </defs>\n",
" <g transform=\"translate(56.90737 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_2\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"128.386316\" xlink:href=\"#m9d56fe06ea\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- −2 -->\n",
" <defs>\n",
" <path d=\"M 19.1875 8.296875 \n",
"L 53.609375 8.296875 \n",
"L 53.609375 0 \n",
"L 7.328125 0 \n",
"L 7.328125 8.296875 \n",
"Q 12.9375 14.109375 22.625 23.890625 \n",
"Q 32.328125 33.6875 34.8125 36.53125 \n",
"Q 39.546875 41.84375 41.421875 45.53125 \n",
"Q 43.3125 49.21875 43.3125 52.78125 \n",
"Q 43.3125 58.59375 39.234375 62.25 \n",
"Q 35.15625 65.921875 28.609375 65.921875 \n",
"Q 23.96875 65.921875 18.8125 64.3125 \n",
"Q 13.671875 62.703125 7.8125 59.421875 \n",
"L 7.8125 69.390625 \n",
"Q 13.765625 71.78125 18.9375 73 \n",
"Q 24.125 74.21875 28.421875 74.21875 \n",
"Q 39.75 74.21875 46.484375 68.546875 \n",
"Q 53.21875 62.890625 53.21875 53.421875 \n",
"Q 53.21875 48.921875 51.53125 44.890625 \n",
"Q 49.859375 40.875 45.40625 35.40625 \n",
"Q 44.1875 33.984375 37.640625 27.21875 \n",
"Q 31.109375 20.453125 19.1875 8.296875 \n",
"z\n",
"\" id=\"DejaVuSans-32\"/>\n",
" </defs>\n",
" <g transform=\"translate(121.015222 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_3\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"192.494169\" xlink:href=\"#m9d56fe06ea\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 0 -->\n",
" <defs>\n",
" <path d=\"M 31.78125 66.40625 \n",
"Q 24.171875 66.40625 20.328125 58.90625 \n",
"Q 16.5 51.421875 16.5 36.375 \n",
"Q 16.5 21.390625 20.328125 13.890625 \n",
"Q 24.171875 6.390625 31.78125 6.390625 \n",
"Q 39.453125 6.390625 43.28125 13.890625 \n",
"Q 47.125 21.390625 47.125 36.375 \n",
"Q 47.125 51.421875 43.28125 58.90625 \n",
"Q 39.453125 66.40625 31.78125 66.40625 \n",
"z\n",
"M 31.78125 74.21875 \n",
"Q 44.046875 74.21875 50.515625 64.515625 \n",
"Q 56.984375 54.828125 56.984375 36.375 \n",
"Q 56.984375 17.96875 50.515625 8.265625 \n",
"Q 44.046875 -1.421875 31.78125 -1.421875 \n",
"Q 19.53125 -1.421875 13.0625 8.265625 \n",
"Q 6.59375 17.96875 6.59375 36.375 \n",
"Q 6.59375 54.828125 13.0625 64.515625 \n",
"Q 19.53125 74.21875 31.78125 74.21875 \n",
"z\n",
"\" id=\"DejaVuSans-30\"/>\n",
" </defs>\n",
" <g transform=\"translate(189.312919 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"256.602021\" xlink:href=\"#m9d56fe06ea\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(253.420771 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_5\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"320.709874\" xlink:href=\"#m9d56fe06ea\" y=\"239.758125\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(317.528624 254.356563)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_6\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" id=\"m37ef433189\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m37ef433189\" y=\"221.645495\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- −4 -->\n",
" <g transform=\"translate(7.2 225.444713)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_7\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m37ef433189\" y=\"175.726389\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- −2 -->\n",
" <g transform=\"translate(7.2 179.525608)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m37ef433189\" y=\"129.807283\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(15.579688 133.606502)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_9\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m37ef433189\" y=\"83.888178\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(15.579688 87.687396)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m37ef433189\" y=\"37.969072\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(15.579688 41.768291)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 28.942188 239.758125 \n",
"L 28.942188 22.318125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 363.742188 239.758125 \n",
"L 363.742188 22.318125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 28.942187 239.758125 \n",
"L 363.742187 239.758125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 28.942187 22.318125 \n",
"L 363.742187 22.318125 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- Routing Number: 9 -->\n",
" <defs>\n",
" <path d=\"M 44.390625 34.1875 \n",
"Q 47.5625 33.109375 50.5625 29.59375 \n",
"Q 53.5625 26.078125 56.59375 19.921875 \n",
"L 66.609375 0 \n",
"L 56 0 \n",
"L 46.6875 18.703125 \n",
"Q 43.0625 26.03125 39.671875 28.421875 \n",
"Q 36.28125 30.8125 30.421875 30.8125 \n",
"L 19.671875 30.8125 \n",
"L 19.671875 0 \n",
"L 9.8125 0 \n",
"L 9.8125 72.90625 \n",
"L 32.078125 72.90625 \n",
"Q 44.578125 72.90625 50.734375 67.671875 \n",
"Q 56.890625 62.453125 56.890625 51.90625 \n",
"Q 56.890625 45.015625 53.6875 40.46875 \n",
"Q 50.484375 35.9375 44.390625 34.1875 \n",
"z\n",
"M 19.671875 64.796875 \n",
"L 19.671875 38.921875 \n",
"L 32.078125 38.921875 \n",
"Q 39.203125 38.921875 42.84375 42.21875 \n",
"Q 46.484375 45.515625 46.484375 51.90625 \n",
"Q 46.484375 58.296875 42.84375 61.546875 \n",
"Q 39.203125 64.796875 32.078125 64.796875 \n",
"z\n",
"\" id=\"DejaVuSans-52\"/>\n",
" <path d=\"M 30.609375 48.390625 \n",
"Q 23.390625 48.390625 19.1875 42.75 \n",
"Q 14.984375 37.109375 14.984375 27.296875 \n",
"Q 14.984375 17.484375 19.15625 11.84375 \n",
"Q 23.34375 6.203125 30.609375 6.203125 \n",
"Q 37.796875 6.203125 41.984375 11.859375 \n",
"Q 46.1875 17.53125 46.1875 27.296875 \n",
"Q 46.1875 37.015625 41.984375 42.703125 \n",
"Q 37.796875 48.390625 30.609375 48.390625 \n",
"z\n",
"M 30.609375 56 \n",
"Q 42.328125 56 49.015625 48.375 \n",
"Q 55.71875 40.765625 55.71875 27.296875 \n",
"Q 55.71875 13.875 49.015625 6.21875 \n",
"Q 42.328125 -1.421875 30.609375 -1.421875 \n",
"Q 18.84375 -1.421875 12.171875 6.21875 \n",
"Q 5.515625 13.875 5.515625 27.296875 \n",
"Q 5.515625 40.765625 12.171875 48.375 \n",
"Q 18.84375 56 30.609375 56 \n",
"z\n",
"\" id=\"DejaVuSans-6f\"/>\n",
" <path d=\"M 8.5 21.578125 \n",
"L 8.5 54.6875 \n",
"L 17.484375 54.6875 \n",
"L 17.484375 21.921875 \n",
"Q 17.484375 14.15625 20.5 10.265625 \n",
"Q 23.53125 6.390625 29.59375 6.390625 \n",
"Q 36.859375 6.390625 41.078125 11.03125 \n",
"Q 45.3125 15.671875 45.3125 23.6875 \n",
"L 45.3125 54.6875 \n",
"L 54.296875 54.6875 \n",
"L 54.296875 0 \n",
"L 45.3125 0 \n",
"L 45.3125 8.40625 \n",
"Q 42.046875 3.421875 37.71875 1 \n",
"Q 33.40625 -1.421875 27.6875 -1.421875 \n",
"Q 18.265625 -1.421875 13.375 4.4375 \n",
"Q 8.5 10.296875 8.5 21.578125 \n",
"z\n",
"M 31.109375 56 \n",
"z\n",
"\" id=\"DejaVuSans-75\"/>\n",
" <path d=\"M 18.3125 70.21875 \n",
"L 18.3125 54.6875 \n",
"L 36.8125 54.6875 \n",
"L 36.8125 47.703125 \n",
"L 18.3125 47.703125 \n",
"L 18.3125 18.015625 \n",
"Q 18.3125 11.328125 20.140625 9.421875 \n",
"Q 21.96875 7.515625 27.59375 7.515625 \n",
"L 36.8125 7.515625 \n",
"L 36.8125 0 \n",
"L 27.59375 0 \n",
"Q 17.1875 0 13.234375 3.875 \n",
"Q 9.28125 7.765625 9.28125 18.015625 \n",
"L 9.28125 47.703125 \n",
"L 2.6875 47.703125 \n",
"L 2.6875 54.6875 \n",
"L 9.28125 54.6875 \n",
"L 9.28125 70.21875 \n",
"z\n",
"\" id=\"DejaVuSans-74\"/>\n",
" <path d=\"M 9.421875 54.6875 \n",
"L 18.40625 54.6875 \n",
"L 18.40625 0 \n",
"L 9.421875 0 \n",
"z\n",
"M 9.421875 75.984375 \n",
"L 18.40625 75.984375 \n",
"L 18.40625 64.59375 \n",
"L 9.421875 64.59375 \n",
"z\n",
"\" id=\"DejaVuSans-69\"/>\n",
" <path d=\"M 54.890625 33.015625 \n",
"L 54.890625 0 \n",
"L 45.90625 0 \n",
"L 45.90625 32.71875 \n",
"Q 45.90625 40.484375 42.875 44.328125 \n",
"Q 39.84375 48.1875 33.796875 48.1875 \n",
"Q 26.515625 48.1875 22.3125 43.546875 \n",
"Q 18.109375 38.921875 18.109375 30.90625 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 21.34375 51.125 25.703125 53.5625 \n",
"Q 30.078125 56 35.796875 56 \n",
"Q 45.21875 56 50.046875 50.171875 \n",
"Q 54.890625 44.34375 54.890625 33.015625 \n",
"z\n",
"\" id=\"DejaVuSans-6e\"/>\n",
" <path d=\"M 45.40625 27.984375 \n",
"Q 45.40625 37.75 41.375 43.109375 \n",
"Q 37.359375 48.484375 30.078125 48.484375 \n",
"Q 22.859375 48.484375 18.828125 43.109375 \n",
"Q 14.796875 37.75 14.796875 27.984375 \n",
"Q 14.796875 18.265625 18.828125 12.890625 \n",
"Q 22.859375 7.515625 30.078125 7.515625 \n",
"Q 37.359375 7.515625 41.375 12.890625 \n",
"Q 45.40625 18.265625 45.40625 27.984375 \n",
"z\n",
"M 54.390625 6.78125 \n",
"Q 54.390625 -7.171875 48.1875 -13.984375 \n",
"Q 42 -20.796875 29.203125 -20.796875 \n",
"Q 24.46875 -20.796875 20.265625 -20.09375 \n",
"Q 16.0625 -19.390625 12.109375 -17.921875 \n",
"L 12.109375 -9.1875 \n",
"Q 16.0625 -11.328125 19.921875 -12.34375 \n",
"Q 23.78125 -13.375 27.78125 -13.375 \n",
"Q 36.625 -13.375 41.015625 -8.765625 \n",
"Q 45.40625 -4.15625 45.40625 5.171875 \n",
"L 45.40625 9.625 \n",
"Q 42.625 4.78125 38.28125 2.390625 \n",
"Q 33.9375 0 27.875 0 \n",
"Q 17.828125 0 11.671875 7.65625 \n",
"Q 5.515625 15.328125 5.515625 27.984375 \n",
"Q 5.515625 40.671875 11.671875 48.328125 \n",
"Q 17.828125 56 27.875 56 \n",
"Q 33.9375 56 38.28125 53.609375 \n",
"Q 42.625 51.21875 45.40625 46.390625 \n",
"L 45.40625 54.6875 \n",
"L 54.390625 54.6875 \n",
"z\n",
"\" id=\"DejaVuSans-67\"/>\n",
" <path id=\"DejaVuSans-20\"/>\n",
" <path d=\"M 9.8125 72.90625 \n",
"L 23.09375 72.90625 \n",
"L 55.421875 11.921875 \n",
"L 55.421875 72.90625 \n",
"L 64.984375 72.90625 \n",
"L 64.984375 0 \n",
"L 51.703125 0 \n",
"L 19.390625 60.984375 \n",
"L 19.390625 0 \n",
"L 9.8125 0 \n",
"z\n",
"\" id=\"DejaVuSans-4e\"/>\n",
" <path d=\"M 52 44.1875 \n",
"Q 55.375 50.25 60.0625 53.125 \n",
"Q 64.75 56 71.09375 56 \n",
"Q 79.640625 56 84.28125 50.015625 \n",
"Q 88.921875 44.046875 88.921875 33.015625 \n",
"L 88.921875 0 \n",
"L 79.890625 0 \n",
"L 79.890625 32.71875 \n",
"Q 79.890625 40.578125 77.09375 44.375 \n",
"Q 74.3125 48.1875 68.609375 48.1875 \n",
"Q 61.625 48.1875 57.5625 43.546875 \n",
"Q 53.515625 38.921875 53.515625 30.90625 \n",
"L 53.515625 0 \n",
"L 44.484375 0 \n",
"L 44.484375 32.71875 \n",
"Q 44.484375 40.625 41.703125 44.40625 \n",
"Q 38.921875 48.1875 33.109375 48.1875 \n",
"Q 26.21875 48.1875 22.15625 43.53125 \n",
"Q 18.109375 38.875 18.109375 30.90625 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 21.1875 51.21875 25.484375 53.609375 \n",
"Q 29.78125 56 35.6875 56 \n",
"Q 41.65625 56 45.828125 52.96875 \n",
"Q 50 49.953125 52 44.1875 \n",
"z\n",
"\" id=\"DejaVuSans-6d\"/>\n",
" <path d=\"M 48.6875 27.296875 \n",
"Q 48.6875 37.203125 44.609375 42.84375 \n",
"Q 40.53125 48.484375 33.40625 48.484375 \n",
"Q 26.265625 48.484375 22.1875 42.84375 \n",
"Q 18.109375 37.203125 18.109375 27.296875 \n",
"Q 18.109375 17.390625 22.1875 11.75 \n",
"Q 26.265625 6.109375 33.40625 6.109375 \n",
"Q 40.53125 6.109375 44.609375 11.75 \n",
"Q 48.6875 17.390625 48.6875 27.296875 \n",
"z\n",
"M 18.109375 46.390625 \n",
"Q 20.953125 51.265625 25.265625 53.625 \n",
"Q 29.59375 56 35.59375 56 \n",
"Q 45.5625 56 51.78125 48.09375 \n",
"Q 58.015625 40.1875 58.015625 27.296875 \n",
"Q 58.015625 14.40625 51.78125 6.484375 \n",
"Q 45.5625 -1.421875 35.59375 -1.421875 \n",
"Q 29.59375 -1.421875 25.265625 0.953125 \n",
"Q 20.953125 3.328125 18.109375 8.203125 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 75.984375 \n",
"L 18.109375 75.984375 \n",
"z\n",
"\" id=\"DejaVuSans-62\"/>\n",
" <path d=\"M 56.203125 29.59375 \n",
"L 56.203125 25.203125 \n",
"L 14.890625 25.203125 \n",
"Q 15.484375 15.921875 20.484375 11.0625 \n",
"Q 25.484375 6.203125 34.421875 6.203125 \n",
"Q 39.59375 6.203125 44.453125 7.46875 \n",
"Q 49.3125 8.734375 54.109375 11.28125 \n",
"L 54.109375 2.78125 \n",
"Q 49.265625 0.734375 44.1875 -0.34375 \n",
"Q 39.109375 -1.421875 33.890625 -1.421875 \n",
"Q 20.796875 -1.421875 13.15625 6.1875 \n",
"Q 5.515625 13.8125 5.515625 26.8125 \n",
"Q 5.515625 40.234375 12.765625 48.109375 \n",
"Q 20.015625 56 32.328125 56 \n",
"Q 43.359375 56 49.78125 48.890625 \n",
"Q 56.203125 41.796875 56.203125 29.59375 \n",
"z\n",
"M 47.21875 32.234375 \n",
"Q 47.125 39.59375 43.09375 43.984375 \n",
"Q 39.0625 48.390625 32.421875 48.390625 \n",
"Q 24.90625 48.390625 20.390625 44.140625 \n",
"Q 15.875 39.890625 15.1875 32.171875 \n",
"z\n",
"\" id=\"DejaVuSans-65\"/>\n",
" <path d=\"M 41.109375 46.296875 \n",
"Q 39.59375 47.171875 37.8125 47.578125 \n",
"Q 36.03125 48 33.890625 48 \n",
"Q 26.265625 48 22.1875 43.046875 \n",
"Q 18.109375 38.09375 18.109375 28.8125 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 20.953125 51.171875 25.484375 53.578125 \n",
"Q 30.03125 56 36.53125 56 \n",
"Q 37.453125 56 38.578125 55.875 \n",
"Q 39.703125 55.765625 41.0625 55.515625 \n",
"z\n",
"\" id=\"DejaVuSans-72\"/>\n",
" <path d=\"M 11.71875 12.40625 \n",
"L 22.015625 12.40625 \n",
"L 22.015625 0 \n",
"L 11.71875 0 \n",
"z\n",
"M 11.71875 51.703125 \n",
"L 22.015625 51.703125 \n",
"L 22.015625 39.3125 \n",
"L 11.71875 39.3125 \n",
"z\n",
"\" id=\"DejaVuSans-3a\"/>\n",
" <path d=\"M 10.984375 1.515625 \n",
"L 10.984375 10.5 \n",
"Q 14.703125 8.734375 18.5 7.8125 \n",
"Q 22.3125 6.890625 25.984375 6.890625 \n",
"Q 35.75 6.890625 40.890625 13.453125 \n",
"Q 46.046875 20.015625 46.78125 33.40625 \n",
"Q 43.953125 29.203125 39.59375 26.953125 \n",
"Q 35.25 24.703125 29.984375 24.703125 \n",
"Q 19.046875 24.703125 12.671875 31.3125 \n",
"Q 6.296875 37.9375 6.296875 49.421875 \n",
"Q 6.296875 60.640625 12.9375 67.421875 \n",
"Q 19.578125 74.21875 30.609375 74.21875 \n",
"Q 43.265625 74.21875 49.921875 64.515625 \n",
"Q 56.59375 54.828125 56.59375 36.375 \n",
"Q 56.59375 19.140625 48.40625 8.859375 \n",
"Q 40.234375 -1.421875 26.421875 -1.421875 \n",
"Q 22.703125 -1.421875 18.890625 -0.6875 \n",
"Q 15.09375 0.046875 10.984375 1.515625 \n",
"z\n",
"M 30.609375 32.421875 \n",
"Q 37.25 32.421875 41.125 36.953125 \n",
"Q 45.015625 41.5 45.015625 49.421875 \n",
"Q 45.015625 57.28125 41.125 61.84375 \n",
"Q 37.25 66.40625 30.609375 66.40625 \n",
"Q 23.96875 66.40625 20.09375 61.84375 \n",
"Q 16.21875 57.28125 16.21875 49.421875 \n",
"Q 16.21875 41.5 20.09375 36.953125 \n",
"Q 23.96875 32.421875 30.609375 32.421875 \n",
"z\n",
"\" id=\"DejaVuSans-39\"/>\n",
" </defs>\n",
" <g transform=\"translate(139.317813 16.318125)scale(0.12 -0.12)\">\n",
" <use xlink:href=\"#DejaVuSans-52\"/>\n",
" <use x=\"69.419922\" xlink:href=\"#DejaVuSans-6f\"/>\n",
" <use x=\"130.601562\" xlink:href=\"#DejaVuSans-75\"/>\n",
" <use x=\"193.980469\" xlink:href=\"#DejaVuSans-74\"/>\n",
" <use x=\"233.189453\" xlink:href=\"#DejaVuSans-69\"/>\n",
" <use x=\"260.972656\" xlink:href=\"#DejaVuSans-6e\"/>\n",
" <use x=\"324.351562\" xlink:href=\"#DejaVuSans-67\"/>\n",
" <use x=\"387.828125\" xlink:href=\"#DejaVuSans-20\"/>\n",
" <use x=\"419.615234\" xlink:href=\"#DejaVuSans-4e\"/>\n",
" <use x=\"494.419922\" xlink:href=\"#DejaVuSans-75\"/>\n",
" <use x=\"557.798828\" xlink:href=\"#DejaVuSans-6d\"/>\n",
" <use x=\"655.210938\" xlink:href=\"#DejaVuSans-62\"/>\n",
" <use x=\"718.6875\" xlink:href=\"#DejaVuSans-65\"/>\n",
" <use x=\"780.210938\" xlink:href=\"#DejaVuSans-72\"/>\n",
" <use x=\"821.308594\" xlink:href=\"#DejaVuSans-3a\"/>\n",
" <use x=\"855\" xlink:href=\"#DejaVuSans-20\"/>\n",
" <use x=\"886.787109\" xlink:href=\"#DejaVuSans-39\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p781199e933\">\n",
" <rect height=\"217.44\" width=\"334.8\" x=\"28.942188\" y=\"22.318125\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"iterations=10\n",
"selected_node_feats=[points_1.data.numpy().T]\n",
"for i in range(iterations):\n",
" g.update_all(capsule_msg, v2_reduce, squash)\n",
" g.update_edge(edge_func=update_edge)\n",
" \n",
" new_=g.get_n_repr()['h'][num_points:num_points+num_points2]\n",
" np_new=new_.data.numpy()\n",
" if i in (1,3,7,9):\n",
" selected_node_feats.append(np_new)\n",
" plt.scatter(*np_new.T,color='blue')\n",
" plt.scatter(*points_1, color='red')\n",
" plt.title(f\"Routing Number: {i}\")\n",
" plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Created with matplotlib (http://matplotlib.org/) -->\n",
"<svg height=\"252.018125pt\" version=\"1.1\" viewBox=\"0 0 374.442187 252.018125\" width=\"374.442187pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
" <defs>\n",
" <style type=\"text/css\">\n",
"*{stroke-linecap:butt;stroke-linejoin:round;}\n",
" </style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 252.018125 \n",
"L 374.442187 252.018125 \n",
"L 374.442187 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill:none;\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 28.942188 228.14 \n",
"L 363.742188 228.14 \n",
"L 363.742188 10.7 \n",
"L 28.942188 10.7 \n",
"z\n",
"\" style=\"fill:#ffffff;\"/>\n",
" </g>\n",
" <g id=\"PathCollection_1\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"ma3f06967fa\" style=\"stroke:#0000ff;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p52c97a1474)\">\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"210.926016\" xlink:href=\"#ma3f06967fa\" y=\"215.257566\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"345.576554\" xlink:href=\"#ma3f06967fa\" y=\"103.88061\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"129.406616\" xlink:href=\"#ma3f06967fa\" y=\"168.365519\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"191.61543\" xlink:href=\"#ma3f06967fa\" y=\"208.370611\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"47.107821\" xlink:href=\"#ma3f06967fa\" y=\"117.206502\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"213.521183\" xlink:href=\"#ma3f06967fa\" y=\"24.672107\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"237.366127\" xlink:href=\"#ma3f06967fa\" y=\"23.846954\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"200.018814\" xlink:href=\"#ma3f06967fa\" y=\"198.321974\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"266.574836\" xlink:href=\"#ma3f06967fa\" y=\"23.503201\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"310.95209\" xlink:href=\"#ma3f06967fa\" y=\"65.866741\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_2\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"mce88fad94f\" style=\"stroke:#ff0000;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p52c97a1474)\">\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"210.926016\" xlink:href=\"#mce88fad94f\" y=\"215.257566\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"345.576554\" xlink:href=\"#mce88fad94f\" y=\"103.88061\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"129.406616\" xlink:href=\"#mce88fad94f\" y=\"168.365519\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"191.61543\" xlink:href=\"#mce88fad94f\" y=\"208.370611\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"47.107821\" xlink:href=\"#mce88fad94f\" y=\"117.206502\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"213.521183\" xlink:href=\"#mce88fad94f\" y=\"24.672107\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"237.366127\" xlink:href=\"#mce88fad94f\" y=\"23.846954\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"200.018814\" xlink:href=\"#mce88fad94f\" y=\"198.321974\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"266.574836\" xlink:href=\"#mce88fad94f\" y=\"23.503201\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"310.95209\" xlink:href=\"#mce88fad94f\" y=\"65.866741\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_3\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"m4a86dbf850\" style=\"stroke:#00ffff;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p52c97a1474)\">\n",
" <use style=\"fill:#00ffff;stroke:#00ffff;\" x=\"210.926016\" xlink:href=\"#m4a86dbf850\" y=\"215.257566\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" id=\"ma274f0d1bf\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"66.765468\" xlink:href=\"#ma274f0d1bf\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- −4 -->\n",
" <defs>\n",
" <path d=\"M 10.59375 35.5 \n",
"L 73.1875 35.5 \n",
"L 73.1875 27.203125 \n",
"L 10.59375 27.203125 \n",
"z\n",
"\" id=\"DejaVuSans-2212\"/>\n",
" <path d=\"M 37.796875 64.3125 \n",
"L 12.890625 25.390625 \n",
"L 37.796875 25.390625 \n",
"z\n",
"M 35.203125 72.90625 \n",
"L 47.609375 72.90625 \n",
"L 47.609375 25.390625 \n",
"L 58.015625 25.390625 \n",
"L 58.015625 17.1875 \n",
"L 47.609375 17.1875 \n",
"L 47.609375 0 \n",
"L 37.796875 0 \n",
"L 37.796875 17.1875 \n",
"L 4.890625 17.1875 \n",
"L 4.890625 26.703125 \n",
"z\n",
"\" id=\"DejaVuSans-34\"/>\n",
" </defs>\n",
" <g transform=\"translate(59.394374 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_2\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"129.666051\" xlink:href=\"#ma274f0d1bf\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- −2 -->\n",
" <defs>\n",
" <path d=\"M 19.1875 8.296875 \n",
"L 53.609375 8.296875 \n",
"L 53.609375 0 \n",
"L 7.328125 0 \n",
"L 7.328125 8.296875 \n",
"Q 12.9375 14.109375 22.625 23.890625 \n",
"Q 32.328125 33.6875 34.8125 36.53125 \n",
"Q 39.546875 41.84375 41.421875 45.53125 \n",
"Q 43.3125 49.21875 43.3125 52.78125 \n",
"Q 43.3125 58.59375 39.234375 62.25 \n",
"Q 35.15625 65.921875 28.609375 65.921875 \n",
"Q 23.96875 65.921875 18.8125 64.3125 \n",
"Q 13.671875 62.703125 7.8125 59.421875 \n",
"L 7.8125 69.390625 \n",
"Q 13.765625 71.78125 18.9375 73 \n",
"Q 24.125 74.21875 28.421875 74.21875 \n",
"Q 39.75 74.21875 46.484375 68.546875 \n",
"Q 53.21875 62.890625 53.21875 53.421875 \n",
"Q 53.21875 48.921875 51.53125 44.890625 \n",
"Q 49.859375 40.875 45.40625 35.40625 \n",
"Q 44.1875 33.984375 37.640625 27.21875 \n",
"Q 31.109375 20.453125 19.1875 8.296875 \n",
"z\n",
"\" id=\"DejaVuSans-32\"/>\n",
" </defs>\n",
" <g transform=\"translate(122.294957 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_3\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"192.566634\" xlink:href=\"#ma274f0d1bf\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 0 -->\n",
" <defs>\n",
" <path d=\"M 31.78125 66.40625 \n",
"Q 24.171875 66.40625 20.328125 58.90625 \n",
"Q 16.5 51.421875 16.5 36.375 \n",
"Q 16.5 21.390625 20.328125 13.890625 \n",
"Q 24.171875 6.390625 31.78125 6.390625 \n",
"Q 39.453125 6.390625 43.28125 13.890625 \n",
"Q 47.125 21.390625 47.125 36.375 \n",
"Q 47.125 51.421875 43.28125 58.90625 \n",
"Q 39.453125 66.40625 31.78125 66.40625 \n",
"z\n",
"M 31.78125 74.21875 \n",
"Q 44.046875 74.21875 50.515625 64.515625 \n",
"Q 56.984375 54.828125 56.984375 36.375 \n",
"Q 56.984375 17.96875 50.515625 8.265625 \n",
"Q 44.046875 -1.421875 31.78125 -1.421875 \n",
"Q 19.53125 -1.421875 13.0625 8.265625 \n",
"Q 6.59375 17.96875 6.59375 36.375 \n",
"Q 6.59375 54.828125 13.0625 64.515625 \n",
"Q 19.53125 74.21875 31.78125 74.21875 \n",
"z\n",
"\" id=\"DejaVuSans-30\"/>\n",
" </defs>\n",
" <g transform=\"translate(189.385384 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"255.467217\" xlink:href=\"#ma274f0d1bf\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(252.285967 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_5\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"318.3678\" xlink:href=\"#ma274f0d1bf\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(315.18655 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_6\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" id=\"m85ed0e08fd\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m85ed0e08fd\" y=\"207.82665\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- −4 -->\n",
" <g transform=\"translate(7.2 211.625869)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_7\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m85ed0e08fd\" y=\"163.002775\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- −2 -->\n",
" <g transform=\"translate(7.2 166.801994)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m85ed0e08fd\" y=\"118.178899\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(15.579688 121.978118)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_9\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m85ed0e08fd\" y=\"73.355024\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(15.579688 77.154242)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m85ed0e08fd\" y=\"28.531148\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(15.579688 32.330367)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 28.942188 228.14 \n",
"L 28.942188 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 363.742188 228.14 \n",
"L 363.742188 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 28.942188 228.14 \n",
"L 363.742188 228.14 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 28.942188 10.7 \n",
"L 363.742188 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p52c97a1474\">\n",
" <rect height=\"217.44\" width=\"334.8\" x=\"28.942188\" y=\"10.7\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Created with matplotlib (http://matplotlib.org/) -->\n",
"<svg height=\"252.018125pt\" version=\"1.1\" viewBox=\"0 0 374.442187 252.018125\" width=\"374.442187pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
" <defs>\n",
" <style type=\"text/css\">\n",
"*{stroke-linecap:butt;stroke-linejoin:round;}\n",
" </style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 252.018125 \n",
"L 374.442187 252.018125 \n",
"L 374.442187 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill:none;\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 28.942188 228.14 \n",
"L 363.742188 228.14 \n",
"L 363.742188 10.7 \n",
"L 28.942188 10.7 \n",
"z\n",
"\" style=\"fill:#ffffff;\"/>\n",
" </g>\n",
" <g id=\"PathCollection_1\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"m62a3bf5a63\" style=\"stroke:#0000ff;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#pef5d34a61a)\">\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"213.047736\" xlink:href=\"#m62a3bf5a63\" y=\"100.753809\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"199.257259\" xlink:href=\"#m62a3bf5a63\" y=\"140.586095\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_2\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"m5f29994dba\" style=\"stroke:#ff0000;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#pef5d34a61a)\">\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"211.200217\" xlink:href=\"#m5f29994dba\" y=\"217.64223\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"348.382418\" xlink:href=\"#m5f29994dba\" y=\"103.54112\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"128.148112\" xlink:href=\"#m5f29994dba\" y=\"169.603256\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"191.526559\" xlink:href=\"#m5f29994dba\" y=\"210.586827\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"44.301957\" xlink:href=\"#m5f29994dba\" y=\"117.192948\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"213.844178\" xlink:href=\"#m5f29994dba\" y=\"22.395266\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"238.137448\" xlink:href=\"#m5f29994dba\" y=\"21.549931\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"200.087941\" xlink:href=\"#m5f29994dba\" y=\"200.292412\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"267.895332\" xlink:href=\"#m5f29994dba\" y=\"21.19777\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"313.106955\" xlink:href=\"#m5f29994dba\" y=\"64.597474\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_3\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"mfd0bf6d766\" style=\"stroke:#00ffff;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#pef5d34a61a)\">\n",
" <use style=\"fill:#00ffff;stroke:#00ffff;\" x=\"213.047736\" xlink:href=\"#mfd0bf6d766\" y=\"100.753809\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" id=\"m6c4c582f49\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"64.329202\" xlink:href=\"#m6c4c582f49\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- −4 -->\n",
" <defs>\n",
" <path d=\"M 10.59375 35.5 \n",
"L 73.1875 35.5 \n",
"L 73.1875 27.203125 \n",
"L 10.59375 27.203125 \n",
"z\n",
"\" id=\"DejaVuSans-2212\"/>\n",
" <path d=\"M 37.796875 64.3125 \n",
"L 12.890625 25.390625 \n",
"L 37.796875 25.390625 \n",
"z\n",
"M 35.203125 72.90625 \n",
"L 47.609375 72.90625 \n",
"L 47.609375 25.390625 \n",
"L 58.015625 25.390625 \n",
"L 58.015625 17.1875 \n",
"L 47.609375 17.1875 \n",
"L 47.609375 0 \n",
"L 37.796875 0 \n",
"L 37.796875 17.1875 \n",
"L 4.890625 17.1875 \n",
"L 4.890625 26.703125 \n",
"z\n",
"\" id=\"DejaVuSans-34\"/>\n",
" </defs>\n",
" <g transform=\"translate(56.958108 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_2\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"128.412424\" xlink:href=\"#m6c4c582f49\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- −2 -->\n",
" <defs>\n",
" <path d=\"M 19.1875 8.296875 \n",
"L 53.609375 8.296875 \n",
"L 53.609375 0 \n",
"L 7.328125 0 \n",
"L 7.328125 8.296875 \n",
"Q 12.9375 14.109375 22.625 23.890625 \n",
"Q 32.328125 33.6875 34.8125 36.53125 \n",
"Q 39.546875 41.84375 41.421875 45.53125 \n",
"Q 43.3125 49.21875 43.3125 52.78125 \n",
"Q 43.3125 58.59375 39.234375 62.25 \n",
"Q 35.15625 65.921875 28.609375 65.921875 \n",
"Q 23.96875 65.921875 18.8125 64.3125 \n",
"Q 13.671875 62.703125 7.8125 59.421875 \n",
"L 7.8125 69.390625 \n",
"Q 13.765625 71.78125 18.9375 73 \n",
"Q 24.125 74.21875 28.421875 74.21875 \n",
"Q 39.75 74.21875 46.484375 68.546875 \n",
"Q 53.21875 62.890625 53.21875 53.421875 \n",
"Q 53.21875 48.921875 51.53125 44.890625 \n",
"Q 49.859375 40.875 45.40625 35.40625 \n",
"Q 44.1875 33.984375 37.640625 27.21875 \n",
"Q 31.109375 20.453125 19.1875 8.296875 \n",
"z\n",
"\" id=\"DejaVuSans-32\"/>\n",
" </defs>\n",
" <g transform=\"translate(121.041331 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_3\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"192.495647\" xlink:href=\"#m6c4c582f49\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 0 -->\n",
" <defs>\n",
" <path d=\"M 31.78125 66.40625 \n",
"Q 24.171875 66.40625 20.328125 58.90625 \n",
"Q 16.5 51.421875 16.5 36.375 \n",
"Q 16.5 21.390625 20.328125 13.890625 \n",
"Q 24.171875 6.390625 31.78125 6.390625 \n",
"Q 39.453125 6.390625 43.28125 13.890625 \n",
"Q 47.125 21.390625 47.125 36.375 \n",
"Q 47.125 51.421875 43.28125 58.90625 \n",
"Q 39.453125 66.40625 31.78125 66.40625 \n",
"z\n",
"M 31.78125 74.21875 \n",
"Q 44.046875 74.21875 50.515625 64.515625 \n",
"Q 56.984375 54.828125 56.984375 36.375 \n",
"Q 56.984375 17.96875 50.515625 8.265625 \n",
"Q 44.046875 -1.421875 31.78125 -1.421875 \n",
"Q 19.53125 -1.421875 13.0625 8.265625 \n",
"Q 6.59375 17.96875 6.59375 36.375 \n",
"Q 6.59375 54.828125 13.0625 64.515625 \n",
"Q 19.53125 74.21875 31.78125 74.21875 \n",
"z\n",
"\" id=\"DejaVuSans-30\"/>\n",
" </defs>\n",
" <g transform=\"translate(189.314397 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"256.57887\" xlink:href=\"#m6c4c582f49\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(253.39762 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_5\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"320.662093\" xlink:href=\"#m6c4c582f49\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(317.480843 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_6\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" id=\"m3b072ae0b1\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m3b072ae0b1\" y=\"210.029562\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- −4 -->\n",
" <g transform=\"translate(7.2 213.828781)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_7\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m3b072ae0b1\" y=\"164.109345\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- −2 -->\n",
" <g transform=\"translate(7.2 167.908564)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m3b072ae0b1\" y=\"118.189129\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(15.579688 121.988347)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_9\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m3b072ae0b1\" y=\"72.268912\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(15.579688 76.06813)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m3b072ae0b1\" y=\"26.348695\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(15.579688 30.147914)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 28.942188 228.14 \n",
"L 28.942188 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 363.742188 228.14 \n",
"L 363.742188 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 28.942187 228.14 \n",
"L 363.742188 228.14 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 28.942187 10.7 \n",
"L 363.742188 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"pef5d34a61a\">\n",
" <rect height=\"217.44\" width=\"334.8\" x=\"28.942188\" y=\"10.7\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Created with matplotlib (http://matplotlib.org/) -->\n",
"<svg height=\"252.018125pt\" version=\"1.1\" viewBox=\"0 0 374.442187 252.018125\" width=\"374.442187pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
" <defs>\n",
" <style type=\"text/css\">\n",
"*{stroke-linecap:butt;stroke-linejoin:round;}\n",
" </style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 252.018125 \n",
"L 374.442187 252.018125 \n",
"L 374.442187 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill:none;\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 28.942188 228.14 \n",
"L 363.742188 228.14 \n",
"L 363.742188 10.7 \n",
"L 28.942188 10.7 \n",
"z\n",
"\" style=\"fill:#ffffff;\"/>\n",
" </g>\n",
" <g id=\"PathCollection_1\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"mff3a49d669\" style=\"stroke:#0000ff;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p00c0127347)\">\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"211.249207\" xlink:href=\"#mff3a49d669\" y=\"99.723797\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"203.137797\" xlink:href=\"#mff3a49d669\" y=\"139.805261\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_2\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"m6b633d9c79\" style=\"stroke:#ff0000;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p00c0127347)\">\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"211.205687\" xlink:href=\"#m6b633d9c79\" y=\"217.638448\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"348.438393\" xlink:href=\"#m6b633d9c79\" y=\"103.541731\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"128.123006\" xlink:href=\"#m6b633d9c79\" y=\"169.601324\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"191.524786\" xlink:href=\"#m6b633d9c79\" y=\"210.583317\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"44.245982\" xlink:href=\"#m6b633d9c79\" y=\"117.193034\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"213.850622\" xlink:href=\"#m6b633d9c79\" y=\"22.399001\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"238.152835\" xlink:href=\"#m6b633d9c79\" y=\"21.553699\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"200.08932\" xlink:href=\"#m6b633d9c79\" y=\"200.289298\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"267.921674\" xlink:href=\"#m6b633d9c79\" y=\"21.201552\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"313.149942\" xlink:href=\"#m6b633d9c79\" y=\"64.599585\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_3\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"m3f5b3f42c3\" style=\"stroke:#00ffff;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p00c0127347)\">\n",
" <use style=\"fill:#00ffff;stroke:#00ffff;\" x=\"211.249207\" xlink:href=\"#m3f5b3f42c3\" y=\"99.723797\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" id=\"m97102b0cb2\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"64.2806\" xlink:href=\"#m97102b0cb2\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- −4 -->\n",
" <defs>\n",
" <path d=\"M 10.59375 35.5 \n",
"L 73.1875 35.5 \n",
"L 73.1875 27.203125 \n",
"L 10.59375 27.203125 \n",
"z\n",
"\" id=\"DejaVuSans-2212\"/>\n",
" <path d=\"M 37.796875 64.3125 \n",
"L 12.890625 25.390625 \n",
"L 37.796875 25.390625 \n",
"z\n",
"M 35.203125 72.90625 \n",
"L 47.609375 72.90625 \n",
"L 47.609375 25.390625 \n",
"L 58.015625 25.390625 \n",
"L 58.015625 17.1875 \n",
"L 47.609375 17.1875 \n",
"L 47.609375 0 \n",
"L 37.796875 0 \n",
"L 37.796875 17.1875 \n",
"L 4.890625 17.1875 \n",
"L 4.890625 26.703125 \n",
"z\n",
"\" id=\"DejaVuSans-34\"/>\n",
" </defs>\n",
" <g transform=\"translate(56.909507 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_2\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"128.387416\" xlink:href=\"#m97102b0cb2\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- −2 -->\n",
" <defs>\n",
" <path d=\"M 19.1875 8.296875 \n",
"L 53.609375 8.296875 \n",
"L 53.609375 0 \n",
"L 7.328125 0 \n",
"L 7.328125 8.296875 \n",
"Q 12.9375 14.109375 22.625 23.890625 \n",
"Q 32.328125 33.6875 34.8125 36.53125 \n",
"Q 39.546875 41.84375 41.421875 45.53125 \n",
"Q 43.3125 49.21875 43.3125 52.78125 \n",
"Q 43.3125 58.59375 39.234375 62.25 \n",
"Q 35.15625 65.921875 28.609375 65.921875 \n",
"Q 23.96875 65.921875 18.8125 64.3125 \n",
"Q 13.671875 62.703125 7.8125 59.421875 \n",
"L 7.8125 69.390625 \n",
"Q 13.765625 71.78125 18.9375 73 \n",
"Q 24.125 74.21875 28.421875 74.21875 \n",
"Q 39.75 74.21875 46.484375 68.546875 \n",
"Q 53.21875 62.890625 53.21875 53.421875 \n",
"Q 53.21875 48.921875 51.53125 44.890625 \n",
"Q 49.859375 40.875 45.40625 35.40625 \n",
"Q 44.1875 33.984375 37.640625 27.21875 \n",
"Q 31.109375 20.453125 19.1875 8.296875 \n",
"z\n",
"\" id=\"DejaVuSans-32\"/>\n",
" </defs>\n",
" <g transform=\"translate(121.016322 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_3\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"192.494231\" xlink:href=\"#m97102b0cb2\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 0 -->\n",
" <defs>\n",
" <path d=\"M 31.78125 66.40625 \n",
"Q 24.171875 66.40625 20.328125 58.90625 \n",
"Q 16.5 51.421875 16.5 36.375 \n",
"Q 16.5 21.390625 20.328125 13.890625 \n",
"Q 24.171875 6.390625 31.78125 6.390625 \n",
"Q 39.453125 6.390625 43.28125 13.890625 \n",
"Q 47.125 21.390625 47.125 36.375 \n",
"Q 47.125 51.421875 43.28125 58.90625 \n",
"Q 39.453125 66.40625 31.78125 66.40625 \n",
"z\n",
"M 31.78125 74.21875 \n",
"Q 44.046875 74.21875 50.515625 64.515625 \n",
"Q 56.984375 54.828125 56.984375 36.375 \n",
"Q 56.984375 17.96875 50.515625 8.265625 \n",
"Q 44.046875 -1.421875 31.78125 -1.421875 \n",
"Q 19.53125 -1.421875 13.0625 8.265625 \n",
"Q 6.59375 17.96875 6.59375 36.375 \n",
"Q 6.59375 54.828125 13.0625 64.515625 \n",
"Q 19.53125 74.21875 31.78125 74.21875 \n",
"z\n",
"\" id=\"DejaVuSans-30\"/>\n",
" </defs>\n",
" <g transform=\"translate(189.312981 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"256.601046\" xlink:href=\"#m97102b0cb2\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(253.419796 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_5\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"320.707862\" xlink:href=\"#m97102b0cb2\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(317.526612 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_6\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" id=\"m33fc612b7d\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m33fc612b7d\" y=\"210.026074\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- −4 -->\n",
" <g transform=\"translate(7.2 213.825293)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_7\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m33fc612b7d\" y=\"164.107625\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- −2 -->\n",
" <g transform=\"translate(7.2 167.906844)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m33fc612b7d\" y=\"118.189176\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(15.579688 121.988395)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_9\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m33fc612b7d\" y=\"72.270727\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(15.579688 76.069946)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m33fc612b7d\" y=\"26.352278\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(15.579688 30.151497)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 28.942188 228.14 \n",
"L 28.942188 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 363.742188 228.14 \n",
"L 363.742188 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 28.942187 228.14 \n",
"L 363.742188 228.14 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 28.942187 10.7 \n",
"L 363.742188 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p00c0127347\">\n",
" <rect height=\"217.44\" width=\"334.8\" x=\"28.942188\" y=\"10.7\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Created with matplotlib (http://matplotlib.org/) -->\n",
"<svg height=\"252.018125pt\" version=\"1.1\" viewBox=\"0 0 374.442187 252.018125\" width=\"374.442187pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
" <defs>\n",
" <style type=\"text/css\">\n",
"*{stroke-linecap:butt;stroke-linejoin:round;}\n",
" </style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 252.018125 \n",
"L 374.442187 252.018125 \n",
"L 374.442187 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill:none;\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 28.942188 228.14 \n",
"L 363.742188 228.14 \n",
"L 363.742188 10.7 \n",
"L 28.942188 10.7 \n",
"z\n",
"\" style=\"fill:#ffffff;\"/>\n",
" </g>\n",
" <g id=\"PathCollection_1\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"ma2229ffcfb\" style=\"stroke:#0000ff;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#pca37c45639)\">\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"211.28696\" xlink:href=\"#ma2229ffcfb\" y=\"99.74324\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"203.424703\" xlink:href=\"#ma2229ffcfb\" y=\"139.732342\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_2\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"mfa1b62e73f\" style=\"stroke:#ff0000;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#pca37c45639)\">\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"211.205927\" xlink:href=\"#mfa1b62e73f\" y=\"217.63985\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"348.440848\" xlink:href=\"#mfa1b62e73f\" y=\"103.541504\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"128.121904\" xlink:href=\"#mfa1b62e73f\" y=\"169.60204\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"191.524708\" xlink:href=\"#mfa1b62e73f\" y=\"210.584618\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"44.243527\" xlink:href=\"#mfa1b62e73f\" y=\"117.193002\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"213.850904\" xlink:href=\"#mfa1b62e73f\" y=\"22.397617\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"238.15351\" xlink:href=\"#mfa1b62e73f\" y=\"21.552302\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"200.08938\" xlink:href=\"#mfa1b62e73f\" y=\"200.290452\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"267.92283\" xlink:href=\"#mfa1b62e73f\" y=\"21.20015\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"313.151828\" xlink:href=\"#mfa1b62e73f\" y=\"64.598802\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_3\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"ma76c1c8976\" style=\"stroke:#00ffff;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#pca37c45639)\">\n",
" <use style=\"fill:#00ffff;stroke:#00ffff;\" x=\"211.28696\" xlink:href=\"#ma76c1c8976\" y=\"99.74324\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" id=\"m9057646305\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"64.278468\" xlink:href=\"#m9057646305\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- −4 -->\n",
" <defs>\n",
" <path d=\"M 10.59375 35.5 \n",
"L 73.1875 35.5 \n",
"L 73.1875 27.203125 \n",
"L 10.59375 27.203125 \n",
"z\n",
"\" id=\"DejaVuSans-2212\"/>\n",
" <path d=\"M 37.796875 64.3125 \n",
"L 12.890625 25.390625 \n",
"L 37.796875 25.390625 \n",
"z\n",
"M 35.203125 72.90625 \n",
"L 47.609375 72.90625 \n",
"L 47.609375 25.390625 \n",
"L 58.015625 25.390625 \n",
"L 58.015625 17.1875 \n",
"L 47.609375 17.1875 \n",
"L 47.609375 0 \n",
"L 37.796875 0 \n",
"L 37.796875 17.1875 \n",
"L 4.890625 17.1875 \n",
"L 4.890625 26.703125 \n",
"z\n",
"\" id=\"DejaVuSans-34\"/>\n",
" </defs>\n",
" <g transform=\"translate(56.907374 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_2\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"128.386318\" xlink:href=\"#m9057646305\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- −2 -->\n",
" <defs>\n",
" <path d=\"M 19.1875 8.296875 \n",
"L 53.609375 8.296875 \n",
"L 53.609375 0 \n",
"L 7.328125 0 \n",
"L 7.328125 8.296875 \n",
"Q 12.9375 14.109375 22.625 23.890625 \n",
"Q 32.328125 33.6875 34.8125 36.53125 \n",
"Q 39.546875 41.84375 41.421875 45.53125 \n",
"Q 43.3125 49.21875 43.3125 52.78125 \n",
"Q 43.3125 58.59375 39.234375 62.25 \n",
"Q 35.15625 65.921875 28.609375 65.921875 \n",
"Q 23.96875 65.921875 18.8125 64.3125 \n",
"Q 13.671875 62.703125 7.8125 59.421875 \n",
"L 7.8125 69.390625 \n",
"Q 13.765625 71.78125 18.9375 73 \n",
"Q 24.125 74.21875 28.421875 74.21875 \n",
"Q 39.75 74.21875 46.484375 68.546875 \n",
"Q 53.21875 62.890625 53.21875 53.421875 \n",
"Q 53.21875 48.921875 51.53125 44.890625 \n",
"Q 49.859375 40.875 45.40625 35.40625 \n",
"Q 44.1875 33.984375 37.640625 27.21875 \n",
"Q 31.109375 20.453125 19.1875 8.296875 \n",
"z\n",
"\" id=\"DejaVuSans-32\"/>\n",
" </defs>\n",
" <g transform=\"translate(121.015225 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_3\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"192.494169\" xlink:href=\"#m9057646305\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 0 -->\n",
" <defs>\n",
" <path d=\"M 31.78125 66.40625 \n",
"Q 24.171875 66.40625 20.328125 58.90625 \n",
"Q 16.5 51.421875 16.5 36.375 \n",
"Q 16.5 21.390625 20.328125 13.890625 \n",
"Q 24.171875 6.390625 31.78125 6.390625 \n",
"Q 39.453125 6.390625 43.28125 13.890625 \n",
"Q 47.125 21.390625 47.125 36.375 \n",
"Q 47.125 51.421875 43.28125 58.90625 \n",
"Q 39.453125 66.40625 31.78125 66.40625 \n",
"z\n",
"M 31.78125 74.21875 \n",
"Q 44.046875 74.21875 50.515625 64.515625 \n",
"Q 56.984375 54.828125 56.984375 36.375 \n",
"Q 56.984375 17.96875 50.515625 8.265625 \n",
"Q 44.046875 -1.421875 31.78125 -1.421875 \n",
"Q 19.53125 -1.421875 13.0625 8.265625 \n",
"Q 6.59375 17.96875 6.59375 36.375 \n",
"Q 6.59375 54.828125 13.0625 64.515625 \n",
"Q 19.53125 74.21875 31.78125 74.21875 \n",
"z\n",
"\" id=\"DejaVuSans-30\"/>\n",
" </defs>\n",
" <g transform=\"translate(189.312919 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"256.602019\" xlink:href=\"#m9057646305\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(253.420769 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_5\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"320.70987\" xlink:href=\"#m9057646305\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(317.52862 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_6\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" id=\"m4cd1292a67\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m4cd1292a67\" y=\"210.027367\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- −4 -->\n",
" <g transform=\"translate(7.2 213.826586)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_7\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m4cd1292a67\" y=\"164.108263\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- −2 -->\n",
" <g transform=\"translate(7.2 167.907481)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m4cd1292a67\" y=\"118.189158\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(15.579688 121.988377)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_9\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m4cd1292a67\" y=\"72.270054\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(15.579688 76.069273)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m4cd1292a67\" y=\"26.35095\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(15.579688 30.150169)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 28.942188 228.14 \n",
"L 28.942188 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 363.742188 228.14 \n",
"L 363.742188 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 28.942188 228.14 \n",
"L 363.742188 228.14 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 28.942188 10.7 \n",
"L 363.742188 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"pca37c45639\">\n",
" <rect height=\"217.44\" width=\"334.8\" x=\"28.942188\" y=\"10.7\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Created with matplotlib (http://matplotlib.org/) -->\n",
"<svg height=\"252.018125pt\" version=\"1.1\" viewBox=\"0 0 374.442187 252.018125\" width=\"374.442187pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
" <defs>\n",
" <style type=\"text/css\">\n",
"*{stroke-linecap:butt;stroke-linejoin:round;}\n",
" </style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 252.018125 \n",
"L 374.442187 252.018125 \n",
"L 374.442187 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill:none;\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 28.942188 228.14 \n",
"L 363.742188 228.14 \n",
"L 363.742188 10.7 \n",
"L 28.942188 10.7 \n",
"z\n",
"\" style=\"fill:#ffffff;\"/>\n",
" </g>\n",
" <g id=\"PathCollection_1\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"md666680931\" style=\"stroke:#0000ff;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p00597c0688)\">\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"211.287035\" xlink:href=\"#md666680931\" y=\"99.743276\"/>\n",
" <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"203.42531\" xlink:href=\"#md666680931\" y=\"139.732184\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_2\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"m8300658316\" style=\"stroke:#ff0000;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p00597c0688)\">\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"211.205927\" xlink:href=\"#m8300658316\" y=\"217.639853\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"348.440853\" xlink:href=\"#m8300658316\" y=\"103.541504\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"128.121902\" xlink:href=\"#m8300658316\" y=\"169.602042\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"191.524708\" xlink:href=\"#m8300658316\" y=\"210.584621\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"44.243522\" xlink:href=\"#m8300658316\" y=\"117.193002\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"213.850905\" xlink:href=\"#m8300658316\" y=\"22.397614\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"238.153511\" xlink:href=\"#m8300658316\" y=\"21.552299\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"200.08938\" xlink:href=\"#m8300658316\" y=\"200.290455\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"267.922832\" xlink:href=\"#m8300658316\" y=\"21.200147\"/>\n",
" <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"313.151832\" xlink:href=\"#m8300658316\" y=\"64.598801\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_3\">\n",
" <defs>\n",
" <path d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" id=\"mb249b8220b\" style=\"stroke:#00ffff;\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p00597c0688)\">\n",
" <use style=\"fill:#00ffff;stroke:#00ffff;\" x=\"211.287035\" xlink:href=\"#mb249b8220b\" y=\"99.743276\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" id=\"m0293ec3709\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"64.278463\" xlink:href=\"#m0293ec3709\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- −4 -->\n",
" <defs>\n",
" <path d=\"M 10.59375 35.5 \n",
"L 73.1875 35.5 \n",
"L 73.1875 27.203125 \n",
"L 10.59375 27.203125 \n",
"z\n",
"\" id=\"DejaVuSans-2212\"/>\n",
" <path d=\"M 37.796875 64.3125 \n",
"L 12.890625 25.390625 \n",
"L 37.796875 25.390625 \n",
"z\n",
"M 35.203125 72.90625 \n",
"L 47.609375 72.90625 \n",
"L 47.609375 25.390625 \n",
"L 58.015625 25.390625 \n",
"L 58.015625 17.1875 \n",
"L 47.609375 17.1875 \n",
"L 47.609375 0 \n",
"L 37.796875 0 \n",
"L 37.796875 17.1875 \n",
"L 4.890625 17.1875 \n",
"L 4.890625 26.703125 \n",
"z\n",
"\" id=\"DejaVuSans-34\"/>\n",
" </defs>\n",
" <g transform=\"translate(56.90737 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_2\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"128.386316\" xlink:href=\"#m0293ec3709\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- −2 -->\n",
" <defs>\n",
" <path d=\"M 19.1875 8.296875 \n",
"L 53.609375 8.296875 \n",
"L 53.609375 0 \n",
"L 7.328125 0 \n",
"L 7.328125 8.296875 \n",
"Q 12.9375 14.109375 22.625 23.890625 \n",
"Q 32.328125 33.6875 34.8125 36.53125 \n",
"Q 39.546875 41.84375 41.421875 45.53125 \n",
"Q 43.3125 49.21875 43.3125 52.78125 \n",
"Q 43.3125 58.59375 39.234375 62.25 \n",
"Q 35.15625 65.921875 28.609375 65.921875 \n",
"Q 23.96875 65.921875 18.8125 64.3125 \n",
"Q 13.671875 62.703125 7.8125 59.421875 \n",
"L 7.8125 69.390625 \n",
"Q 13.765625 71.78125 18.9375 73 \n",
"Q 24.125 74.21875 28.421875 74.21875 \n",
"Q 39.75 74.21875 46.484375 68.546875 \n",
"Q 53.21875 62.890625 53.21875 53.421875 \n",
"Q 53.21875 48.921875 51.53125 44.890625 \n",
"Q 49.859375 40.875 45.40625 35.40625 \n",
"Q 44.1875 33.984375 37.640625 27.21875 \n",
"Q 31.109375 20.453125 19.1875 8.296875 \n",
"z\n",
"\" id=\"DejaVuSans-32\"/>\n",
" </defs>\n",
" <g transform=\"translate(121.015222 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_3\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"192.494169\" xlink:href=\"#m0293ec3709\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 0 -->\n",
" <defs>\n",
" <path d=\"M 31.78125 66.40625 \n",
"Q 24.171875 66.40625 20.328125 58.90625 \n",
"Q 16.5 51.421875 16.5 36.375 \n",
"Q 16.5 21.390625 20.328125 13.890625 \n",
"Q 24.171875 6.390625 31.78125 6.390625 \n",
"Q 39.453125 6.390625 43.28125 13.890625 \n",
"Q 47.125 21.390625 47.125 36.375 \n",
"Q 47.125 51.421875 43.28125 58.90625 \n",
"Q 39.453125 66.40625 31.78125 66.40625 \n",
"z\n",
"M 31.78125 74.21875 \n",
"Q 44.046875 74.21875 50.515625 64.515625 \n",
"Q 56.984375 54.828125 56.984375 36.375 \n",
"Q 56.984375 17.96875 50.515625 8.265625 \n",
"Q 44.046875 -1.421875 31.78125 -1.421875 \n",
"Q 19.53125 -1.421875 13.0625 8.265625 \n",
"Q 6.59375 17.96875 6.59375 36.375 \n",
"Q 6.59375 54.828125 13.0625 64.515625 \n",
"Q 19.53125 74.21875 31.78125 74.21875 \n",
"z\n",
"\" id=\"DejaVuSans-30\"/>\n",
" </defs>\n",
" <g transform=\"translate(189.312919 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"256.602021\" xlink:href=\"#m0293ec3709\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(253.420771 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_5\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"320.709874\" xlink:href=\"#m0293ec3709\" y=\"228.14\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(317.528624 242.738437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_6\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" id=\"m707957e5ee\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m707957e5ee\" y=\"210.02737\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- −4 -->\n",
" <g transform=\"translate(7.2 213.826588)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_7\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m707957e5ee\" y=\"164.108264\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- −2 -->\n",
" <g transform=\"translate(7.2 167.907483)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m707957e5ee\" y=\"118.189158\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(15.579688 121.988377)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_9\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m707957e5ee\" y=\"72.270053\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(15.579688 76.069271)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"28.942188\" xlink:href=\"#m707957e5ee\" y=\"26.350947\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(15.579688 30.150166)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 28.942188 228.14 \n",
"L 28.942188 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 363.742188 228.14 \n",
"L 363.742188 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 28.942187 228.14 \n",
"L 363.742187 228.14 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 28.942187 10.7 \n",
"L 363.742187 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p00597c0688\">\n",
" <rect height=\"217.44\" width=\"334.8\" x=\"28.942188\" y=\"10.7\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"for feats in selected_node_feats:\n",
" plt.scatter(*feats.T,color='blue')\n",
" plt.scatter(*points_1, color='red')\n",
" plt.scatter(*feats[0].T, color='aqua')\n",
" plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.6",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.1"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
tutorial/capsule/Untitled.ipynb 0 → 100644
View file @ a50bbe58
{
"cells": [
{
"cell_type": "code",
"execution_count": 40,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"env: DGL_LIBRARY_PATH=/data/jinjing/dgl/build\n"
]
}
],
"source": [
"%env DGL_LIBRARY_PATH=/data/jinjing/dgl/build"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import torch as th\n",
"import numpy as np\n",
"import torch.nn as nn\n",
"import torch.nn.functional as F"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [],
"source": [
"num_points=10\n",
"num_points2=10"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [],
"source": [
"theta_dist1=np.pi*(np.random.rand(num_points)*2)\n",
"theta_dist2=np.pi*(np.random.rand(num_points2)*2)\n",
"\n",
"radius_1=np.random.rand(num_points)\n",
"radius_2=np.random.rand(num_points2)*2+3"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [],
"source": [
"points_1=np.array([np.cos(theta_dist1)*radius_1,np.sin(theta_dist1)*radius_1])"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [],
"source": [
"points_2=np.array([np.cos(theta_dist2)*radius_2,np.sin(theta_dist2)*radius_2])"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Populating the interactive namespace from numpy and matplotlib\n"
]
}
],
"source": [
"%pylab inline"
]
},
{
"cell_type": "code",
"execution_count": 73,
"metadata": {},
"outputs": [],
"source": [
"point_3=np.array([1,1])"
]
},
{
"cell_type": "code",
"execution_count": 74,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.collections.PathCollection at 0x7fd47d377cf8>"
]
},
"execution_count": 74,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAEBlJREFUeJzt3X2IXNd9xvHnkSKTbGIwQZM4tbQ7hpqAUB27DCLBhVLFCYprYhII2EwCIYH9pwEHDCbuQktaBAVDmkICYWiUUDrEFBLh4JfaMlUwhvhl5UqqHMnBGK8skaI1JdhmIYmsX/+4M9Vqu28z98zcmXO/HxCz9+7l3B/S6uHsueee44gQACAfO6ouAACQFsEOAJkh2AEgMwQ7AGSGYAeAzBDsAJAZgh0AMkOwA0BmCHYAyMz7qrjp7t27o9lsVnFrAJhaJ06ceCsiGltdV0mwN5tNLS4uVnFrAJhatpe2cx1DMQCQmSQ9dttvSHpH0nuSLkdEK0W7AIDBpRyK+YuIeCthewCAITAUAwCZSRXsIelp2ydszydqEwAwhFRDMX8WERdtf0TSMdvnIuLZ1Rf0An9ekmZnZxPdFgCwVpIee0Rc7H1eknRU0oF1rulERCsiWo3GltMwgaS6XanZlHbsKD673aorAkandLDb/qDt6/tfS/qspDNl2wVS6Xal+XlpaUmKKD7n5wl35CtFj/2jkp6zfUrSi5Iej4h/T9AukMTCgrSycu25lZXiPJCj0mPsEfG6pE8kqAUYifPnBzsPTDumOyJ7Gz2r5xk+ckWwI3uHD0szM9eem5kpzgM5ItiRvXZb6nSkuTnJLj47neI8kKNKVncExq3dJshRH/TYASAzBDsAZIZgB4DMEOwARorlHMaPYAdqooqAZTmHahDsQA1UFbAs51ANgh2ogaoCluUcqkGwAzVQVcCynEM1CHagBqoKWJZzqAbBDtRAVQHLcg7VYEkBoAb6QbqwUAy/zM4WoT6OgGU5h/Ej2IGaIGDrg6EYAMgMwQ4AmUkW7LZ32v5P24+lahMAMLiUPfb7JZ1N2B4AYAhJgt32Hkl/KemfU7QHABheqh77dyU9KOnKRhfYnre9aHtxeXk50W0BAGuVDnbbd0u6FBEnNrsuIjoR0YqIVqPRKHtbAMAGUvTY75D0edtvSHpE0kHb/5qgXQDAEEoHe0Q8FBF7IqIp6V5J/xERXy5dGQBgKMxjB4DMJF1SICJ+IekXKdsEAAyGHjuA7NVt31UWAQOQtf62gP0dpPrbAkr5LopGjx1A1uq47yrBDiBrddx3lWAHkLU67rtKsAPIWh33XSXYUUt1myVRZ3Xcd5VZMaidOs6SqLu6bQtIjx21U8dZEqgXgh21U8dZEqgXgh21U8dZEqgXgh21U8dZEqgXgh21U8dZEqgXZsWgluo2SwL1Qo8dADJDsANAZgh2AMhM6WC3/X7bL9o+ZfsV299OURgAYDgpHp7+TtLBiHjX9i5Jz9l+MiKeT9A2AGBApXvsUXi3d7ir9yfKtgsAw2CBt0Rj7LZ32j4p6ZKkYxHxQop2AWAQ/QXelpakiKsLvNUt3JMEe0S8FxG3Sdoj6YDt/WuvsT1ve9H24vLycorbAsA1WOCtkHRWTET8VtJxSYfW+V4nIloR0Wo0GilvCwCSWOCtL8WsmIbtG3pff0DSZySdK9suAAyKBd4KKXrsH5N03PZpSS+pGGN/LEG7ADAQFngrpJgVczoibo+IWyNif0T8XYrCMBhmAgAs8NbHImAZYKs34CoWeGNJgSwwEwDAagR7BpgJAGA1gj0DzAQAsBrBngFmAgBYjWDPADMBAKzGrJhMMBMAQB89dgDIDMEOAJkh2AEgMwQ7AGSGYAeAERv3Wk7MigGAEapiLSd67AAwQlWs5USwA8AIVbGWE8EOACNUxVpOBDsAjFAVazkR7AAwQlWs5VR6VoztvZL+RdJHJYWkTkT8U9l2ASAX417LKcV0x8uSHoiIl21fL+mE7WMR8asEbQMABpRiM+vfRMTLva/fkXRW0k1l2wUADCfpGLvtpqTbJb2Qsl0AwPYlC3bbH5L0U0nfjIi31/n+vO1F24vLy8upbgsAWCNJsNvepSLUuxHxs/WuiYhORLQiotVoNFLcFgCwjtLBbtuSfijpbER8p3xJAIAyUvTY75D0FUkHbZ/s/bkrQbsAgCGUnu4YEc9JcoJaAAAJ8OYpAGSGYAeAzBDsAJAZgh0AMkOwA0BmCHYAyAzBDgCZIdgBIDMEOwBkhmAHgMwQ7ACQGYIdADJDsANAZgh2AMgMwQ4AmSHYASAzBDsAZIZgB4DMJAl220dsX7J9JkV7AIDhpeqx/1jSoURtARijrqSmijBo9o4x3UpvZi1JEfGs7WaKtgCMT1fSvKSV3vFS71iS2pVUhBQYYwdqbEFXQ71vpXce02tswW573vai7cXl5eVx3RbAJs4PeB7TYWzBHhGdiGhFRKvRaIzrtgA2MTvgeUwHhmKAGjssaWbNuZneeUyvVNMdfyLpl5I+bvuC7a+naBfAaLUldSTNSXLvsyMenE67VLNi7kvRDoDxa4sgzw1DMcC4dLtSsynt2FF8dpkxjtFI0mMHsIVuV5qfl1Z6kwuXlopjSWrTX0Za9NiBcVhYuBrqfSsrxXkgMYIdGIfzG8wM3+g8UALBDozD7AYzwzc6D5RAsAPjcPiwNLNmxvjMTHEeSIxgB8ah3ZY6HWluTrKLz06HB6cYCWbFAOPSbhPkGAt67ACQGYIdADJDsANAZgh2AMgMwQ4AmSHYASAzBDswCilWcmQ1SAyJYAdS66/kuLQkRVxdybHb3X5Yb9YGsAWCHdjMML3mjVZyvP/+7Yc1q0GiBEfE2G/aarVicXFx7PcFBrJ2DXWpWN9lq6UAduwognu75uakN97YXhu2dOXK9ttGVmyfiIjWVtel2vP0kO1Xbb9m+1sp2gQqN2yv+cMfHuw+6y3dy2qQKKF0sNveKen7kj4naZ+k+2zvK9suULlh1lDvdqW33x7sPuuFNatBooQUPfYDkl6LiNcj4veSHpF0T4J2gWoN02teWJD+8Ift32OjsGY1SJSQIthvkvTmquMLvXPXsD1ve9H24vLycoLbAiM2TK950B2RNgvrdrsYe79ypfgk1LFNY5sVExGdiGhFRKvRaIzrtsDwhuk1DzIGPjdHWGMkUgT7RUl7Vx3v6Z0Dpt+gveb1evm7dknXXXftOcbLMUIpgv0lSbfYvtn2dZLulfTzBO0C02e9Xv6PfiQdOcJ4OcYmyTx223dJ+q6knZKORMSmXRHmsWOqdbvFQ9Lz54uhl8OHCWmMxXbnsSfZGi8inpD0RIq2gIm29qWl/tujEuGOicGSAsAgeNUfU4BgBwYxzEtLwJgR7MAgeNUfU4BgBwbBq/6YAgQ7MAhe9ccUSDIrBqiVdpsgx0Sjxw4AmSHYgdWmeZ/Raa4dSU1VsPNzi5Ga5n1Gp7l2JDc1W+MNu0sZsG3NZhGIa623dd2kmebasW3bXVJgaoKdn1uM3DTvMzrNtWPbxrrn6Tjwwh9GbppfPprm2pHc1AQ7P7cYuWl++Wiaa0dyUxPs/Nxi5Kb55aNprh3JTc0Yu8Qy2ADqbazrsY8LL/wBwNamZigGALA9pYLd9pdsv2L7iu0tfz0AAIxe2R77GUlflPRsgloAAAmUGmOPiLOSZDtNNQCA0hhjB4DMbNljt/2MpBvX+dZCRDy63RvZnpc0L0mzvFUEACOzZbBHxJ0pbhQRHUkdqZjHnqJNAMD/x1AMAGSm7HTHL9i+IOlTkh63/VSasgAAwyo7K+aopKOJagEAJMBQDABkhmAHgMwQ7ACQGYIdADJDsANAZgh2AMgMwQ4AmSHYASAzBDsAZIZgB4DMEOwAkBmCHQAyQ7ADQGYIdgDIDMEOAJkh2AEgMwQ7AGSGYAeAzJTd8/Rh2+dsn7Z91PYNqQoDAAynbI/9mKT9EXGrpF9Leqh8SQCAMkoFe0Q8HRGXe4fPS9pTviQAQBkpx9i/JunJhO0BAIbwvq0usP2MpBvX+dZCRDzau2ZB0mVJ3U3amZc0L0mzs7NDFQsA2NqWwR4Rd272fdtflXS3pE9HRGzSTkdSR5JardaG1wEAytky2Ddj+5CkByX9eUSspCkJAFBG2TH270m6XtIx2ydt/yBBTQCAEkr12CPij1MVAgBIgzdPASAzBDsAZIZgB4DMEOwAkBmCHQAyQ7AjiW5XajalHTuKz+6G7yADGLVS0x0BqQjx+XlppfeK2tJScSxJ7XZ1dQF1RY8dpS0sXA31vpWV4jyA8SPYUdr584OdBzBaBDtK22ixThbxBKpBsKO0w4elmZlrz83MFOcBjB/BjtLabanTkebmJLv47HR4cApUhVkxSKLdJsiBSUGPHQAyQ7ADQGYIdgDIDMEOAJkh2AEgMwQ7AGTGETH+m9rLkpbGfuOrdkt6q8L7b2aSa5Mmu75Jrk2ivjImuTZpfPXNRURjq4sqCfaq2V6MiFbVdaxnkmuTJru+Sa5Nor4yJrk2afLqYygGADJDsANAZuoa7J2qC9jEJNcmTXZ9k1ybRH1lTHJt0oTVV8sxdgDIWV177ACQrdoHu+0HbIft3VXX0mf7722ftn3S9tO2/6jqmlaz/bDtc70aj9q+oeqa+mx/yfYrtq/YnphZCrYP2X7V9mu2v1V1PavZPmL7ku0zVdeylu29to/b/lXv3/X+qmvqs/1+2y/aPtWr7dtV19RX62C3vVfSZyVN2iZuD0fErRFxm6THJP1N1QWtcUzS/oi4VdKvJT1UcT2rnZH0RUnPVl1In+2dkr4v6XOS9km6z/a+aqu6xo8lHaq6iA1clvRAROyT9ElJfzVBf3e/k3QwIj4h6TZJh2x/suKaJNU82CX9o6QHJU3Ug4aIeHvV4Qc1efU9HRGXe4fPS9pTZT2rRcTZiHi16jrWOCDptYh4PSJ+L+kRSfdUXNP/iYhnJf1P1XWsJyJ+ExEv975+R9JZSTdVW1UhCu/2Dnf1/kzE/9XaBrvteyRdjIhTVdeyHtuHbb8pqa3J67Gv9jVJT1ZdxIS7SdKbq44vaELCaZrYbkq6XdIL1VZyle2dtk9KuiTpWERMRG1Z76Bk+xlJN67zrQVJf61iGKYSm9UWEY9GxIKkBdsPSfqGpL+dpPp61yyo+FW5O2m1IS+2PyTpp5K+ueY32kpFxHuSbus9Zzpqe39EVP6sIutgj4g71ztv+08k3SzplG2pGEp42faBiPjvKmtbR1fSExpzsG9Vn+2vSrpb0qdjzHNmB/i7mxQXJe1ddbyndw7bYHuXilDvRsTPqq5nPRHxW9vHVTyrqDzYazkUExH/FREfiYhmRDRV/Gr8p+MK9a3YvmXV4T2SzlVVy3psH1LxbOLzEbFSdT1T4CVJt9i+2fZ1ku6V9POKa5oKLnpeP5R0NiK+U3U9q9lu9GeE2f6ApM9oQv6v1jLYp8A/2D5j+7SK4aKJmeLV8z1J10s61puS+YOqC+qz/QXbFyR9StLjtp+quqbeg+ZvSHpKxcO/f4uIV6qt6irbP5H0S0kft33B9terrmmVOyR9RdLB3s/aSdt3VV1Uz8ckHe/9P31JxRj7YxXXJIk3TwEgO/TYASAzBDsAZIZgB4DMEOwAkBmCHQAyQ7ADQGYIdgDIDMEOAJn5XxKAS7tplJrgAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.scatter(*points_1,color='red')\n",
"plt.scatter(*points_2,color='blue')\n",
"plt.scatter(*point_3,color='aqua')"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {},
"outputs": [],
"source": [
"import torch as th\n",
"import dgl\n",
"import networkx as nx"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {},
"outputs": [],
"source": [
"g=dgl.DGLGraph()"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {},
"outputs": [],
"source": [
"g.add_nodes(num_points+num_points2)"
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {},
"outputs": [],
"source": [
"for i in range(num_points):\n",
" for j in range(num_points2):\n",
" g.add_edge(i,j+num_points)"
]
},
{
"cell_type": "code",
"execution_count": 80,
"metadata": {},
"outputs": [],
"source": [
"nodes1=list(range(num_points))\n",
"nodes2=list(range(num_points,num_points+num_points2))"
]
},
{
"cell_type": "code",
"execution_count": 75,
"metadata": {},
"outputs": [],
"source": [
"g.set_n_repr({'h':th.from_numpy(np.concatenate([points_1,points_2],axis=1).T)})"
]
},
{
"cell_type": "code",
"execution_count": 115,
"metadata": {},
"outputs": [],
"source": [
"g.update_edge(edge_func=lambda u,v,edge:{'u_hat':u['h']})"
]
},
{
"cell_type": "code",
"execution_count": 76,
"metadata": {},
"outputs": [],
"source": [
"g.set_e_repr({'b_ij':th.zeros(100)})\n",
"g.set_e_repr({'u_hat':})"
]
},
{
"cell_type": "code",
"execution_count": 119,
"metadata": {},
"outputs": [],
"source": [
"def update_edge(self, u, v, edge):\n",
" return {'b_ij': edge['b_ij'] + (v['h'] * edge['u_hat']).mean(dim=1).sum(dim=1)}"
]
},
{
"cell_type": "code",
"execution_count": 120,
"metadata": {},
"outputs": [],
"source": [
"def v2_reduce(node, msg):\n",
" b_ij_c, u_hat = msg['b_ij'], msg['u_hat']\n",
" c_i = F.softmax(b_ij_c, dim=0)\n",
" s_j = (c_i.unsqueeze(2).unsqueeze(3) * u_hat).sum(dim=1)\n",
" return {'h': s_j}"
]
},
{
"cell_type": "code",
"execution_count": 121,
"metadata": {},
"outputs": [],
"source": [
"def capsule_msg(src, edge):\n",
" return {'b_ij': edge['b_ij'], 'h': src['h'], 'u_hat': edge['u_hat']}\n"
]
},
{
"cell_type": "code",
"execution_count": 124,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"20"
]
},
"execution_count": 124,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"g.number_of_nodes()"
]
},
{
"cell_type": "code",
"execution_count": 123,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"> \u001b[0;32m<ipython-input-120-f6b14912c0b0>\u001b[0m(4)\u001b[0;36mv2_reduce\u001b[0;34m()\u001b[0m\n",
"\u001b[0;32m 1 \u001b[0;31m\u001b[0;32mdef\u001b[0m \u001b[0mv2_reduce\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmsg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0m\u001b[0;32m 2 \u001b[0;31m \u001b[0mb_ij_c\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mu_hat\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmsg\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'b_ij'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmsg\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'u_hat'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0m\u001b[0;32m 3 \u001b[0;31m \u001b[0mc_i\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mF\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msoftmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mb_ij_c\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdim\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0m\u001b[0;32m----> 4 \u001b[0;31m \u001b[0ms_j\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mc_i\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0munsqueeze\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0munsqueeze\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mu_hat\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msum\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdim\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0m\u001b[0;32m 5 \u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m'h'\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0ms_j\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0m\n",
"ipdb> c_i.shape\n",
"torch.Size([10, 40])\n",
"ipdb> u_hat.shape\n",
"torch.Size([10, 40, 2])\n",
"ipdb> q\n"
]
},
{
"ename": "RuntimeError",
"evalue": "The size of tensor a (50) must match the size of tensor b (10) at non-singleton dimension 1",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mRuntimeError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-123-847b35d9c410>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0mget_ipython\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_line_magic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'debug'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m''\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mg\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate_all\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcapsule_msg\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mv2_reduce\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;32m~/.pyenv/versions/3.6.1/lib/python3.6/site-packages/dgl-0.0.1-py3.6.egg/dgl/graph.py\u001b[0m in \u001b[0;36mupdate_all\u001b[0;34m(self, message_func, reduce_func, apply_node_func)\u001b[0m\n\u001b[1;32m 1138\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1139\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mALL\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mALL\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmessage_func\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1140\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrecv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mALL\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mreduce_func\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1141\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapply_nodes\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mALL\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mapply_node_func\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1142\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m~/.pyenv/versions/3.6.1/lib/python3.6/site-packages/dgl-0.0.1-py3.6.egg/dgl/graph.py\u001b[0m in \u001b[0;36mrecv\u001b[0;34m(self, u, reduce_func, apply_node_func)\u001b[0m\n\u001b[1;32m 944\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mreduce_func\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mlist\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtuple\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 945\u001b[0m \u001b[0mreduce_func\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mBundledReduceFunction\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mreduce_func\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 946\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_batch_recv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mu\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mreduce_func\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 947\u001b[0m \u001b[0;31m# optional apply nodes\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 948\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapply_nodes\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mu\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mapply_node_func\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m~/.pyenv/versions/3.6.1/lib/python3.6/site-packages/dgl-0.0.1-py3.6.egg/dgl/graph.py\u001b[0m in \u001b[0;36m_batch_recv\u001b[0;34m(self, v, reduce_func)\u001b[0m\n\u001b[1;32m 990\u001b[0m lambda key: _reshape_fn(in_msgs[key]), self._msg_frame.schemes)\n\u001b[1;32m 991\u001b[0m \u001b[0mreordered_v\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mv_bkt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtousertensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 992\u001b[0;31m \u001b[0mnew_reprs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mreduce_func\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdst_reprs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mreshaped_in_msgs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 993\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 994\u001b[0m \u001b[0;31m# TODO: clear partial messages\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m<ipython-input-120-f6b14912c0b0>\u001b[0m in \u001b[0;36mv2_reduce\u001b[0;34m(node, msg)\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0mb_ij_c\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mu_hat\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmsg\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'b_ij'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmsg\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'u_hat'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0mc_i\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mF\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msoftmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mb_ij_c\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdim\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0ms_j\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mc_i\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0munsqueeze\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0munsqueeze\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mu_hat\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msum\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdim\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 5\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m'h'\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0ms_j\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mRuntimeError\u001b[0m: The size of tensor a (50) must match the size of tensor b (10) at non-singleton dimension 1"
]
}
],
"source": [
"%debug\n",
"g.update_all(capsule_msg,v2_reduce,lambda x:x)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"g.set_e_repr({'u_hat':th.zeros()})"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"b_ij="
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"g.set_e_repr()"
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {},
"outputs": [],
"source": [
"nxg=g.to_networkx()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.6",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.1"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
tutorial/capsule/codes/CHANGELOG.md 0 → 100644
View file @ a50bbe58
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
## [0.4.0] - 2018-01-30
### Added
- Supports and works with CIFAR10 dataset.
### Changed
- Upgrade to PyTorch 0.3.0.
- Supports CUDA 9.
- Drop our custom softmax function and switch to PyTorch softmax function.
- Modify the save_image utils function to handle 3-channel (RGB) image.
### Fixed
- Compatibilities with PyTorch 0.3.0.
## [0.3.0] - 2017-11-27
### Added
- Decoder network PyTorch module.
- Reconstruct image with Decoder network during testing.
- Save the original and recontructed images into file system.
- Log the original and reconstructed images using TensorBoard.
### Changed
- Refactor reconstruction loss function and decoder network.
- Remove image reconstruction from training.
## [0.2.0] - 2017-11-26
### Added
- New dependencies for TensorBoard and tqdm.
- Logging losses and accuracies with TensorBoard.
- New utils functions for:
- computing accuracy
- convert values of the model parameters to numpy.array.
- parsing boolean values with argparse
- Softmax function that takes a dimension.
- More detailed code comments.
- Show margin loss and reconstruction loss in logs.
- Show accuracy in train logs.
### Changed
- Refactor loss functions.
- Clean codes.
### Fixed
- Runtime error during pip install requirements.txt
- Bug in routing algorithm.
## [0.1.0] - 2017-11-12
### Added
- Implemented reconstruction loss.
- Saving reconstructed image as file.
- Improve training speed by using PyTorch DataParallel to wrap our model.
- PyTorch will parallelized the model and data over multiple GPUs.
- Supports training:
- on CPU (tested with macOS Sierra)
- on one GPU (tested with 1 Tesla K80 GPU)
- on multiple GPU (tested with 8 GPUs)
- with or without CUDA (tested with CUDA version 8.0.61)
- cuDNN 5 (tested with cuDNN 5.1.3)
### Changed
- More intuitive variable naming.
### Fixed
- Resolve Pylint warnings and reformat code.
- Missing square in equation 4 for margin (class) loss.
## 0.0.1 - 2017-11-04
### Added
- Initial release. The first beta version. API is stable. The code runs. So, I think it's safe to use for development but not ready for general production usage.
[Unreleased]: https://github.com/cedrickchee/capsule-net-pytorch/compare/v1.0.0...HEAD
[0.1.0]: https://github.com/cedrickchee/capsule-net-pytorch/compare/v0.0.1...v0.1.0
[0.2.0]: https://github.com/cedrickchee/capsule-net-pytorch/compare/v0.1.0...v0.2.0
[0.3.0]: https://github.com/cedrickchee/capsule-net-pytorch/compare/v0.2.0...v0.3.0
[0.4.0]: https://github.com/cedrickchee/capsule-net-pytorch/compare/v0.3.0...v0.4.0
tutorial/capsule/codes/LICENSE 0 → 100644
View file @ a50bbe58
COPYRIGHT
All contributions by Cedric Chee:
Copyright (c) 2017, Cedric Chee.
All rights reserved.
All other contributions:
Copyright (c) 2017, the respective contributors.
All rights reserved.
Each contributor holds copyright over their respective contributions.
The project versioning (Git) records all such contribution source information.
LICENSE
The MIT License (MIT)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
tutorial/capsule/codes/README.md 0 → 100644
View file @ a50bbe58
# PyTorch CapsNet: Capsule Network for PyTorch
[![license](https://img.shields.io/github/license/mashape/apistatus.svg?maxAge=2592000)](https://github.com/cedrickchee/capsule-net-pytorch/blob/master/LICENSE)
![completion](https://img.shields.io/badge/completion%20state-95%25-green.svg?style=plastic)
A CUDA-enabled PyTorch implementation of CapsNet (Capsule Network) based on this paper:
[Sara Sabour, Nicholas Frosst, Geoffrey E Hinton. Dynamic Routing Between Capsules. NIPS 2017](https://arxiv.org/abs/1710.09829)
The current `test error is 0.21%` and the `best test error is 0.20%`. The current `test accuracy is 99.31%` and the `best test accuracy is 99.32%`.
**What is a Capsule**
> A Capsule is a group of neurons whose activity vector represents the instantiation parameters of a specific type of entity such as an object or object part.
You can learn more about Capsule Networks [here](#learning-resources).
**Why another CapsNet implementation?**
I wanted a decent PyTorch implementation of CapsNet and I couldn't find one at the point when I started. The goal of this implementation is focus to help newcomers learn and understand the CapsNet architecture and the idea of Capsules. The implementation is **NOT** focus on rigorous correctness of the results. In addition, the codes are not optimized for speed. To help us read and understand the codes easier, the codes comes with ample comments and the Python classes and functions are documented with Python docstring.
I will try my best to check and fix issues reported. Contributions are highly welcomed. If you find any bugs or errors in the codes, please do not hesitate to open an issue or a pull request. Thank you.
**Status and Latest Updates:**
See the [CHANGELOG](CHANGELOG.md)
**Datasets**
The model was trained on the standard [MNIST](http://yann.lecun.com/exdb/mnist/) data.
*Note: you don't have to manually download, preprocess, and load the MNIST dataset as [TorchVision](https://github.com/pytorch/vision) will take care of this step for you.*
I have tried using other datasets. See the [Other Datasets](#other-datasets) section below for more details.
## Requirements
- Python 3
- Tested with version 3.6.4
- [PyTorch](http://pytorch.org/)
- Tested with version 0.3.0.post4
- Migrate existing code to work in version 0.4.0. [Work-In-Progress]
- Code will not run with version 0.1.2 due to `keepdim` not available in this version.
- Code will not run with version 0.2.0 due to `softmax` function doesn't takes a dimension.
- CUDA 8 and above
- Tested with CUDA 8 and CUDA 9.
- [TorchVision](https://github.com/pytorch/vision)
- [tensorboardX](https://github.com/lanpa/tensorboard-pytorch)
- [tqdm](https://github.com/tqdm/tqdm)
## Usage
### Training and Evaluation
**Step 1.**
Clone this repository with ``git`` and install project dependencies.
```bash
$ git clone https://github.com/cedrickchee/capsule-net-pytorch.git
$ cd capsule-net-pytorch
$ pip install -r requirements.txt
```
**Step 2.**
Start the CapsNet on MNIST training and evaluation:
- Training with default settings:
```bash
$ python main.py
```
- Training on 8 GPUs with 30 epochs and 1 routing iteration:
```bash
$ CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python main.py --epochs 30 --num-routing 1 --threads 16 --batch-size 128 --test-batch-size 128
```
**Step 3.**
Test a pre-trained model:
If you have trained a model in Step 2 above, then the weights for the trained model will be saved to `results/trained_model/model_epoch_10.pth`. [WIP] Now just run the following command to get test results.
```bash
$ python main.py --is-training 0 --weights results/trained_model/model_epoch_10.pth
```
### Pre-trained Model
You can download the weights for the pre-trained model from my Google Drive. We saved the weights (model state dict) and the optimizer state for the model at the end of every training epoch.
- Weights from [epoch 50 checkpoint](https://drive.google.com/uc?export=download&id=1lYtOMSreP4I9hr9un4DsBJZrzodI6l2d) [84 MB].
- Weights from [epoch 40 to 50](https://drive.google.com/uc?export=download&id=1VMuVtJrecz47czsT5HqLxZpFjkLoMKaL) checkpoints [928 MB].
Uncompress and put the weights (.pth files) into `./results/trained_model/`.
*Note: the model was **last trained** on 2017-11-26 and the weights **last updated** on 2017-11-28.*
### The Default Hyper Parameters
| Parameter | Value | CLI arguments |
| --- | --- | --- |
| Training epochs | 10 | --epochs 10 |
| Learning rate | 0.01 | --lr 0.01 |
| Training batch size | 128 | --batch-size 128 |
| Testing batch size | 128 | --test-batch-size 128 |
| Log interval | 10 | --log-interval 10 |
| Disables CUDA training | false | --no-cuda |
| Num. of channels produced by the convolution | 256 | --num-conv-out-channel 256 |
| Num. of input channels to the convolution | 1 | --num-conv-in-channel 1 |
| Num. of primary unit | 8 | --num-primary-unit 8 |
| Primary unit size | 1152 | --primary-unit-size 1152 |
| Num. of digit classes | 10 | --num-classes 10 |
| Output unit size | 16 | --output-unit-size 16 |
| Num. routing iteration | 3 | --num-routing 3 |
| Use reconstruction loss | true | --use-reconstruction-loss |
| Regularization coefficient for reconstruction loss | 0.0005 | --regularization-scale 0.0005 |
| Dataset name (mnist, cifar10) | mnist | --dataset mnist |
| Input image width to the convolution | 28 | --input-width 28 |
| Input image height to the convolution | 28 | --input-height 28 |
## Results
### Test Error
CapsNet classification test error on MNIST. The MNIST average and standard deviation results are reported from 3 trials.
The results can be reproduced by running the following commands.
```bash
python main.py --epochs 50 --num-routing 1 --use-reconstruction-loss no --regularization-scale 0.0 #CapsNet-v1
python main.py --epochs 50 --num-routing 1 --use-reconstruction-loss yes --regularization-scale 0.0005 #CapsNet-v2
python main.py --epochs 50 --num-routing 3 --use-reconstruction-loss no --regularization-scale 0.0 #CapsNet-v3
python main.py --epochs 50 --num-routing 3 --use-reconstruction-loss yes --regularization-scale 0.0005 #CapsNet-v4
```
Method | Routing | Reconstruction | MNIST (%) | *Paper*
:---------|:------:|:---:|:----:|:----:
Baseline | -- | -- | -- | *0.39*
CapsNet-v1 | 1 | no | -- | *0.34 (0.032)*
CapsNet-v2 | 1 | yes | -- | *0.29 (0.011)*
CapsNet-v3 | 3 | no | -- | *0.35 (0.036)*
CapsNet-v4 | 3 | yes | 0.21 | *0.25 (0.005)*
### Training Loss and Accuracy
The training losses and accuracies for CapsNet-v4 (50 epochs, 3 routing iteration, using reconstruction, regularization scale of 0.0005):
![](results/train_loss_accuracy.png)
Training accuracy. Highest training accuracy: 100%
![](results/train_accuracy.png)
Training loss. Lowest training error: 0.1938%
![](results/train_loss.png)
### Test Loss and Accuracy
The test losses and accuracies for CapsNet-v4 (50 epochs, 3 routing iteration, using reconstruction, regularization scale of 0.0005):
![](results/test_loss_accuracy.png)
Test accuracy. Highest test accuracy: 99.32%
![](results/test_accuracy.png)
Test loss. Lowest test error: 0.2002%
![](results/test_loss.png)
### Training Speed
- Around `5.97s / batch` or `8min / epoch` on a single Tesla K80 GPU with batch size of 704.
- Around `3.25s / batch` or `25min / epoch` on a single Tesla K80 GPUwith batch size of 128.
![](results/training_speed.png)
In my case, these are the hyperparameters I used for the training setup:
- batch size: 128
- Epochs: 50
- Num. of routing: 3
- Use reconstruction loss: yes
- Regularization scale for reconstruction loss: 0.0005
### Reconstruction
The results of CapsNet-v4.
Digits at left are reconstructed images.
<table>
<tr>
<td>
<img src="results/reconstructed_images.png"/>
</td>
<td>
<p>[WIP] Ground truth image from dataset</p>
</td>
</tr>
</table>
### Model Design
```bash
Model architecture:
------------------
Net (
(conv1): ConvLayer (
(conv0): Conv2d(1, 256, kernel_size=(9, 9), stride=(1, 1))
(relu): ReLU (inplace)
)
(primary): CapsuleLayer (
(conv_units): ModuleList (
(0): Conv2d(256, 32, kernel_size=(9, 9), stride=(2, 2))
(1): Conv2d(256, 32, kernel_size=(9, 9), stride=(2, 2))
(2): Conv2d(256, 32, kernel_size=(9, 9), stride=(2, 2))
(3): Conv2d(256, 32, kernel_size=(9, 9), stride=(2, 2))
(4): Conv2d(256, 32, kernel_size=(9, 9), stride=(2, 2))
(5): Conv2d(256, 32, kernel_size=(9, 9), stride=(2, 2))
(6): Conv2d(256, 32, kernel_size=(9, 9), stride=(2, 2))
(7): Conv2d(256, 32, kernel_size=(9, 9), stride=(2, 2))
)
)
(digits): CapsuleLayer (
)
(decoder): Decoder (
(fc1): Linear (160 -> 512)
(fc2): Linear (512 -> 1024)
(fc3): Linear (1024 -> 784)
(relu): ReLU (inplace)
(sigmoid): Sigmoid ()
)
)
Parameters and size:
-------------------
conv1.conv0.weight: [256, 1, 9, 9]
conv1.conv0.bias: [256]
primary.conv_units.0.weight: [32, 256, 9, 9]
primary.conv_units.0.bias: [32]
primary.conv_units.1.weight: [32, 256, 9, 9]
primary.conv_units.1.bias: [32]
primary.conv_units.2.weight: [32, 256, 9, 9]
primary.conv_units.2.bias: [32]
primary.conv_units.3.weight: [32, 256, 9, 9]
primary.conv_units.3.bias: [32]
primary.conv_units.4.weight: [32, 256, 9, 9]
primary.conv_units.4.bias: [32]
primary.conv_units.5.weight: [32, 256, 9, 9]
primary.conv_units.5.bias: [32]
primary.conv_units.6.weight: [32, 256, 9, 9]
primary.conv_units.6.bias: [32]
primary.conv_units.7.weight: [32, 256, 9, 9]
primary.conv_units.7.bias: [32]
digits.weight: [1, 1152, 10, 16, 8]
decoder.fc1.weight: [512, 160]
decoder.fc1.bias: [512]
decoder.fc2.weight: [1024, 512]
decoder.fc2.bias: [1024]
decoder.fc3.weight: [784, 1024]
decoder.fc3.bias: [784]
Total number of parameters on (with reconstruction network): 8227088 (8 million)
```
### TensorBoard
We logged the training and test losses and accuracies using tensorboardX. TensorBoard helps us visualize how the machine learn over time. We can visualize statistics, such as how the objective function is changing or weights or accuracy varied during training.
TensorBoard operates by reading TensorFlow data (events files).
#### How to Use TensorBoard
1. Download a [copy of the events files](https://drive.google.com/uc?export=download&id=1lZVffeZTkUQfSxmZmYDViRzmhb59wBWL) for the latest run from my Google Drive.
2. Uncompress the file and put it into `./runs`.
3. Check to ensure you have installed tensorflow (CPU version). We need this for TensorBoard server and dashboard.
4. Start TensorBoard.
```bash
$ tensorboard --logdir runs
```
5. Open TensorBoard dashboard in your web browser using this URL: http://localhost:6006
### Other Datasets
#### CIFAR10
In the spirit of experiment, I have tried using other datasets. I have updated the implementation so that it supports and works with CIFAR10. Need to note that I have not tested throughly our capsule model on CIFAR10.
Here's how we can train and test the model on CIFAR10 by running the following commands.
```bash
python main.py --dataset cifar10 --num-conv-in-channel 3 --input-width 32 --input-height 32 --primary-unit-size 2048 --epochs 80 --num-routing 1 --use-reconstruction-loss yes --regularization-scale 0.0005
```
##### Training Loss and Accuracy
The training losses and accuracies for CapsNet-v4 (80 epochs, 3 routing iteration, using reconstruction, regularization scale of 0.0005):
![](results/cifar10/train_loss_accuracy.png)
- Highest training accuracy: 100%
- Lowest training error: 0.3589%
##### Test Loss and Accuracy
The test losses and accuracies for CapsNet-v4 (80 epochs, 3 routing iteration, using reconstruction, regularization scale of 0.0005):
![](results/cifar10/test_loss_accuracy.png)
- Highest test accuracy: 71%
- Lowest test error: 0.5735%
## TODO
- [x] Publish results.
- [x] More testing.
- [ ] Inference mode - command to test a pre-trained model.
- [ ] Jupyter Notebook version.
- [x] Create a sample to show how we can apply CapsNet to real-world application.
- [ ] Experiment with CapsNet:
* [x] Try using another dataset.
* [ ] Come out a more creative model structure.
- [x] Pre-trained model and weights.
- [x] Add visualization for training and evaluation metrics.
- [x] Implement recontruction loss.
- [x] Check algorithm for correctness.
- [x] Update results from TensorBoard after making improvements and bug fixes.
- [x] Publish updated pre-trained model weights.
- [x] Log the original and reconstructed images using TensorBoard.
- [ ] Update results with reconstructed image and original image.
- [ ] Resume training by loading model checkpoint.
- [ ] Migrate existing code to work in PyTorch 0.4.0.
*WIP is an acronym for Work-In-Progress*
## Credits
Referenced these implementations mainly for sanity check:
1. [TensorFlow implementation by @naturomics](https://github.com/naturomics/CapsNet-Tensorflow)
## Learning Resources
Here's some resources that we think will be helpful if you want to learn more about Capsule Networks:
- Articles and blog posts:
- [Understanding Hinton's Capsule Networks. Part I: Intuition.](https://medium.com/@pechyonkin/understanding-hintons-capsule-networks-part-i-intuition-b4b559d1159b)
- [Dynamic routing between capsules](https://blog.acolyer.org/2017/11/13/dynamic-routing-between-capsules/)
- [What is a CapsNet or Capsule Network?](https://hackernoon.com/what-is-a-capsnet-or-capsule-network-2bfbe48769cc)
- [Capsule Networks Are Shaking up AI — Here's How to Use Them](https://hackernoon.com/capsule-networks-are-shaking-up-ai-heres-how-to-use-them-c233a0971952)
- [Capsule Networks Explained](https://kndrck.co/posts/capsule_networks_explained/)
- Videos:
- [Capsule Networks: An Improvement to Convolutional Networks](https://www.youtube.com/watch?v=VKoLGnq15RM)
- [Capsule Networks (CapsNets) – Tutorial](https://www.youtube.com/watch?v=pPN8d0E3900)
## Other Implementations
- TensorFlow:
- The first author of the paper, [Sara Sabour has released the code](https://github.com/Sarasra/models/tree/master/research/capsules).
## Real-world Application of CapsNet
The following is a few samples in the wild that show how we can apply CapsNet to real-world use cases.
- [An attempt to implement CapsNet for car make-model classification](https://www.reddit.com/r/MachineLearning/comments/80eiz3/p_implementing_a_capsnet_for_car_makemodel/)
- [A Keras implementation of Capsule Network on Fashion MNIST dataset](https://github.com/XifengGuo/CapsNet-Fashion-MNIST)
\ No newline at end of file
tutorial/capsule/codes/capsule_layer.py 0 → 100644
View file @ a50bbe58
"""Capsule layer
PyTorch implementation of CapsNet in Sabour, Hinton et al.'s paper
Dynamic Routing Between Capsules. NIPS 2017.
https://arxiv.org/abs/1710.09829
Author: Cedric Chee
"""
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable
import utils
class CapsuleLayer(nn.Module):
"""
The core implementation of the idea of capsules
"""
def __init__(self, in_unit, in_channel, num_unit, unit_size, use_routing,
num_routing, cuda_enabled):
super(CapsuleLayer, self).__init__()
self.in_unit = in_unit
self.in_channel = in_channel
self.num_unit = num_unit
self.use_routing = use_routing
self.num_routing = num_routing
self.cuda_enabled = cuda_enabled
if self.use_routing:
"""
Based on the paper, DigitCaps which is capsule layer(s) with
capsule inputs use a routing algorithm that uses this weight matrix, Wij
"""
# weight shape:
# [1 x primary_unit_size x num_classes x output_unit_size x num_primary_unit]
# == [1 x 1152 x 10 x 16 x 8]
self.weight = nn.Parameter(torch.randn(1, in_channel, num_unit, unit_size, in_unit))
else:
"""
According to the CapsNet architecture section in the paper,
we have routing only between two consecutive capsule layers (e.g. PrimaryCapsules and DigitCaps).
No routing is used between Conv1 and PrimaryCapsules.
This means PrimaryCapsules is composed of several convolutional units.
"""
# Define 8 convolutional units.
self.conv_units = nn.ModuleList([
nn.Conv2d(self.in_channel, 32, 9, 2) for u in range(self.num_unit)
])
def forward(self, x):
if self.use_routing:
# Currently used by DigitCaps layer.
return self.routing(x)
else:
# Currently used by PrimaryCaps layer.
return self.no_routing(x)
def routing(self, x):
"""
Routing algorithm for capsule.
:input: tensor x of shape [128, 8, 1152]
:return: vector output of capsule j
"""
batch_size = x.size(0)
x = x.transpose(1, 2) # dim 1 and dim 2 are swapped. out tensor shape: [128, 1152, 8]
# Stacking and adding a dimension to a tensor.
# stack ops output shape: [128, 1152, 10, 8]
# unsqueeze ops output shape: [128, 1152, 10, 8, 1]
x = torch.stack([x] * self.num_unit, dim=2).unsqueeze(4)
# Convert single weight to batch weight.
# [1 x 1152 x 10 x 16 x 8] to: [128, 1152, 10, 16, 8]
batch_weight = torch.cat([self.weight] * batch_size, dim=0)
# u_hat is "prediction vectors" from the capsules in the layer below.
# Transform inputs by weight matrix.
# Matrix product of 2 tensors with shape: [128, 1152, 10, 16, 8] x [128, 1152, 10, 8, 1]
# u_hat shape: [128, 1152, 10, 16, 1]
u_hat = torch.matmul(batch_weight, x)
# All the routing logits (b_ij in the paper) are initialized to zero.
# self.in_channel = primary_unit_size = 32 * 6 * 6 = 1152
# self.num_unit = num_classes = 10
# b_ij shape: [1, 1152, 10, 1]
b_ij = Variable(torch.zeros(1, self.in_channel, self.num_unit, 1))
if self.cuda_enabled:
b_ij = b_ij.cuda()
# From the paper in the "Capsules on MNIST" section,
# the sample MNIST test reconstructions of a CapsNet with 3 routing iterations.
num_iterations = self.num_routing
for iteration in range(num_iterations):
# Routing algorithm
# Calculate routing or also known as coupling coefficients (c_ij).
# c_ij shape: [1, 1152, 10, 1]
c_ij = F.softmax(b_ij, dim=2) # Convert routing logits (b_ij) to softmax.
# c_ij shape from: [128, 1152, 10, 1] to: [128, 1152, 10, 1, 1]
c_ij = torch.cat([c_ij] * batch_size, dim=0).unsqueeze(4)
# Implement equation 2 in the paper.
# s_j is total input to a capsule, is a weigthed sum over all "prediction vectors".
# u_hat is weighted inputs, prediction ˆuj|i made by capsule i.
# c_ij * u_hat shape: [128, 1152, 10, 16, 1]
# s_j output shape: [batch_size=128, 1, 10, 16, 1]
# Sum of Primary Capsules outputs, 1152D becomes 1D.
s_j = (c_ij * u_hat).sum(dim=1, keepdim=True)
# Squash the vector output of capsule j.
# v_j shape: [batch_size, weighted sum of PrimaryCaps output,
# num_classes, output_unit_size from u_hat, 1]
# == [128, 1, 10, 16, 1]
# So, the length of the output vector of a capsule is 16, which is in dim 3.
v_j = utils.squash(s_j, dim=3)
# in_channel is 1152.
# v_j1 shape: [128, 1152, 10, 16, 1]
v_j1 = torch.cat([v_j] * self.in_channel, dim=1)
# The agreement.
# Transpose u_hat with shape [128, 1152, 10, 16, 1] to [128, 1152, 10, 1, 16],
# so we can do matrix product u_hat and v_j1.
# u_vj1 shape: [1, 1152, 10, 1]
u_vj1 = torch.matmul(u_hat.transpose(3, 4), v_j1).squeeze(4).mean(dim=0, keepdim=True)
# Update routing (b_ij) by adding the agreement to the initial logit.
b_ij = b_ij + u_vj1
return v_j.squeeze(1) # shape: [128, 10, 16, 1]
def no_routing(self, x):
"""
Get output for each unit.
A unit has batch, channels, height, width.
An example of a unit output shape is [128, 32, 6, 6]
:return: vector output of capsule j
"""
# Create 8 convolutional unit.
# A convolutional unit uses normal convolutional layer with a non-linearity (squash).
unit = [self.conv_units[i](x) for i, l in enumerate(self.conv_units)]
# Stack all unit outputs.
# Stacked of 8 unit output shape: [128, 8, 32, 6, 6]
unit = torch.stack(unit, dim=1)
batch_size = x.size(0)
# Flatten the 32 of 6x6 grid into 1152.
# Shape: [128, 8, 1152]
unit = unit.view(batch_size, self.num_unit, -1)
# Add non-linearity
# Return squashed outputs of shape: [128, 8, 1152]
return utils.squash(unit, dim=2) # dim 2 is the third dim (1152D array) in our tensor
tutorial/capsule/codes/conv_layer.py 0 → 100644
View file @ a50bbe58
"""Convolutional layer
PyTorch implementation of CapsNet in Sabour, Hinton et al.'s paper
Dynamic Routing Between Capsules. NIPS 2017.
https://arxiv.org/abs/1710.09829
Author: Cedric Chee
"""
import torch
import torch.nn as nn
class ConvLayer(nn.Module):
"""
Conventional Conv2d layer
"""
def __init__(self, in_channel, out_channel, kernel_size):
super(ConvLayer, self).__init__()
self.conv0 = nn.Conv2d(in_channels=in_channel,
out_channels=out_channel,
kernel_size=kernel_size,
stride=1)
self.relu = nn.ReLU(inplace=True)
def forward(self, x):
"""Forward pass"""
# x shape: [128, 1, 28, 28]
# out_conv0 shape: [128, 256, 20, 20]
out_conv0 = self.conv0(x)
# out_relu shape: [128, 256, 20, 20]
out_relu = self.relu(out_conv0)
return out_relu
tutorial/capsule/codes/decoder.py 0 → 100644
View file @ a50bbe58
"""Decoder Network
PyTorch implementation of CapsNet in Sabour, Hinton et al.'s paper
Dynamic Routing Between Capsules. NIPS 2017.
https://arxiv.org/abs/1710.09829
Author: Cedric Chee
"""
import torch
import torch.nn as nn
import torch.nn.functional as F
import utils
class Decoder(nn.Module):
"""
Implement Decoder structure in section 4.1, Figure 2 to reconstruct a digit
from the `DigitCaps` layer representation.
The decoder network consists of 3 fully connected layers. For each
[10, 16] output, we mask out the incorrect predictions, and send
the [16,] vector to the decoder network to reconstruct a [784,] size
image.
This Decoder network is used in training and prediction (testing).
"""
def __init__(self, num_classes, output_unit_size, input_width,
input_height, num_conv_in_channel, cuda_enabled):
"""
The decoder network consists of 3 fully connected layers, with
512, 1024, 784 (or 3072 for CIFAR10) neurons each.
"""
super(Decoder, self).__init__()
self.cuda_enabled = cuda_enabled
fc1_output_size = 512
fc2_output_size = 1024
self.fc3_output_size = input_width * input_height * num_conv_in_channel
self.fc1 = nn.Linear(num_classes * output_unit_size, fc1_output_size) # input dim 10 * 16.
self.fc2 = nn.Linear(fc1_output_size, fc2_output_size)
self.fc3 = nn.Linear(fc2_output_size, self.fc3_output_size)
# Activation functions
self.relu = nn.ReLU(inplace=True)
self.sigmoid = nn.Sigmoid()
def forward(self, x, target):
"""
We send the outputs of the `DigitCaps` layer, which is a
[batch_size, 10, 16] size tensor into the Decoder network, and
reconstruct a [batch_size, fc3_output_size] size tensor representing the image.
Args:
x: [batch_size, 10, 16] The output of the digit capsule.
target: [batch_size, 10] One-hot MNIST dataset labels.
Returns:
reconstruction: [batch_size, fc3_output_size] Tensor of reconstructed images.
"""
batch_size = target.size(0)
"""
First, do masking.
"""
# Method 1: mask with y.
# Note: we have not implement method 2 which is masking with true label.
# masked_caps shape: [batch_size, 10, 16, 1]
masked_caps = utils.mask(x, self.cuda_enabled)
"""
Second, reconstruct the images with 3 Fully Connected layers.
"""
# vector_j shape: [batch_size, 160=10*16]
vector_j = masked_caps.view(x.size(0), -1) # reshape the masked_caps tensor
# Forward pass of the network
fc1_out = self.relu(self.fc1(vector_j))
fc2_out = self.relu(self.fc2(fc1_out)) # shape: [batch_size, 1024]
reconstruction = self.sigmoid(self.fc3(fc2_out)) # shape: [batch_size, fc3_output_size]
assert reconstruction.size() == torch.Size([batch_size, self.fc3_output_size])
return reconstruction
tutorial/capsule/codes/dgl_capsule_batch.py 0 → 100644
View file @ a50bbe58
import dgl
import torch
import torch.nn.functional as F
from torch import nn
from capsule_layer import CapsuleLayer
# import main
from utils import writer, step
# global_step = main.global_step
device = "cuda" if torch.cuda.is_available() else "cpu"
class DGLFeature():
"""
To wrap different shape of representation tensor into the same shape
"""
def __init__(self, tensor, pad_to):
# self.tensor = tensor
self.node_num = tensor.size(0)
self.flat_tensor = tensor.contiguous().view(self.node_num, -1)
self.node_feature_dim = self.flat_tensor.size(1)
self.flat_pad_tensor = F.pad(self.flat_tensor, (0, pad_to - self.flat_tensor.size(1)))
self.shape = tensor.shape
@property
def tensor(self):
"""
:return: Tensor with original shape
"""
return self.flat_tensor.index_select(1, torch.arange(0, self.node_feature_dim).to(device)).view(self.shape)
@property
def padded_tensor(self):
"""
:return: Flatted and padded Tensor
"""
return self.flat_pad_tensor
class DGLBatchCapsuleLayer(CapsuleLayer):
def __init__(self, in_unit, in_channel, num_unit, unit_size, use_routing,
num_routing, cuda_enabled):
super(DGLBatchCapsuleLayer, self).__init__(in_unit, in_channel, num_unit, unit_size, use_routing,
num_routing, cuda_enabled)
self.unit_size = unit_size
self.weight = nn.Parameter(torch.randn(in_channel, num_unit, unit_size, in_unit))
def routing(self, x):
self.batch_size = x.size(0)
self.g = dgl.DGLGraph()
self.g.add_nodes_from([i for i in range(self.in_channel)])
self.g.add_nodes_from([i + self.in_channel for i in range(self.num_unit)])
for i in range(self.in_channel):
for j in range(self.num_unit):
index_j = j + self.in_channel
self.g.add_edge(i, index_j)
self.edge_features = torch.zeros(self.in_channel, self.num_unit).to('cuda')
x_ = x.transpose(0, 2)
x_ = DGLFeature(x_, self.batch_size * self.unit_size)
x = x.transpose(1, 2)
x = torch.stack([x] * self.num_unit, dim=2).unsqueeze(4)
W = torch.cat([self.weight.unsqueeze(0)] * self.batch_size, dim=0)
u_hat = torch.matmul(W, x).permute(1, 2, 0, 3, 4).squeeze().contiguous()
self.node_feature = DGLFeature(torch.zeros(self.num_unit, self.batch_size, self.unit_size).to('cuda'),
self.batch_size * self.unit_size)
nf = torch.cat([x_.padded_tensor, self.node_feature.padded_tensor], dim=0)
self.g.set_e_repr({'b_ij': self.edge_features.view(-1)})
self.g.set_n_repr({'h': nf})
self.g.set_e_repr({'u_hat': u_hat.view(-1, self.batch_size, self.unit_size)})
for i in range(self.num_routing):
self.i = i
self.g.update_all(self.capsule_msg, self.capsule_reduce,
lambda x: {'h': DGLFeature(x['h'], self.batch_size * self.unit_size).padded_tensor},
batchable=True)
self.g.update_edge(dgl.base.ALL, dgl.base.ALL, self.update_edge, batchable=True)
self.node_feature = self.g.get_n_repr()['h'] \
.index_select(0, torch.arange(self.in_channel, self.in_channel + self.num_unit).to(device)) \
.view(self.num_unit, self.batch_size, self.unit_size)
return self.node_feature.transpose(0, 1).unsqueeze(1).unsqueeze(4).squeeze(1)
def update_edge(self, u, v, edge):
return {
'b_ij': edge['b_ij'] + (v['h'].view(-1, self.batch_size, self.unit_size) * edge['u_hat']).mean(dim=1).sum(
dim=1)}
@staticmethod
def capsule_msg(src, edge):
return {'b_ij': edge['b_ij'], 'h': src['h'], 'u_hat': edge['u_hat']}
def capsule_reduce(self, node, msg):
b_ij_c, h_c, u_hat_c = msg['b_ij'], msg['h'], msg['u_hat']
u_hat = u_hat_c
c_i = F.softmax(b_ij_c, dim=0)
writer.add_histogram(f"c_i{self.i}", c_i, step['step'])
s_j = (c_i.unsqueeze(2).unsqueeze(3) * u_hat).sum(dim=1)
v_j = self.squash(s_j)
return {'h': v_j.view(-1, self.batch_size * self.unit_size)}
@staticmethod
def squash(s):
# This is equation 1 from the paper.
mag_sq = torch.sum(s ** 2, dim=2, keepdim=True)
mag = torch.sqrt(mag_sq)
s = (mag_sq / (1.0 + mag_sq)) * (s / mag)
return s
tutorial/capsule/codes/main.py 0 → 100644
View file @ a50bbe58
"""
PyTorch implementation of CapsNet in Sabour, Hinton et al.'s paper
Dynamic Routing Between Capsules. NIPS 2017.
https://arxiv.org/abs/1710.09829
Usage:
python main.py
python main.py --epochs 30
python main.py --epochs 30 --num-routing 1
Author: Cedric Chee
"""
from __future__ import print_function
import argparse
import os
from timeit import default_timer as timer
import torch
import torch.optim as optim
import torchvision.utils as vutils
from torch.autograd import Variable
from torch.backends import cudnn
from tqdm import tqdm
import utils
from model import Net
from utils import writer, step
def train(model, data_loader, optimizer, epoch, writer):
"""
Train CapsuleNet model on training set
Args:
model: The CapsuleNet model.
data_loader: An interator over the dataset. It combines a dataset and a sampler.
optimizer: Optimization algorithm.
epoch: Current epoch.
"""
print('===> Training mode')
num_batches = len(data_loader) # iteration per epoch. e.g: 469
total_step = args.epochs * num_batches
epoch_tot_acc = 0
# Switch to train mode
model.train()
if args.cuda:
# When we wrap a Module in DataParallel for multi-GPUs
model = model.module
start_time = timer()
for batch_idx, (data, target) in enumerate(tqdm(data_loader, unit='batch')):
batch_size = data.size(0)
global_step = batch_idx + (epoch * num_batches) - num_batches
step['step'] = global_step
labels = target
target_one_hot = utils.one_hot_encode(target, length=args.num_classes)
assert target_one_hot.size() == torch.Size([batch_size, 10])
data, target = Variable(data), Variable(target_one_hot)
if args.cuda:
data = data.cuda()
target = target.cuda()
# Train step - forward, backward and optimize
optimizer.zero_grad()
output = model(data) # output from DigitCaps (out_digit_caps)
loss, margin_loss, recon_loss = model.loss(data, output, target)
loss.backward()
optimizer.step()
# Calculate accuracy for each step and average accuracy for each epoch
acc = utils.accuracy(output, labels, args.cuda)
epoch_tot_acc += acc
epoch_avg_acc = epoch_tot_acc / (batch_idx + 1)
# TensorBoard logging
# 1) Log the scalar values
writer.add_scalar('train/total_loss', loss.item(), global_step)
writer.add_scalar('train/margin_loss', margin_loss.item(), global_step)
if args.use_reconstruction_loss:
writer.add_scalar('train/reconstruction_loss', recon_loss.item(), global_step)
writer.add_scalar('train/batch_accuracy', acc, global_step)
writer.add_scalar('train/accuracy', epoch_avg_acc, global_step)
# 2) Log values and gradients of the parameters (histogram)
# for tag, value in model.named_parameters():
# tag = tag.replace('.', '/')
# writer.add_histogram(tag, utils.to_np(value), global_step)
# writer.add_histogram(tag + '/grad', utils.to_np(value.grad), global_step)
# Print losses
if batch_idx % args.log_interval == 0:
template = 'Epoch {}/{}, ' \
'Step {}/{}: ' \
'[Total loss: {:.6f},' \
'\tMargin loss: {:.6f},' \
'\tReconstruction loss: {:.6f},' \
'\tBatch accuracy: {:.6f},' \
'\tAccuracy: {:.6f}]'
tqdm.write(template.format(
epoch,
args.epochs,
global_step,
total_step,
loss.item(),
margin_loss.item(),
recon_loss.item() if args.use_reconstruction_loss else 0,
acc,
epoch_avg_acc))
# Print time elapsed for an epoch
end_time = timer()
print('Time elapsed for epoch {}: {:.0f}s.'.format(epoch, end_time - start_time))
def test(model, data_loader, num_train_batches, epoch, writer):
"""
Evaluate model on validation set
Args:
model: The CapsuleNet model.
data_loader: An interator over the dataset. It combines a dataset and a sampler.
"""
print('===> Evaluate mode')
# Switch to evaluate mode
model.eval()
if args.cuda:
# When we wrap a Module in DataParallel for multi-GPUs
model = model.module
loss = 0
margin_loss = 0
recon_loss = 0
correct = 0
num_batches = len(data_loader)
global_step = epoch * num_train_batches + num_train_batches
step['step'] = global_step
for data, target in data_loader:
batch_size = data.size(0)
target_indices = target
target_one_hot = utils.one_hot_encode(target_indices, length=args.num_classes)
assert target_one_hot.size() == torch.Size([batch_size, 10])
data, target = Variable(data, volatile=True), Variable(target_one_hot)
if args.cuda:
data = data.cuda()
target = target.cuda()
# Output predictions
output = model(data) # output from DigitCaps (out_digit_caps)
# Sum up batch loss
t_loss, m_loss, r_loss = model.loss(data, output, target, size_average=False)
loss += t_loss.data[0]
margin_loss += m_loss.data[0]
recon_loss += r_loss.data[0]
# Count number of correct predictions
# v_magnitude shape: [128, 10, 1, 1]
v_magnitude = torch.sqrt((output ** 2).sum(dim=2, keepdim=True))
# pred shape: [128, 1, 1, 1]
pred = v_magnitude.data.max(1, keepdim=True)[1].cpu()
correct += pred.eq(target_indices.view_as(pred)).sum()
# Get the reconstructed images of the last batch
if args.use_reconstruction_loss:
reconstruction = model.decoder(output, target)
# Input image size and number of channel.
# By default, for MNIST, the image width and height is 28x28 and 1 channel for black/white.
image_width = args.input_width
image_height = args.input_height
image_channel = args.num_conv_in_channel
recon_img = reconstruction.view(-1, image_channel, image_width, image_height)
assert recon_img.size() == torch.Size([batch_size, image_channel, image_width, image_height])
# Save the image into file system
utils.save_image(recon_img, 'results/recons_image_test_{}_{}.png'.format(epoch, global_step))
utils.save_image(data, 'results/original_image_test_{}_{}.png'.format(epoch, global_step))
# Add and visualize the image in TensorBoard
recon_img = vutils.make_grid(recon_img.data, normalize=True, scale_each=True)
original_img = vutils.make_grid(data.data, normalize=True, scale_each=True)
writer.add_image('test/recons-image-{}-{}'.format(epoch, global_step), recon_img, global_step)
writer.add_image('test/original-image-{}-{}'.format(epoch, global_step), original_img, global_step)
# Log test losses
loss /= num_batches
margin_loss /= num_batches
recon_loss /= num_batches
# Log test accuracies
num_test_data = len(data_loader.dataset)
accuracy = correct / num_test_data
accuracy_percentage = 100. * accuracy
# TensorBoard logging
# 1) Log the scalar values
writer.add_scalar('test/total_loss', loss, global_step)
writer.add_scalar('test/margin_loss', margin_loss, global_step)
if args.use_reconstruction_loss:
writer.add_scalar('test/reconstruction_loss', recon_loss, global_step)
writer.add_scalar('test/accuracy', accuracy, global_step)
# Print test losses and accuracy
print('Test: [Loss: {:.6f},' \
'\tMargin loss: {:.6f},' \
'\tReconstruction loss: {:.6f}]'.format(
loss,
margin_loss,
recon_loss if args.use_reconstruction_loss else 0))
print('Test Accuracy: {}/{} ({:.0f}%)\n'.format(
correct, num_test_data, accuracy_percentage))
def main():
"""The main function
Entry point.
"""
global args
# Setting the hyper parameters
parser = argparse.ArgumentParser(description='Example of Capsule Network')
parser.add_argument('--epochs', type=int, default=10,
help='number of training epochs. default=10')
parser.add_argument('--lr', type=float, default=0.01,
help='learning rate. default=0.01')
parser.add_argument('--batch-size', type=int, default=128,
help='training batch size. default=128')
parser.add_argument('--test-batch-size', type=int,
default=128, help='testing batch size. default=128')
parser.add_argument('--log-interval', type=int, default=10,
help='how many batches to wait before logging training status. default=10')
parser.add_argument('--no-cuda', action='store_true', default=False,
help='disables CUDA training. default=false')
parser.add_argument('--threads', type=int, default=4,
help='number of threads for data loader to use. default=4')
parser.add_argument('--seed', type=int, default=42,
help='random seed for training. default=42')
parser.add_argument('--num-conv-out-channel', type=int, default=256,
help='number of channels produced by the convolution. default=256')
parser.add_argument('--num-conv-in-channel', type=int, default=1,
help='number of input channels to the convolution. default=1')
parser.add_argument('--num-primary-unit', type=int, default=8,
help='number of primary unit. default=8')
parser.add_argument('--primary-unit-size', type=int,
default=1152, help='primary unit size is 32 * 6 * 6. default=1152')
parser.add_argument('--num-classes', type=int, default=10,
help='number of digit classes. 1 unit for one MNIST digit. default=10')
parser.add_argument('--output-unit-size', type=int,
default=16, help='output unit size. default=16')
parser.add_argument('--num-routing', type=int,
default=3, help='number of routing iteration. default=3')
parser.add_argument('--use-reconstruction-loss', type=utils.str2bool, nargs='?', default=True,
help='use an additional reconstruction loss. default=True')
parser.add_argument('--regularization-scale', type=float, default=0.0005,
help='regularization coefficient for reconstruction loss. default=0.0005')
parser.add_argument('--dataset', help='the name of dataset (mnist, cifar10)', default='mnist')
parser.add_argument('--input-width', type=int,
default=28, help='input image width to the convolution. default=28 for MNIST')
parser.add_argument('--input-height', type=int,
default=28, help='input image height to the convolution. default=28 for MNIST')
args = parser.parse_args()
print(args)
# Check GPU or CUDA is available
args.cuda = not args.no_cuda and torch.cuda.is_available()
# Get reproducible results by manually seed the random number generator
torch.manual_seed(args.seed)
if args.cuda:
torch.cuda.manual_seed(args.seed)
# Load data
train_loader, test_loader = utils.load_data(args)
# Build Capsule Network
print('===> Building model')
model = Net(num_conv_in_channel=args.num_conv_in_channel,
num_conv_out_channel=args.num_conv_out_channel,
num_primary_unit=args.num_primary_unit,
primary_unit_size=args.primary_unit_size,
num_classes=args.num_classes,
output_unit_size=args.output_unit_size,
num_routing=args.num_routing,
use_reconstruction_loss=args.use_reconstruction_loss,
regularization_scale=args.regularization_scale,
input_width=args.input_width,
input_height=args.input_height,
cuda_enabled=args.cuda)
if args.cuda:
print('Utilize GPUs for computation')
print('Number of GPU available', torch.cuda.device_count())
model.cuda()
cudnn.benchmark = True
model = torch.nn.DataParallel(model)
# Print the model architecture and parameters
print('Model architectures:\n{}\n'.format(model))
print('Parameters and size:')
for name, param in model.named_parameters():
print('{}: {}'.format(name, list(param.size())))
# CapsNet has:
# - 8.2M parameters and 6.8M parameters without the reconstruction subnet on MNIST.
# - 11.8M parameters and 8.0M parameters without the reconstruction subnet on CIFAR10.
num_params = sum([param.nelement() for param in model.parameters()])
# The coupling coefficients c_ij are not included in the parameter list,
# we need to add them manually, which is 1152 * 10 = 11520 (on MNIST) or 2048 * 10 (on CIFAR10)
print('\nTotal number of parameters: {}\n'.format(num_params + (11520 if args.dataset == 'mnist' else 20480)))
# Optimizer
optimizer = optim.Adam(model.parameters(), lr=args.lr)
# Make model checkpoint directory
if not os.path.exists('results/trained_model'):
os.makedirs('results/trained_model')
# Train and test
for epoch in range(1, args.epochs + 1):
train(model, train_loader, optimizer, epoch, writer)
test(model, test_loader, len(train_loader), epoch, writer)
# Save model checkpoint
utils.checkpoint({
'epoch': epoch + 1,
'state_dict': model.state_dict(),
'optimizer': optimizer.state_dict()
}, epoch)
writer.close()
if __name__ == "__main__":
main()
tutorial/capsule/codes/model.py 0 → 100644
View file @ a50bbe58
"""CapsNet Architecture
PyTorch implementation of CapsNet in Sabour, Hinton et al.'s paper
Dynamic Routing Between Capsules. NIPS 2017.
https://arxiv.org/abs/1710.09829
Author: Cedric Chee
"""
import torch
import torch.nn as nn
from torch.autograd import Variable
from capsule_layer import CapsuleLayer
from conv_layer import ConvLayer
from decoder import Decoder
from dgl_capsule_batch import DGLBatchCapsuleLayer
class Net(nn.Module):
"""
A simple CapsNet with 3 layers
"""
def __init__(self, num_conv_in_channel, num_conv_out_channel, num_primary_unit,
primary_unit_size, num_classes, output_unit_size, num_routing,
use_reconstruction_loss, regularization_scale, input_width, input_height,
cuda_enabled):
"""
In the constructor we instantiate one ConvLayer module and two CapsuleLayer modules
and assign them as member variables.
"""
super(Net, self).__init__()
self.cuda_enabled = cuda_enabled
# Configurations used for image reconstruction.
self.use_reconstruction_loss = use_reconstruction_loss
# Input image size and number of channel.
# By default, for MNIST, the image width and height is 28x28
# and 1 channel for black/white.
self.image_width = input_width
self.image_height = input_height
self.image_channel = num_conv_in_channel
# Also known as lambda reconstruction. Default value is 0.0005.
# We use sum of squared errors (SSE) similar to paper.
self.regularization_scale = regularization_scale
# Layer 1: Conventional Conv2d layer.
self.conv1 = ConvLayer(in_channel=num_conv_in_channel,
out_channel=num_conv_out_channel,
kernel_size=9)
# PrimaryCaps
# Layer 2: Conv2D layer with `squash` activation.
self.primary = CapsuleLayer(in_unit=0,
in_channel=num_conv_out_channel,
num_unit=num_primary_unit,
unit_size=primary_unit_size, # capsule outputs
use_routing=False,
num_routing=num_routing,
cuda_enabled=cuda_enabled)
# DigitCaps
# Final layer: Capsule layer where the routing algorithm is.
self.digits = CapsuleLayer(in_unit=num_primary_unit,
in_channel=primary_unit_size,
num_unit=num_classes,
unit_size=output_unit_size, # 16D capsule per digit class
use_routing=True,
num_routing=num_routing,
cuda_enabled=cuda_enabled)
# Reconstruction network
if use_reconstruction_loss:
self.decoder = Decoder(num_classes, output_unit_size, input_width,
input_height, num_conv_in_channel, cuda_enabled)
def forward(self, x):
"""
Defines the computation performed at every forward pass.
"""
# x shape: [128, 1, 28, 28]. 128 is for the batch size.
# out_conv1 shape: [128, 256, 20, 20]
out_conv1 = self.conv1(x)
# out_primary_caps shape: [128, 8, 1152].
# Total PrimaryCapsules has [32 × 6 × 6 = 1152] capsule outputs.
out_primary_caps = self.primary(out_conv1)
# out_digit_caps shape: [128, 10, 16, 1]
# batch size: 128, 10 digit class, 16D capsule per digit class.
out_digit_caps = self.digits(out_primary_caps)
return out_digit_caps
def loss(self, image, out_digit_caps, target, size_average=True):
"""Custom loss function
Args:
image: [batch_size, 1, 28, 28] MNIST samples.
out_digit_caps: [batch_size, 10, 16, 1] The output from `DigitCaps` layer.
target: [batch_size, 10] One-hot MNIST dataset labels.
size_average: A boolean to enable mean loss (average loss over batch size).
Returns:
total_loss: A scalar Variable of total loss.
m_loss: A scalar of margin loss.
recon_loss: A scalar of reconstruction loss.
"""
recon_loss = 0
m_loss = self.margin_loss(out_digit_caps, target)
if size_average:
m_loss = m_loss.mean()
total_loss = m_loss
if self.use_reconstruction_loss:
# Reconstruct the image from the Decoder network
reconstruction = self.decoder(out_digit_caps, target)
recon_loss = self.reconstruction_loss(reconstruction, image)
# Mean squared error
if size_average:
recon_loss = recon_loss.mean()
# In order to keep in line with the paper,
# they scale down the reconstruction loss by 0.0005
# so that it does not dominate the margin loss.
total_loss = m_loss + recon_loss * self.regularization_scale
return total_loss, m_loss, (recon_loss * self.regularization_scale)
def margin_loss(self, input, target):
"""
Class loss
Implement equation 4 in section 3 'Margin loss for digit existence' in the paper.
Args:
input: [batch_size, 10, 16, 1] The output from `DigitCaps` layer.
target: target: [batch_size, 10] One-hot MNIST labels.
Returns:
l_c: A scalar of class loss or also know as margin loss.
"""
batch_size = input.size(0)
# ||vc|| also known as norm.
v_c = torch.sqrt((input ** 2).sum(dim=2, keepdim=True))
# Calculate left and right max() terms.
zero = Variable(torch.zeros(1))
if self.cuda_enabled:
zero = zero.cuda()
m_plus = 0.9
m_minus = 0.1
loss_lambda = 0.5
max_left = torch.max(m_plus - v_c, zero).view(batch_size, -1) ** 2
max_right = torch.max(v_c - m_minus, zero).view(batch_size, -1) ** 2
t_c = target
# Lc is margin loss for each digit of class c
l_c = t_c * max_left + loss_lambda * (1.0 - t_c) * max_right
l_c = l_c.sum(dim=1)
return l_c
def reconstruction_loss(self, reconstruction, image):
"""
The reconstruction loss is the sum of squared differences between
the reconstructed image (outputs of the logistic units) and
the original image (input image).
Implement section 4.1 'Reconstruction as a regularization method' in the paper.
Based on naturomics's implementation.
Args:
reconstruction: [batch_size, 784] Decoder outputs of reconstructed image tensor.
image: [batch_size, 1, 28, 28] MNIST samples.
Returns:
recon_error: A scalar Variable of reconstruction loss.
"""
# Calculate reconstruction loss.
batch_size = image.size(0) # or another way recon_img.size(0)
# error = (recon_img - image).view(batch_size, -1)
image = image.view(batch_size, -1) # flatten 28x28 by reshaping to [batch_size, 784]
error = reconstruction - image
squared_error = error ** 2
# Scalar Variable
recon_error = torch.sum(squared_error, dim=1)
return recon_error
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment