Commit e773dfcc authored by qianyj's avatar qianyj
Browse files

create branch for v2.9

parents
b8a9880a36233005ade7a8dae6d428a8
\ No newline at end of file
.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "tutorials/hpo_quickstart_tensorflow/main.py"
.. LINE NUMBERS ARE GIVEN BELOW.
.. only:: html
.. note::
:class: sphx-glr-download-link-note
Click :ref:`here <sphx_glr_download_tutorials_hpo_quickstart_tensorflow_main.py>`
to download the full example code
.. rst-class:: sphx-glr-example-title
.. _sphx_glr_tutorials_hpo_quickstart_tensorflow_main.py:
HPO Quickstart with TensorFlow
==============================
This tutorial optimizes the model in `official TensorFlow quickstart`_ with auto-tuning.
The tutorial consists of 4 steps:
1. Modify the model for auto-tuning.
2. Define hyperparameters' search space.
3. Configure the experiment.
4. Run the experiment.
.. _official TensorFlow quickstart: https://www.tensorflow.org/tutorials/quickstart/beginner
.. GENERATED FROM PYTHON SOURCE LINES 17-34
Step 1: Prepare the model
-------------------------
In first step, we need to prepare the model to be tuned.
The model should be put in a separate script.
It will be evaluated many times concurrently,
and possibly will be trained on distributed platforms.
In this tutorial, the model is defined in :doc:`model.py <model>`.
In short, it is a TensorFlow model with 3 additional API calls:
1. Use :func:`nni.get_next_parameter` to fetch the hyperparameters to be evalutated.
2. Use :func:`nni.report_intermediate_result` to report per-epoch accuracy metrics.
3. Use :func:`nni.report_final_result` to report final accuracy.
Please understand the model code before continue to next step.
.. GENERATED FROM PYTHON SOURCE LINES 36-58
Step 2: Define search space
---------------------------
In model code, we have prepared 4 hyperparameters to be tuned:
*dense_units*, *activation_type*, *dropout_rate*, and *learning_rate*.
Here we need to define their *search space* so the tuning algorithm can sample them in desired range.
Assuming we have following prior knowledge for these hyperparameters:
1. *dense_units* should be one of 64, 128, 256.
2. *activation_type* should be one of 'relu', 'tanh', 'swish', or None.
3. *dropout_rate* should be a float between 0.5 and 0.9.
4. *learning_rate* should be a float between 0.0001 and 0.1, and it follows exponential distribution.
In NNI, the space of *dense_units* and *activation_type* is called ``choice``;
the space of *dropout_rate* is called ``uniform``;
and the space of *learning_rate* is called ``loguniform``.
You may have noticed, these names are derived from ``numpy.random``.
For full specification of search space, check :doc:`the reference </hpo/search_space>`.
Now we can define the search space as follow:
.. GENERATED FROM PYTHON SOURCE LINES 58-66
.. code-block:: default
search_space = {
'dense_units': {'_type': 'choice', '_value': [64, 128, 256]},
'activation_type': {'_type': 'choice', '_value': ['relu', 'tanh', 'swish', None]},
'dropout_rate': {'_type': 'uniform', '_value': [0.5, 0.9]},
'learning_rate': {'_type': 'loguniform', '_value': [0.0001, 0.1]},
}
.. GENERATED FROM PYTHON SOURCE LINES 67-74
Step 3: Configure the experiment
--------------------------------
NNI uses an *experiment* to manage the HPO process.
The *experiment config* defines how to train the models and how to explore the search space.
In this tutorial we use a *local* mode experiment,
which means models will be trained on local machine, without using any special training platform.
.. GENERATED FROM PYTHON SOURCE LINES 74-77
.. code-block:: default
from nni.experiment import Experiment
experiment = Experiment('local')
.. GENERATED FROM PYTHON SOURCE LINES 78-84
Now we start to configure the experiment.
Configure trial code
^^^^^^^^^^^^^^^^^^^^
In NNI evaluation of each hyperparameter set is called a *trial*.
So the model script is called *trial code*.
.. GENERATED FROM PYTHON SOURCE LINES 84-86
.. code-block:: default
experiment.config.trial_command = 'python model.py'
experiment.config.trial_code_directory = '.'
.. GENERATED FROM PYTHON SOURCE LINES 87-96
When ``trial_code_directory`` is a relative path, it relates to current working directory.
To run ``main.py`` in a different path, you can set trial code directory to ``Path(__file__).parent``.
(`__file__ <https://docs.python.org/3.10/reference/datamodel.html#index-43>`__
is only available in standard Python, not in Jupyter Notebook.)
.. attention::
If you are using Linux system without Conda,
you may need to change ``"python model.py"`` to ``"python3 model.py"``.
.. GENERATED FROM PYTHON SOURCE LINES 98-100
Configure search space
^^^^^^^^^^^^^^^^^^^^^^
.. GENERATED FROM PYTHON SOURCE LINES 100-102
.. code-block:: default
experiment.config.search_space = search_space
.. GENERATED FROM PYTHON SOURCE LINES 103-106
Configure tuning algorithm
^^^^^^^^^^^^^^^^^^^^^^^^^^
Here we use :doc:`TPE tuner </hpo/tuners>`.
.. GENERATED FROM PYTHON SOURCE LINES 106-109
.. code-block:: default
experiment.config.tuner.name = 'TPE'
experiment.config.tuner.class_args['optimize_mode'] = 'maximize'
.. GENERATED FROM PYTHON SOURCE LINES 110-113
Configure how many trials to run
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Here we evaluate 10 sets of hyperparameters in total, and concurrently evaluate 2 sets at a time.
.. GENERATED FROM PYTHON SOURCE LINES 113-115
.. code-block:: default
experiment.config.max_trial_number = 10
experiment.config.trial_concurrency = 2
.. GENERATED FROM PYTHON SOURCE LINES 116-126
You may also set ``max_experiment_duration = '1h'`` to limit running time.
If neither ``max_trial_number`` nor ``max_experiment_duration`` are set,
the experiment will run forever until you press Ctrl-C.
.. note::
``max_trial_number`` is set to 10 here for a fast example.
In real world it should be set to a larger number.
With default config TPE tuner requires 20 trials to warm up.
.. GENERATED FROM PYTHON SOURCE LINES 128-133
Step 4: Run the experiment
--------------------------
Now the experiment is ready. Choose a port and launch it. (Here we use port 8080.)
You can use the web portal to view experiment status: http://localhost:8080.
.. GENERATED FROM PYTHON SOURCE LINES 133-135
.. code-block:: default
experiment.run(8080)
.. rst-class:: sphx-glr-script-out
Out:
.. code-block:: none
[2022-04-13 12:11:34] Creating experiment, Experiment ID: enw27qxj
[2022-04-13 12:11:34] Starting web server...
[2022-04-13 12:11:35] Setting up...
[2022-04-13 12:11:35] Web portal URLs: http://127.0.0.1:8080 http://192.168.100.103:8080
True
.. GENERATED FROM PYTHON SOURCE LINES 136-143
After the experiment is done
----------------------------
Everything is done and it is safe to exit now. The following are optional.
If you are using standard Python instead of Jupyter Notebook,
you can add ``input()`` or ``signal.pause()`` to prevent Python from exiting,
allowing you to view the web portal after the experiment is done.
.. GENERATED FROM PYTHON SOURCE LINES 143-147
.. code-block:: default
# input('Press enter to quit')
experiment.stop()
.. rst-class:: sphx-glr-script-out
Out:
.. code-block:: none
[2022-04-13 12:12:55] Stopping experiment, please wait...
[2022-04-13 12:12:58] Experiment stopped
.. GENERATED FROM PYTHON SOURCE LINES 148-158
:meth:`nni.experiment.Experiment.stop` is automatically invoked when Python exits,
so it can be omitted in your code.
After the experiment is stopped, you can run :meth:`nni.experiment.Experiment.view` to restart web portal.
.. tip::
This example uses :doc:`Python API </reference/experiment>` to create experiment.
You can also create and manage experiments with :doc:`command line tool <../hpo_nnictl/nnictl>`.
.. rst-class:: sphx-glr-timing
**Total running time of the script:** ( 1 minutes 24.384 seconds)
.. _sphx_glr_download_tutorials_hpo_quickstart_tensorflow_main.py:
.. only :: html
.. container:: sphx-glr-footer
:class: sphx-glr-footer-example
.. container:: sphx-glr-download sphx-glr-download-python
:download:`Download Python source code: main.py <main.py>`
.. container:: sphx-glr-download sphx-glr-download-jupyter
:download:`Download Jupyter notebook: main.ipynb <main.ipynb>`
.. only:: html
.. rst-class:: sphx-glr-signature
`Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n# Port TensorFlow Quickstart to NNI\nThis is a modified version of `TensorFlow quickstart`_.\n\nIt can be run directly and will have the exact same result as original version.\n\nFurthermore, it enables the ability of auto tuning with an NNI *experiment*, which will be detailed later.\n\nIt is recommended to run this script directly first to verify the environment.\n\nThere are 3 key differences from the original version:\n\n1. In `Get optimized hyperparameters`_ part, it receives generated hyperparameters.\n2. In `(Optional) Report intermediate results`_ part, it reports per-epoch accuracy metrics.\n3. In `Report final result`_ part, it reports final accuracy.\n\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import nni\nimport tensorflow as tf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Hyperparameters to be tuned\nThese are the hyperparameters that will be tuned later.\n\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"params = {\n 'dense_units': 128,\n 'activation_type': 'relu',\n 'dropout_rate': 0.2,\n 'learning_rate': 0.001,\n}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Get optimized hyperparameters\nIf run directly, :func:`nni.get_next_parameter` is a no-op and returns an empty dict.\nBut with an NNI *experiment*, it will receive optimized hyperparameters from tuning algorithm.\n\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"optimized_params = nni.get_next_parameter()\nparams.update(optimized_params)\nprint(params)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Load dataset\n\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"mnist = tf.keras.datasets.mnist\n\n(x_train, y_train), (x_test, y_test) = mnist.load_data()\nx_train, x_test = x_train / 255.0, x_test / 255.0"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Build model with hyperparameters\n\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"model = tf.keras.models.Sequential([\n tf.keras.layers.Flatten(input_shape=(28, 28)),\n tf.keras.layers.Dense(params['dense_units'], activation=params['activation_type']),\n tf.keras.layers.Dropout(params['dropout_rate']),\n tf.keras.layers.Dense(10)\n])\n\nadam = tf.keras.optimizers.Adam(learning_rate=params['learning_rate'])\nloss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)\nmodel.compile(optimizer=adam, loss=loss_fn, metrics=['accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## (Optional) Report intermediate results\nThe callback reports per-epoch accuracy to show learning curve in the web portal.\nYou can also leverage the metrics for early stopping with :doc:`NNI assessors </hpo/assessors>`.\n\nThis part can be safely skipped and the experiment will work fine.\n\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"callback = tf.keras.callbacks.LambdaCallback(\n on_epoch_end = lambda epoch, logs: nni.report_intermediate_result(logs['accuracy'])\n)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Train and evluate the model\n\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"model.fit(x_train, y_train, epochs=5, verbose=2, callbacks=[callback])\nloss, accuracy = model.evaluate(x_test, y_test, verbose=2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Report final result\nReport final accuracy to NNI so the tuning algorithm can suggest better hyperparameters.\n\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"nni.report_final_result(accuracy)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
\ No newline at end of file
"""
Port TensorFlow Quickstart to NNI
=================================
This is a modified version of `TensorFlow quickstart`_.
It can be run directly and will have the exact same result as original version.
Furthermore, it enables the ability of auto tuning with an NNI *experiment*, which will be detailed later.
It is recommended to run this script directly first to verify the environment.
There are 3 key differences from the original version:
1. In `Get optimized hyperparameters`_ part, it receives generated hyperparameters.
2. In `(Optional) Report intermediate results`_ part, it reports per-epoch accuracy metrics.
3. In `Report final result`_ part, it reports final accuracy.
.. _TensorFlow quickstart: https://www.tensorflow.org/tutorials/quickstart/beginner
"""
# %%
import nni
import tensorflow as tf
# %%
# Hyperparameters to be tuned
# ---------------------------
# These are the hyperparameters that will be tuned later.
params = {
'dense_units': 128,
'activation_type': 'relu',
'dropout_rate': 0.2,
'learning_rate': 0.001,
}
# %%
# Get optimized hyperparameters
# -----------------------------
# If run directly, :func:`nni.get_next_parameter` is a no-op and returns an empty dict.
# But with an NNI *experiment*, it will receive optimized hyperparameters from tuning algorithm.
optimized_params = nni.get_next_parameter()
params.update(optimized_params)
print(params)
# %%
# Load dataset
# ------------
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
# %%
# Build model with hyperparameters
# --------------------------------
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(params['dense_units'], activation=params['activation_type']),
tf.keras.layers.Dropout(params['dropout_rate']),
tf.keras.layers.Dense(10)
])
adam = tf.keras.optimizers.Adam(learning_rate=params['learning_rate'])
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer=adam, loss=loss_fn, metrics=['accuracy'])
# %%
# (Optional) Report intermediate results
# --------------------------------------
# The callback reports per-epoch accuracy to show learning curve in the web portal.
# You can also leverage the metrics for early stopping with :doc:`NNI assessors </hpo/assessors>`.
#
# This part can be safely skipped and the experiment will work fine.
callback = tf.keras.callbacks.LambdaCallback(
on_epoch_end = lambda epoch, logs: nni.report_intermediate_result(logs['accuracy'])
)
# %%
# Train and evluate the model
# ---------------------------
model.fit(x_train, y_train, epochs=5, verbose=2, callbacks=[callback])
loss, accuracy = model.evaluate(x_test, y_test, verbose=2)
# %%
# Report final result
# -------------------
# Report final accuracy to NNI so the tuning algorithm can suggest better hyperparameters.
nni.report_final_result(accuracy)
d0c869d9d7c7f9208abc10357de52056
\ No newline at end of file
:orphan:
.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "tutorials/hpo_quickstart_tensorflow/model.py"
.. LINE NUMBERS ARE GIVEN BELOW.
.. only:: html
.. note::
:class: sphx-glr-download-link-note
Click :ref:`here <sphx_glr_download_tutorials_hpo_quickstart_tensorflow_model.py>`
to download the full example code
.. rst-class:: sphx-glr-example-title
.. _sphx_glr_tutorials_hpo_quickstart_tensorflow_model.py:
Port TensorFlow Quickstart to NNI
=================================
This is a modified version of `TensorFlow quickstart`_.
It can be run directly and will have the exact same result as original version.
Furthermore, it enables the ability of auto tuning with an NNI *experiment*, which will be detailed later.
It is recommended to run this script directly first to verify the environment.
There are 3 key differences from the original version:
1. In `Get optimized hyperparameters`_ part, it receives generated hyperparameters.
2. In `(Optional) Report intermediate results`_ part, it reports per-epoch accuracy metrics.
3. In `Report final result`_ part, it reports final accuracy.
.. _TensorFlow quickstart: https://www.tensorflow.org/tutorials/quickstart/beginner
.. GENERATED FROM PYTHON SOURCE LINES 22-25
.. code-block:: default
import nni
import tensorflow as tf
.. GENERATED FROM PYTHON SOURCE LINES 26-29
Hyperparameters to be tuned
---------------------------
These are the hyperparameters that will be tuned later.
.. GENERATED FROM PYTHON SOURCE LINES 29-36
.. code-block:: default
params = {
'dense_units': 128,
'activation_type': 'relu',
'dropout_rate': 0.2,
'learning_rate': 0.001,
}
.. GENERATED FROM PYTHON SOURCE LINES 37-41
Get optimized hyperparameters
-----------------------------
If run directly, :func:`nni.get_next_parameter` is a no-op and returns an empty dict.
But with an NNI *experiment*, it will receive optimized hyperparameters from tuning algorithm.
.. GENERATED FROM PYTHON SOURCE LINES 41-45
.. code-block:: default
optimized_params = nni.get_next_parameter()
params.update(optimized_params)
print(params)
.. rst-class:: sphx-glr-script-out
Out:
.. code-block:: none
{'dense_units': 128, 'activation_type': 'relu', 'dropout_rate': 0.2, 'learning_rate': 0.001}
.. GENERATED FROM PYTHON SOURCE LINES 46-48
Load dataset
------------
.. GENERATED FROM PYTHON SOURCE LINES 48-53
.. code-block:: default
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
.. GENERATED FROM PYTHON SOURCE LINES 54-56
Build model with hyperparameters
--------------------------------
.. GENERATED FROM PYTHON SOURCE LINES 56-67
.. code-block:: default
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(params['dense_units'], activation=params['activation_type']),
tf.keras.layers.Dropout(params['dropout_rate']),
tf.keras.layers.Dense(10)
])
adam = tf.keras.optimizers.Adam(learning_rate=params['learning_rate'])
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer=adam, loss=loss_fn, metrics=['accuracy'])
.. GENERATED FROM PYTHON SOURCE LINES 68-74
(Optional) Report intermediate results
--------------------------------------
The callback reports per-epoch accuracy to show learning curve in the web portal.
You can also leverage the metrics for early stopping with :doc:`NNI assessors </hpo/assessors>`.
This part can be safely skipped and the experiment will work fine.
.. GENERATED FROM PYTHON SOURCE LINES 74-78
.. code-block:: default
callback = tf.keras.callbacks.LambdaCallback(
on_epoch_end = lambda epoch, logs: nni.report_intermediate_result(logs['accuracy'])
)
.. GENERATED FROM PYTHON SOURCE LINES 79-81
Train and evluate the model
---------------------------
.. GENERATED FROM PYTHON SOURCE LINES 81-84
.. code-block:: default
model.fit(x_train, y_train, epochs=5, verbose=2, callbacks=[callback])
loss, accuracy = model.evaluate(x_test, y_test, verbose=2)
.. rst-class:: sphx-glr-script-out
Out:
.. code-block:: none
Epoch 1/5
[2022-03-21 01:25:00] INFO (nni/MainThread) Intermediate result: 0.9153500199317932 (Index 0)
1875/1875 - 17s - loss: 0.2914 - accuracy: 0.9154 - 17s/epoch - 9ms/step
Epoch 2/5
[2022-03-21 01:25:18] INFO (nni/MainThread) Intermediate result: 0.9588666558265686 (Index 1)
1875/1875 - 18s - loss: 0.1387 - accuracy: 0.9589 - 18s/epoch - 10ms/step
Epoch 3/5
[2022-03-21 01:25:38] INFO (nni/MainThread) Intermediate result: 0.9677000045776367 (Index 2)
1875/1875 - 20s - loss: 0.1073 - accuracy: 0.9677 - 20s/epoch - 11ms/step
Epoch 4/5
[2022-03-21 01:25:56] INFO (nni/MainThread) Intermediate result: 0.9738666415214539 (Index 3)
1875/1875 - 18s - loss: 0.0866 - accuracy: 0.9739 - 18s/epoch - 10ms/step
Epoch 5/5
[2022-03-21 01:26:16] INFO (nni/MainThread) Intermediate result: 0.977483332157135 (Index 4)
1875/1875 - 21s - loss: 0.0728 - accuracy: 0.9775 - 21s/epoch - 11ms/step
313/313 - 2s - loss: 0.0702 - accuracy: 0.9776 - 2s/epoch - 6ms/step
.. GENERATED FROM PYTHON SOURCE LINES 85-88
Report final result
-------------------
Report final accuracy to NNI so the tuning algorithm can suggest better hyperparameters.
.. GENERATED FROM PYTHON SOURCE LINES 88-89
.. code-block:: default
nni.report_final_result(accuracy)
.. rst-class:: sphx-glr-script-out
Out:
.. code-block:: none
[2022-03-21 01:27:08] INFO (nni/MainThread) Final result: 0.9775999784469604
.. rst-class:: sphx-glr-timing
**Total running time of the script:** ( 2 minutes 27.156 seconds)
.. _sphx_glr_download_tutorials_hpo_quickstart_tensorflow_model.py:
.. only :: html
.. container:: sphx-glr-footer
:class: sphx-glr-footer-example
.. container:: sphx-glr-download sphx-glr-download-python
:download:`Download Python source code: model.py <model.py>`
.. container:: sphx-glr-download sphx-glr-download-jupyter
:download:`Download Jupyter notebook: model.ipynb <model.ipynb>`
.. only:: html
.. rst-class:: sphx-glr-signature
`Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
:orphan:
.. _sphx_glr_tutorials_hpo_quickstart_tensorflow_sg_execution_times:
Computation times
=================
**01:24.384** total execution time for **tutorials_hpo_quickstart_tensorflow** files:
+-----------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_tutorials_hpo_quickstart_tensorflow_main.py` (``main.py``) | 01:24.384 | 0.0 MB |
+-----------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_tutorials_hpo_quickstart_tensorflow_model.py` (``model.py``) | 00:00.000 | 0.0 MB |
+-----------------------------------------------------------------------------+-----------+--------+
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment