Commit 1011377c authored by qianyj's avatar qianyj
Browse files

the source code of NNI for DCU

parent abc22158
#############################
Auto (Hyper-parameter) Tuning
#############################
Auto tuning is one of the key features provided by NNI; a main application scenario being
hyper-parameter tuning. Tuning specifically applies to trial code. We provide a lot of popular
auto tuning algorithms (called Tuner), and some early stop algorithms (called Assessor).
NNI supports running trials on various training platforms, for example, on a local machine,
on several servers in a distributed manner, or on platforms such as OpenPAI, Kubernetes, etc.
Other key features of NNI, such as model compression, feature engineering, can also be further
enhanced by auto tuning, which we'll described when introducing those features.
NNI has high extensibility, advanced users can customize their own Tuner, Assessor, and Training Service
according to their needs.
.. toctree::
:maxdepth: 2
Write Trial <TrialExample/Trials>
Tuners <builtin_tuner>
Assessors <builtin_assessor>
Training Platform <training_services>
Examples <examples>
WebUI <Tutorial/WebUI>
How to Debug <Tutorial/HowToDebug>
Advanced <hpo_advanced>
HPO Benchmarks <hpo_benchmark>
############
Installation
############
Currently we support installation on Linux, Mac and Windows. We also allow you to use docker.
.. toctree::
:maxdepth: 2
Linux & Mac <Tutorial/InstallationLinux>
Windows <Tutorial/InstallationWin>
Use Docker <Tutorial/HowToUseDocker>
\ No newline at end of file
#################
Model Compression
#################
Deep neural networks (DNNs) have achieved great success in many tasks. However, typical neural networks are both
computationally expensive and energy intensive, can be difficult to be deployed on devices with low computation
resources or with strict latency requirements. Therefore, a natural thought is to perform model compression to
reduce model size and accelerate model training/inference without losing performance significantly. Model compression
techniques can be divided into two categories: pruning and quantization. The pruning methods explore the redundancy
in the model weights and try to remove/prune the redundant and uncritical weights. Quantization refers to compressing
models by reducing the number of bits required to represent weights or activations.
NNI provides an easy-to-use toolkit to help user design and use model pruning and quantization algorithms.
It supports Tensorflow and PyTorch with unified interface.
For users to compress their models, they only need to add several lines in their code.
There are some popular model compression algorithms built-in in NNI.
Users could further use NNI's auto tuning power to find the best compressed model,
which is detailed in Auto Model Compression.
On the other hand, users could easily customize their new compression algorithms using NNI's interface.
For details, please refer to the following tutorials:
.. toctree::
:maxdepth: 2
Overview <Compression/Overview>
Quick Start <Compression/QuickStart>
Tutorial <Compression/Tutorial>
Pruning <Compression/pruning>
Pruning V2 <Compression/v2_pruning>
Quantization <Compression/quantization>
Utilities <Compression/CompressionUtils>
Advanced Usage <Compression/advanced>
API Reference <Compression/CompressionReference>
#############################################
Retiarii for Neural Architecture Search (NAS)
#############################################
Automatic neural architecture search is taking an increasingly important role on finding better models.
Recent research works have proved the feasibility of automatic NAS, and also found some models that could beat manually tuned models.
Some of representative works are NASNet, ENAS, DARTS, Network Morphism, and Evolution. Moreover, new innovations keep emerging.
However, it takes great efforts to implement NAS algorithms, and it is hard to reuse code base of existing algorithms in a new one.
To facilitate NAS innovations (e.g., design and implement new NAS models, compare different NAS models side-by-side),
an easy-to-use and flexible programming interface is crucial.
Thus, we design `Retiarii <https://www.usenix.org/system/files/osdi20-zhang_quanlu.pdf>`__. It is a deep learning framework that supports the exploratory training on a neural network model space, rather than on a single neural network model.
Exploratory training with Retiarii allows user to express various search spaces for *Neural Architecture Search* and *Hyper-Parameter Tuning* with high flexibility.
Some frequently used terminologies in this document:
* *Model search space*: it means a set of models from which the best model is explored/searched. Sometimes we use *search space* or *model space* in short.
* *Exploration strategy*: the algorithm that is used to explore a model search space.
* *Model evaluator*: it is used to train a model and evaluate the model's performance.
Follow the instructions below to start your journey with Retiarii.
.. toctree::
:maxdepth: 2
Overview <NAS/Overview>
Quick Start <NAS/QuickStart>
Construct Model Space <NAS/construct_space>
Multi-trial NAS <NAS/multi_trial_nas>
One-shot NAS <NAS/one_shot_nas>
Hardware-aware NAS <NAS/HardwareAwareNAS>
NAS Benchmarks <NAS/Benchmarks>
NAS API References <NAS/ApiReference>
References
==================
.. toctree::
:maxdepth: 2
nnictl Commands <Tutorial/Nnictl>
Experiment Configuration <reference/experiment_config>
Experiment Configuration (legacy) <Tutorial/ExperimentConfig>
Search Space <Tutorial/SearchSpaceSpec>
NNI Annotation <Tutorial/AnnotationSpec>
SDK API References <sdk_reference>
Supported Framework Library <SupportedFramework_Library>
Launch from Python <Tutorial/HowToLaunchFromPython>
Shared Storage <Tutorial/HowToUseSharedStorage>
Tensorboard <Tutorial/Tensorboard>
===========================
Experiment Config Reference
===========================
A config file is needed when creating an experiment. This document describes the rules to write a config file and provides some examples.
.. Note::
1. This document lists field names with ``camelCase``. If users use these fields in the pythonic way with NNI Python APIs (e.g., ``nni.experiment``), the field names should be converted to ``snake_case``.
2. In this document, the type of fields are formatted as `Python type hint <https://docs.python.org/3.10/library/typing.html>`_. Therefore JSON objects are called `dict` and arrays are called `list`.
.. _path:
3. Some fields take a path to a file or directory. Unless otherwise noted, both absolute path and relative path are supported, and ``~`` will be expanded to the home directory.
- When written in the YAML file, relative paths are relative to the directory containing that file.
- When assigned in Python code, relative paths are relative to the current working directory.
- All relative paths are converted to absolute when loading YAML file into Python class, and when saving Python class to YAML file.
4. Setting a field to ``None`` or ``null`` is equivalent to not setting the field.
.. contents:: Contents
:local:
:depth: 3
Examples
========
Local Mode
^^^^^^^^^^
.. code-block:: yaml
experimentName: MNIST
searchSpaceFile: search_space.json
trialCommand: python mnist.py
trialCodeDirectory: .
trialGpuNumber: 1
trialConcurrency: 2
maxExperimentDuration: 24h
maxTrialNumber: 100
tuner:
name: TPE
classArgs:
optimize_mode: maximize
trainingService:
platform: local
useActiveGpu: True
Local Mode (Inline Search Space)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: yaml
searchSpace:
batch_size:
_type: choice
_value: [16, 32, 64]
learning_rate:
_type: loguniform
_value: [0.0001, 0.1]
trialCommand: python mnist.py
trialGpuNumber: 1
trialConcurrency: 2
tuner:
name: TPE
classArgs:
optimize_mode: maximize
trainingService:
platform: local
useActiveGpu: True
Remote Mode
^^^^^^^^^^^
.. code-block:: yaml
experimentName: MNIST
searchSpaceFile: search_space.json
trialCommand: python mnist.py
trialCodeDirectory: .
trialGpuNumber: 1
trialConcurrency: 2
maxExperimentDuration: 24h
maxTrialNumber: 100
tuner:
name: TPE
classArgs:
optimize_mode: maximize
trainingService:
platform: remote
machineList:
- host: 11.22.33.44
user: alice
password: xxxxx
- host: my.domain.com
user: bob
sshKeyFile: ~/.ssh/id_rsa
Reference
=========
ExperimentConfig
^^^^^^^^^^^^^^^^
.. list-table::
:widths: 10 10 80
:header-rows: 1
* - Field Name
- Type
- Description
* - experimentName
- ``str``, optional
- Mnemonic name of the experiment, which will be shown in WebUI and nnictl.
* - searchSpaceFile
- ``str``, optional
- Path_ to the JSON file containing the search space.
Search space format is determined by tuner. The common format for built-in tuners is documented `here <../Tutorial/SearchSpaceSpec.rst>`__.
Mutually exclusive to ``searchSpace``.
* - searchSpace
- ``JSON``, optional
- Search space object.
The format is determined by tuner. Common format for built-in tuners is documented `here <../Tutorial/SearchSpaceSpec.rst>`__.
Note that ``None`` means "no such field" so empty search space should be written as ``{}``.
Mutually exclusive to ``searchSpaceFile``.
* - trialCommand
- ``str``
- Command to launch trial.
The command will be executed in bash on Linux and macOS, and in PowerShell on Windows.
Note that using ``python3`` on Linux and macOS, and using ``python`` on Windows.
* - trialCodeDirectory
- ``str``, optional
- Default: ``"."``. `Path`_ to the directory containing trial source files.
All files in this directory will be sent to the training machine, unless in the ``.nniignore`` file.
(See :ref:`nniignore <nniignore>` for details.)
* - trialConcurrency
- ``int``
- Specify how many trials should be run concurrently.
The real concurrency also depends on hardware resources and may be less than this value.
* - trialGpuNumber
- ``int`` or ``None``, optional
- Default: None. This field might have slightly different meanings for various training services,
especially when set to ``0`` or ``None``.
See `training service's document <../training_services.rst>`__ for details.
In local mode, setting the field to ``0`` will prevent trials from accessing GPU (by empty ``HIP_VISIBLE_DEVICES``).
And when set to ``None``, trials will be created and scheduled as if they did not use GPU,
but they can still use all GPU resources if they want.
* - maxExperimentDuration
- ``str``, optional
- Limit the duration of this experiment if specified. The duration is unlimited if not set.
Format: ``number + s|m|h|d``.
Examples: ``"10m"``, ``"0.5h"``.
When time runs out, the experiment will stop creating trials but continue to serve WebUI.
* - maxTrialNumber
- ``int``, optional
- Limit the number of trials to create if specified. The trial number is unlimited if not set.
When the budget runs out, the experiment will stop creating trials but continue to serve WebUI.
* - maxTrialDuration
- ``str``, optional
- Limit the duration of trial job if specified. The duration is unlimited if not set.
Format: ``number + s|m|h|d``.
Examples: ``"10m"``, ``"0.5h"``.
When time runs out, the current trial job will stop.
* - nniManagerIp
- ``str``, optional
- Default: default connection chosen by system. IP of the current machine, used by training machines to access NNI manager. Not used in local mode.
Except for the local mode, it is highly recommended to set this field manually.
* - useAnnotation
- ``bool``, optional
- Default: ``False``. Enable `annotation <../Tutorial/AnnotationSpec.rst>`__.
When using annotation, ``searchSpace`` and ``searchSpaceFile`` should not be specified manually.
* - debug
- ``bool``, optional
- Default: ``False``. Enable debug mode.
When enabled, logging will be more verbose and some internal validation will be loosened.
* - logLevel
- ``str``, optional
- Default: ``info`` or ``debug``, depending on ``debug`` option. Set log level of the whole system.
values: ``"trace"``, ``"debug"``, ``"info"``, ``"warning"``, ``"error"``, ``"fatal"``
When debug mode is enabled, Loglevel is set to "debug", otherwise, Loglevel is set to "info".
Most modules of NNI will be affected by this value, including NNI manager, tuner, training service, etc.
The exception is trial, whose logging level is directly managed by trial code.
For Python modules, "trace" acts as logging level 0 and "fatal" acts as ``logging.CRITICAL``.
* - experimentWorkingDirectory
- ``str``, optional
- Default: ``~/nni-experiments``.
Specify the :ref:`directory <path>` to place log, checkpoint, metadata, and other run-time stuff.
NNI will create a subdirectory named by experiment ID, so it is safe to use the same directory for multiple experiments.
* - tunerGpuIndices
- ``list[int]`` or ``str`` or ``int``, optional
- Limit the GPUs visible to tuner, assessor, and advisor.
This will be the ``HIP_VISIBLE_DEVICES`` environment variable of tuner process.
Because tuner, assessor, and advisor run in the same process, this option will affect them all.
* - tuner
- ``AlgorithmConfig``, optional
- Specify the tuner.
The built-in tuners can be found `here <../builtin_tuner.rst>`__ and you can follow `this tutorial <../Tuner/CustomizeTuner.rst>`__ to customize a new tuner.
* - assessor
- ``AlgorithmConfig``, optional
- Specify the assessor.
The built-in assessors can be found `here <../builtin_assessor.rst>`__ and you can follow `this tutorial <../Assessor/CustomizeAssessor.rst>`__ to customize a new assessor.
* - advisor
- ``AlgorithmConfig``, optional
- Specify the advisor.
NNI provides two built-in advisors: `BOHB <../Tuner/BohbAdvisor.rst>`__ and `Hyperband <../Tuner/HyperbandAdvisor.rst>`__, and you can follow `this tutorial <../Tuner/CustomizeAdvisor.rst>`__ to customize a new advisor.
* - trainingService
- ``TrainingServiceConfig``
- Specify the `training service <../TrainingService/Overview.rst>`__.
* - sharedStorage
- ``SharedStorageConfig``, optional
- Configure the shared storage, detailed usage can be found `here <../Tutorial/HowToUseSharedStorage.rst>`__.
AlgorithmConfig
^^^^^^^^^^^^^^^
``AlgorithmConfig`` describes a tuner / assessor / advisor algorithm.
For customized algorithms, there are two ways to describe them:
1. `Register the algorithm <../Tutorial/InstallCustomizedAlgos.rst>`__ to use it like built-in. (preferred)
2. Specify code directory and class name directly.
.. list-table::
:widths: 10 10 80
:header-rows: 1
* - Field Name
- Type
- Description
* - name
- ``str`` or ``None``, optional
- Default: None. Name of the built-in or registered algorithm.
``str`` for the built-in and registered algorithm, ``None`` for other customized algorithms.
* - className
- ``str`` or ``None``, optional
- Default: None. Qualified class name of not registered customized algorithm.
``None`` for the built-in and registered algorithm, ``str`` for other customized algorithms.
example: ``"my_tuner.MyTuner"``
* - codeDirectory
- ``str`` or ``None``, optional
- Default: None. Path_ to the directory containing the customized algorithm class.
``None`` for the built-in and registered algorithm, ``str`` for other customized algorithms.
* - classArgs
- ``dict[str, Any]``, optional
- Keyword arguments passed to algorithm class' constructor.
See algorithm's document for supported value.
TrainingServiceConfig
^^^^^^^^^^^^^^^^^^^^^
One of the following:
- `LocalConfig`_
- `RemoteConfig`_
- `OpenpaiConfig`_
- `AmlConfig`_
- `DlcConfig`_
- `HybridConfig`_
For `Kubeflow <../TrainingService/KubeflowMode.rst>`_, `FrameworkController <../TrainingService/FrameworkControllerMode.rst>`_, and `AdaptDL <../TrainingService/AdaptDLMode.rst>`_ training platforms, it is suggested to use `v1 config schema <../Tutorial/ExperimentConfig.rst>`_ for now.
LocalConfig
-----------
Detailed usage can be found `here <../TrainingService/LocalMode.rst>`__.
.. list-table::
:widths: 10 10 80
:header-rows: 1
* - Field Name
- Type
- Description
* - platform
- ``"local"``
-
* - useActiveGpu
- ``bool``, optional
- Default: ``False``. Specify whether NNI should submit trials to GPUs occupied by other tasks.
Must be set when ``trialGpuNumber`` greater than zero.
Following processes can make GPU "active":
- non-NNI HIP programs
- graphical desktop
- trials submitted by other NNI instances, if you have more than one NNI experiments running at same time
- other users' HIP programs, if you are using a shared server
If you are using a graphical OS like Windows 10 or Ubuntu desktop, set this field to ``True``, otherwise, the GUI will prevent NNI from launching any trial.
When you create multiple NNI experiments and ``useActiveGpu`` is set to ``True``, they will submit multiple trials to the same GPU(s) simultaneously.
* - maxTrialNumberPerGpu
- ``int``, optional
- Default: ``1``. Specify how many trials can share one GPU.
* - gpuIndices
- ``list[int]`` or ``str`` or ``int``, optional
- Limit the GPUs visible to trial processes.
If ``trialGpuNumber`` is less than the length of this value, only a subset will be visible to each trial.
This will be used as ``HIP_VISIBLE_DEVICES`` environment variable.
RemoteConfig
------------
Detailed usage can be found `here <../TrainingService/RemoteMachineMode.rst>`__.
.. list-table::
:widths: 10 10 80
:header-rows: 1
* - Field Name
- Type
- Description
* - platform
- ``"remote"``
-
* - machineList
- ``List[RemoteMachineConfig]``
- List of training machines.
* - reuseMode
- ``bool``, optional
- Default: ``True``. Enable `reuse mode <../TrainingService/Overview.rst#training-service-under-reuse-mode>`__.
RemoteMachineConfig
"""""""""""""""""""
.. list-table::
:widths: 10 10 80
:header-rows: 1
* - Field Name
- Type
- Description
* - host
- ``str``
- IP or hostname (domain name) of the machine.
* - port
- ``int``, optional
- Default: ``22``. SSH service port.
* - user
- ``str``
- Login user name.
* - password
- ``str``, optional
- If not specified, ``sshKeyFile`` will be used instead.
* - sshKeyFile
- ``str``, optional
- `Path`_ to ``sshKeyFile`` (identity file).
Only used when ``password`` is not specified.
* - sshPassphrase
- ``str``, optional
- Passphrase of SSH identity file.
* - useActiveGpu
- ``bool``, optional
- Default: ``False``. Specify whether NNI should submit trials to GPUs occupied by other tasks.
Must be set when ``trialGpuNumber`` greater than zero.
Following processes can make GPU "active":
- non-NNI CUDA programs
- graphical desktop
- trials submitted by other NNI instances, if you have more than one NNI experiments running at same time
- other users' CUDA programs, if you are using a shared server
If your remote machine is a graphical OS like Ubuntu desktop, set this field to ``True``, otherwise, the GUI will prevent NNI from launching any trial.
When you create multiple NNI experiments and ``useActiveGpu`` is set to ``True``, they will submit multiple trials to the same GPU(s) simultaneously.
* - maxTrialNumberPerGpu
- ``int``, optional
- Default: ``1``. Specify how many trials can share one GPU.
* - gpuIndices
- ``list[int]`` or ``str`` or ``int``, optional
- Limit the GPUs visible to trial processes.
If ``trialGpuNumber`` is less than the length of this value, only a subset will be visible to each trial.
This will be used as ``HIP_VISIBLE_DEVICES`` environment variable.
* - pythonPath
- ``str``, optional
- Specify a Python environment.
This path will be inserted at the front of PATH. Here are some examples:
- (linux) pythonPath: ``/opt/python3.7/bin``
- (windows) pythonPath: ``C:/Python37``
If you are working on Anaconda, there is some difference. On Windows, you also have to add ``../script`` and ``../Library/bin`` separated by ``;``. Examples are as below:
- (linux anaconda) pythonPath: ``/home/yourname/anaconda3/envs/myenv/bin/``
- (windows anaconda) pythonPath: ``C:/Users/yourname/.conda/envs/myenv``; ``C:/Users/yourname/.conda/envs/myenv/Scripts``; ``C:/Users/yourname/.conda/envs/myenv/Library/bin``
This is useful if preparing steps vary for different machines.
OpenpaiConfig
-------------
Detailed usage can be found `here <../TrainingService/PaiMode.rst>`__.
.. list-table::
:widths: 10 10 80
:header-rows: 1
* - Field Name
- Type
- Description
* - platform
- ``"openpai"``
-
* - host
- ``str``
- Hostname of OpenPAI service.
This may include ``https://`` or ``http://`` prefix.
HTTPS will be used by default.
* - username
- ``str``
- OpenPAI user name.
* - token
- ``str``
- OpenPAI user token.
This can be found in your OpenPAI user settings page.
* - trialCpuNumber
- ``int``
- Specify the CPU number of each trial to be used in OpenPAI container.
* - trialMemorySize
- ``str``
- Specify the memory size of each trial to be used in OpenPAI container.
format: ``number + tb|gb|mb|kb``.
examples: ``"8gb"``, ``"8192mb"``.
* - storageConfigName
- ``str``
- Specify the storage name used in OpenPAI.
* - dockerImage
- ``str``, optional
- Default: ``"msranni/nni:latest"``. Name and tag of docker image to run the trials.
* - localStorageMountPoint
- ``str``
- :ref:`Mount point <path>` of storage service (typically NFS) on the local machine.
* - containerStorageMountPoint
- ``str``
- Mount point of storage service (typically NFS) in docker container.
This must be an absolute path.
* - reuseMode
- ``bool``, optional
- Default: ``True``. Enable `reuse mode <../TrainingService/Overview.rst#training-service-under-reuse-mode>`__.
* - openpaiConfig
- ``JSON``, optional
- Embedded OpenPAI config file.
* - openpaiConfigFile
- ``str``, optional
- `Path`_ to OpenPAI config file.
An example can be found `here <https://github.com/microsoft/pai/blob/master/docs/manual/cluster-user/examples/hello-world-job.yaml>`__.
AmlConfig
---------
Detailed usage can be found `here <../TrainingService/AMLMode.rst>`__.
.. list-table::
:widths: 10 10 80
:header-rows: 1
* - Field Name
- Type
- Description
* - platform
- ``"aml"``
-
* - dockerImage
- ``str``, optional
- Default: ``"msranni/nni:latest"``. Name and tag of docker image to run the trials.
* - subscriptionId
- ``str``
- Azure subscription ID.
* - resourceGroup
- ``str``
- Azure resource group name.
* - workspaceName
- ``str``
- Azure workspace name.
* - computeTarget
- ``str``
- AML compute cluster name.
DlcConfig
---------
Detailed usage can be found `here <../TrainingService/DlcMode.rst>`__.
.. list-table::
:widths: 10 10 80
:header-rows: 1
* - Field Name
- Type
- Description
* - platform
- ``"dlc"``
-
* - type
- ``str``, optional
- Default: ``"Worker"``. Job spec type.
* - image
- ``str``
- Name and tag of docker image to run the trials.
* - jobType
- ``str``, optional
- Default: ``"TFJob"``. PAI-DLC training job type, ``"TFJob"`` or ``"PyTorchJob"``.
* - podCount
- ``str``
- Pod count to run a single training job.
* - ecsSpec
- ``str``
- Training server config spec string.
* - region
- ``str``
- The region where PAI-DLC public-cluster locates.
* - nasDataSourceId
- ``str``
- The NAS datasource id configurated in PAI-DLC side.
* - accessKeyId
- ``str``
- The accessKeyId of your cloud account.
* - accessKeySecret
- ``str``
- The accessKeySecret of your cloud account.
* - localStorageMountPoint
- ``str``
- The mount point of the NAS on PAI-DSW server, default is /home/admin/workspace/.
* - containerStorageMountPoint
- ``str``
- The mount point of the NAS on PAI-DLC side, default is /root/data/.
HybridConfig
------------
Currently only support `LocalConfig`_, `RemoteConfig`_, `OpenpaiConfig`_ and `AmlConfig`_ . Detailed usage can be found `here <../TrainingService/HybridMode.rst>`__.
SharedStorageConfig
^^^^^^^^^^^^^^^^^^^
Detailed usage can be found `here <../Tutorial/HowToUseSharedStorage.rst>`__.
nfsConfig
---------
.. list-table::
:widths: 10 10 80
:header-rows: 1
* - Field Name
- Type
- Description
* - storageType
- ``"NFS"``
-
* - localMountPoint
- ``str``
- The path that the storage has been or will be mounted in the local machine.
If the path does not exist, it will be created automatically. Recommended to use an absolute path, i.e. ``/tmp/nni-shared-storage``.
* - remoteMountPoint
- ``str``
- The path that the storage will be mounted in the remote machine.
If the path does not exist, it will be created automatically. Recommended to use a relative path. i.e. ``./nni-shared-storage``.
* - localMounted
- ``str``
- Specify the object and status to mount the shared storage.
values: ``"usermount"``, ``"nnimount"``, ``"nomount"``
``usermount`` means the user has already mounted this storage on localMountPoint. ``nnimount`` means NNI will try to mount this storage on localMountPoint. ``nomount`` means storage will not mount in the local machine, will support partial storages in the future.
* - nfsServer
- ``str``
- NFS server host.
* - exportedDirectory
- ``str``
- Exported directory of NFS server, detailed `here <https://www.ibm.com/docs/en/aix/7.2?topic=system-nfs-exporting-mounting>`_.
azureBlobConfig
---------------
.. list-table::
:widths: 10 10 80
:header-rows: 1
* - Field Name
- Type
- Description
* - storageType
- ``"AzureBlob"``
-
* - localMountPoint
- ``str``
- The path that the storage has been or will be mounted in the local machine.
If the path does not exist, it will be created automatically. Recommended to use an absolute path, i.e. ``/tmp/nni-shared-storage``.
* - remoteMountPoint
- ``str``
- The path that the storage will be mounted in the remote machine.
If the path does not exist, it will be created automatically. Recommended to use a relative path. i.e. ``./nni-shared-storage``.
Note that the directory must be empty when using AzureBlob.
* - localMounted
- ``str``
- Specify the object and status to mount the shared storage.
values: ``"usermount"``, ``"nnimount"``, ``"nomount"``.
``usermount`` means the user has already mounted this storage on localMountPoint. ``nnimount`` means NNI will try to mount this storage on localMountPoint. ``nomount`` means storage will not mount in the local machine, will support partial storages in the future.
* - storageAccountName
- ``str``
- Azure storage account name.
* - storageAccountKey
- ``str``
- Azure storage account key.
* - containerName
- ``str``
- AzureBlob container name.
####################
Python API Reference
####################
.. toctree::
:maxdepth: 1
Auto Tune <autotune_ref>
NAS <NAS/ApiReference>
Compression <Compression/CompressionReference>
Python API <Tutorial/HowToLaunchFromPython>
\ No newline at end of file
Introduction to NNI Training Services
=====================================
.. toctree::
Overview <./TrainingService/Overview>
Local<./TrainingService/LocalMode>
Remote<./TrainingService/RemoteMachineMode>
OpenPAI<./TrainingService/PaiMode>
Kubeflow<./TrainingService/KubeflowMode>
AdaptDL<./TrainingService/AdaptDLMode>
FrameworkController<./TrainingService/FrameworkControllerMode>
DLTS<./TrainingService/DLTSMode>
AML<./TrainingService/AMLMode>
PAI-DLC<./TrainingService/DLCMode>
Hybrid<./TrainingService/HybridMode>
<svg id="SvgjsSvg1006" width="709" height="537.0000305175781" xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:svgjs="http://svgjs.com/svgjs"><defs id="SvgjsDefs1007"><marker id="SvgjsMarker1034" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1035" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1038" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1039" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1042" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1043" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1046" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1047" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1050" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1051" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1054" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1055" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1080" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1081" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker></defs><g id="SvgjsG1008" transform="translate(25,24.999980449676514)"><path id="SvgjsPath1009" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc00"></path><g id="SvgjsG1010"><text id="SvgjsText1011" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1012" dy="16" x="35"><tspan id="SvgjsTspan1013" style="text-decoration:;">input</tspan></tspan></text></g></g><g id="SvgjsG1014" transform="translate(267,140.9999804496765)"><path id="SvgjsPath1015" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1016"><text id="SvgjsText1017" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1018" dy="16" x="35"><tspan id="SvgjsTspan1019" style="text-decoration:;">node 1</tspan></tspan></text></g></g><g id="SvgjsG1020" transform="translate(25,293.9999804496765)"><path id="SvgjsPath1021" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1022"><text id="SvgjsText1023" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1024" dy="16" x="35"><tspan id="SvgjsTspan1025" style="text-decoration:;">node 2</tspan></tspan></text></g></g><g id="SvgjsG1026" transform="translate(267,441.9999804496765)"><path id="SvgjsPath1027" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1028"><text id="SvgjsText1029" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1030" dy="16" x="35"><tspan id="SvgjsTspan1031" style="text-decoration:;">node 3</tspan></tspan></text></g></g><g id="SvgjsG1032"><path id="SvgjsPath1033" d="M60 94.99998044967651C 45 172.86665678024292 44 214.86665678024292 60 293.9999804496765" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1034)"></path></g><g id="SvgjsG1036"><path id="SvgjsPath1037" d="M60 94.99998044967651C 74.7485647305657 263.5768467107039 316.7485647305658 273.42311418864915 302 441.9999804496765" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1038)"></path></g><g id="SvgjsG1040"><path id="SvgjsPath1041" d="M302 210.9999804496765C 310.0531906298836 303.04837055335383 310.0531906298836 349.9515903459992 302 441.9999804496765" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1042)"></path></g><g id="SvgjsG1044"><path id="SvgjsPath1045" d="M60 363.9999804496765C 68.86407754147842 465.3168503645069 310.8640775414784 340.68311053484615 302 441.9999804496765" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1046)"></path></g><g id="SvgjsG1048"><path id="SvgjsPath1049" d="M60 94.99998044967651C 68.58773798957196 193.15827483289445 310.587737989572 42.841686066458635 302 140.9999804496765" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1050)"></path></g><g id="SvgjsG1052"><path id="SvgjsPath1053" d="M302 210.9999804496765C 310.9190947616536 312.94570006866127 68.91909476165358 192.05426083069176 60 293.9999804496765" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1054)"></path></g><g id="SvgjsG1056" transform="translate(382,279.8666567802429)"><path id="SvgjsPath1057" d="M 0 26C 0 -8.666666666666666 52 -8.666666666666666 52 26C 52 60.666666666666664 0 60.666666666666664 0 26Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1058"><text id="SvgjsText1059" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="32px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.55" transform="rotate(0)"></text></g></g><g id="SvgjsG1060" transform="translate(455,295.8666567802429)"><path id="SvgjsPath1061" d="M 0 0L 220 0L 220 20L 0 20Z" stroke="none" fill="none"></path><g id="SvgjsG1062"><text id="SvgjsText1063" font-family="微软雅黑" text-anchor="start" font-size="13px" width="220px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-7.45" transform="rotate(0)"><tspan id="SvgjsTspan1064" dy="16" x="0"><tspan id="SvgjsTspan1065" style="text-decoration:;">choose none/one/multiple input(s) </tspan></tspan><tspan id="SvgjsTspan1066" dy="16" x="0"><tspan id="SvgjsTspan1067" style="text-decoration:;">then add them as output</tspan></tspan></text></g></g><g id="SvgjsG1068" transform="translate(382,210.9999804496765)"><path id="SvgjsPath1069" d="M 0 26C 0 -8.666666666666666 52 -8.666666666666666 52 26C 52 60.666666666666664 0 60.666666666666664 0 26Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc00"></path><g id="SvgjsG1070"><text id="SvgjsText1071" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="32px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.55" transform="rotate(0)"></text></g></g><g id="SvgjsG1072" transform="translate(455,226.9999804496765)"><path id="SvgjsPath1073" d="M 0 0L 220 0L 220 20L 0 20Z" stroke="none" fill="none"></path><g id="SvgjsG1074"><text id="SvgjsText1075" font-family="微软雅黑" text-anchor="start" font-size="13px" width="220px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="0.55" transform="rotate(0)"><tspan id="SvgjsTspan1076" dy="16" x="0"><tspan id="SvgjsTspan1077" style="text-decoration:;">the output of the previous cell</tspan></tspan></text></g></g><g id="SvgjsG1078"><path id="SvgjsPath1079" d="M376 374.8666567802429L404.5 374.8666567802429L404.5 374.8666567802429L433 374.8666567802429" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1080)"></path></g><g id="SvgjsG1082" transform="translate(463,363.9999804496765)"><path id="SvgjsPath1083" d="M 0 0L 220 0L 220 20L 0 20Z" stroke="none" fill="none"></path><g id="SvgjsG1084"><text id="SvgjsText1085" font-family="微软雅黑" text-anchor="start" font-size="13px" width="220px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-14.95" transform="rotate(0)"><tspan id="SvgjsTspan1086" dy="16" x="0"><tspan id="SvgjsTspan1087" style="text-decoration:;">choose one operation from </tspan></tspan><tspan id="SvgjsTspan1088" dy="16" x="0"><tspan id="SvgjsTspan1089" style="text-decoration:;">MaxPool, AvgPool,, Conv1x1, </tspan></tspan><tspan id="SvgjsTspan1090" dy="16" x="0"><tspan id="SvgjsTspan1091" style="text-decoration:;">Conv3x3, SkipConnect, Zeroize</tspan></tspan></text></g></g></svg>
\ No newline at end of file
<svg id="SvgjsSvg1006" width="825.4666442871094" height="869.1333465576172" xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:svgjs="http://svgjs.com/svgjs"><defs id="SvgjsDefs1007"><marker id="SvgjsMarker1046" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1047" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1050" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1051" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1054" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1055" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1058" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1059" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1062" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1063" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1066" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1067" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1070" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1071" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1074" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1075" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1078" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1079" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1082" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1083" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1086" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1087" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1090" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1091" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1094" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1095" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1098" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1099" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1108" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1109" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1112" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1113" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1116" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1117" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1120" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1121" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1134" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1135" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1170" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1171" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker></defs><g id="SvgjsG1008" transform="translate(102.03997802734375,25.008010864257812)"><path id="SvgjsPath1009" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc66"></path><g id="SvgjsG1010"><text id="SvgjsText1011" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1012" dy="16" x="35"><tspan id="SvgjsTspan1013" style="text-decoration:;">input 0</tspan></tspan></text></g></g><g id="SvgjsG1014" transform="translate(346.03997802734375,91.00801086425781)"><path id="SvgjsPath1015" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc66"></path><g id="SvgjsG1016"><text id="SvgjsText1017" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1018" dy="16" x="35"><tspan id="SvgjsTspan1019" style="text-decoration:;">input 1</tspan></tspan></text></g></g><g id="SvgjsG1020" transform="translate(102.03997802734375,247.0080108642578)"><path id="SvgjsPath1021" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1022"><text id="SvgjsText1023" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1024" dy="16" x="35"><tspan id="SvgjsTspan1025" style="text-decoration:;">node 0</tspan></tspan></text></g></g><g id="SvgjsG1026" transform="translate(346.03997802734375,386.24000549316406)"><path id="SvgjsPath1027" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff33"></path><g id="SvgjsG1028"><text id="SvgjsText1029" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1030" dy="16" x="35"><tspan id="SvgjsTspan1031" style="text-decoration:;">node 1</tspan></tspan></text></g></g><g id="SvgjsG1032" transform="translate(102.03997802734375,503.24000549316406)"><path id="SvgjsPath1033" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff33"></path><g id="SvgjsG1034"><text id="SvgjsText1035" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1036" dy="16" x="35"><tspan id="SvgjsTspan1037" style="text-decoration:;">node 2</tspan></tspan></text></g></g><g id="SvgjsG1038" transform="translate(346.03997802734375,660.2400054931641)"><path id="SvgjsPath1039" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff33"></path><g id="SvgjsG1040"><text id="SvgjsText1041" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1042" dy="16" x="35"><tspan id="SvgjsTspan1043" style="text-decoration:;">node 3</tspan></tspan></text></g></g><g id="SvgjsG1044"><path id="SvgjsPath1045" d="M137.03997802734375 95.00801086425781C 142.3390471864014 155.57664850823596 142.3390471864014 186.43937322027966 137.03997802734375 247.0080108642578" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1046)"></path></g><g id="SvgjsG1048"><path id="SvgjsPath1049" d="M137.03997802734375 95.00801086425781C 150.28545233965042 246.404475028805 394.2854523396504 234.84354132861688 381.03997802734375 386.24000549316406" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1050)"></path></g><g id="SvgjsG1052"><path id="SvgjsPath1053" d="M381.03997802734375 161.0080108642578C 388.89208274031125 250.75797842023576 388.89208274031137 296.4900379371861 381.03997802734375 386.24000549316406" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1054)"></path></g><g id="SvgjsG1056"><path id="SvgjsPath1057" d="M381.03997802734375 161.0080108642578C 390.059279443249 264.0990977824248 146.05927944324907 143.91692394609086 137.03997802734375 247.0080108642578" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1058)"></path></g><g id="SvgjsG1060"><path id="SvgjsPath1061" d="M137.03997802734375 317.0080108642578C 143.53245315344782 391.21734113000457 143.53245315344788 429.0306752274173 137.03997802734375 503.24000549316406" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1062)"></path></g><g id="SvgjsG1064"><path id="SvgjsPath1065" d="M137.03997802734375 317.0080108642578C 145.8821637569588 418.0746562244878 389.8821637569588 285.1733601329341 381.03997802734375 386.24000549316406" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1066)"></path></g><g id="SvgjsG1068"><path id="SvgjsPath1069" d="M137.03997802734375 573.2400054931641C 146.07092807311756 676.4642368599839 390.07092807311756 557.0157741263442 381.03997802734375 660.2400054931641" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1070)"></path></g><g id="SvgjsG1072"><path id="SvgjsPath1073" d="M381.03997802734375 456.24000549316406C 388.1518866355526 537.5294928574505 388.1518866355527 578.9505181288777 381.03997802734375 660.2400054931641" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1074)"></path></g><g id="SvgjsG1076"><path id="SvgjsPath1077" d="M137.03997802734375 317.0080108642578C 151.72127748132198 484.81603149573016 395.721277481322 492.4319848616917 381.03997802734375 660.2400054931641" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1078)"></path></g><g id="SvgjsG1080"><path id="SvgjsPath1081" d="M381.03997802734375 456.24000549316406C 389.70275028689764 555.2559455067756 145.7027502868977 404.22406547955256 137.03997802734375 503.24000549316406" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1082)"></path></g><g id="SvgjsG1084"><path id="SvgjsPath1085" d="M381.03997802734375 161.0080108642578C 395.6928772148526 328.491414964573 151.69287721485261 335.7566013928489 137.03997802734375 503.24000549316406" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1086)"></path></g><g id="SvgjsG1088"><path id="SvgjsPath1089" d="M137.03997802734375 95.00801086425781C 24.999984741210938 260.9920015335083 77.99998474121094 362.9920015335083 137.03997802734375 503.24000549316406" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1090)"></path></g><g id="SvgjsG1092"><path id="SvgjsPath1093" d="M137.03997802734375 95.00801086425781C 158.5029026859753 340.33036228264245 402.5029026859754 414.9176540747794 381.03997802734375 660.2400054931641" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1094)"></path></g><g id="SvgjsG1096"><path id="SvgjsPath1097" d="M381.03997802734375 161.0080108642578C 474.99998474121094 357.7520065307617 436.99998474121094 463.7520065307617 381.03997802734375 660.2400054931641" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1098)"></path></g><g id="SvgjsG1100" transform="translate(213.99998474121094,774.1439895629883)"><path id="SvgjsPath1101" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#cc99ff"></path><g id="SvgjsG1102"><text id="SvgjsText1103" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1104" dy="16" x="35"><tspan id="SvgjsTspan1105" style="text-decoration:;">output</tspan></tspan></text></g></g><g id="SvgjsG1106"><path id="SvgjsPath1107" d="M137.03997802734375 317.0080108642578C 153.44780421895462 504.5503224089872 265.40781093282186 586.6016780182589 248.99998474121094 774.1439895629883" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1108)"></path></g><g id="SvgjsG1110"><path id="SvgjsPath1111" d="M137.03997802734375 573.2400054931641C 145.05811339639388 664.8877121320274 257.01812011026107 682.4962829241249 248.99998474121094 774.1439895629883" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1112)"></path></g><g id="SvgjsG1114"><path id="SvgjsPath1115" d="M381.03997802734375 456.24000549316406C 393.0407891056441 593.409903793695 261.0007958195113 636.9740912624574 248.99998474121094 774.1439895629883" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1116)"></path></g><g id="SvgjsG1118"><path id="SvgjsPath1119" d="M381.03997802734375 730.2400054931641C 385.89099114909015 785.6873391961069 253.85099786295734 718.6966558600454 248.99998474121094 774.1439895629883" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1120)"></path></g><g id="SvgjsG1122" transform="translate(543.471981048584,627.2400054931641)"><path id="SvgjsPath1123" d="M 0 0L 255 0L 255 33L 0 33Z" stroke="none" fill="none"></path><g id="SvgjsG1124"><text id="SvgjsText1125" font-family="微软雅黑" text-anchor="start" font-size="13px" width="255px" fill="#323232" font-weight="400" align="top" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-1.95" transform="rotate(0)"><tspan id="SvgjsTspan1126" dy="16" x="0"><tspan id="SvgjsTspan1127" style="text-decoration:;">choose an operation from AvgPool, </tspan></tspan><tspan id="SvgjsTspan1128" dy="16" x="0"><tspan id="SvgjsTspan1129" style="text-decoration:;">MaxPool, 3x3 SepConv, 5x5SepConv, </tspan></tspan><tspan id="SvgjsTspan1130" dy="16" x="0"><tspan id="SvgjsTspan1131" style="text-decoration:;">3x3DilConv, 5x5DilConv, SkipConnect</tspan></tspan></text></g></g><g id="SvgjsG1132"><path id="SvgjsPath1133" d="M473.471981048584 655.6359786987305L523.471981048584 655.6359786987305" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1134)"></path></g><g id="SvgjsG1136" transform="translate(484.471981048584,682.1359786987305)"><path id="SvgjsPath1137" d="M 0 13.104026794433594C 0 -4.368008931477864 29 -4.368008931477864 29 13.104026794433594C 29 30.57606252034505 0 30.57606252034505 0 13.104026794433594Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc00"></path><g id="SvgjsG1138"><text id="SvgjsText1139" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="9px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="3.6540267944335936" transform="rotate(0)"></text></g></g><g id="SvgjsG1140" transform="translate(484.471981048584,720.2400054931641)"><path id="SvgjsPath1141" d="M 0 13.104026794433594C 0 -4.368008931477864 29 -4.368008931477864 29 13.104026794433594C 29 30.57606252034505 0 30.57606252034505 0 13.104026794433594Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1142"><text id="SvgjsText1143" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="9px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="3.6540267944335936" transform="rotate(0)"></text></g></g><g id="SvgjsG1144" transform="translate(484.471981048584,760.2400054931641)"><path id="SvgjsPath1145" d="M 0 13.104026794433594C 0 -4.368008931477864 29 -4.368008931477864 29 13.104026794433594C 29 30.57606252034505 0 30.57606252034505 0 13.104026794433594Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#cc99ff"></path><g id="SvgjsG1146"><text id="SvgjsText1147" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="9px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="3.6540267944335936" transform="rotate(0)"></text></g></g><g id="SvgjsG1148" transform="translate(544.471981048584,684.4480590820312)"><path id="SvgjsPath1149" d="M 0 0L 232 0L 232 35.79194641113281L 0 35.79194641113281Z" stroke="none" fill="none"></path><g id="SvgjsG1150"><text id="SvgjsText1151" font-family="微软雅黑" text-anchor="start" font-size="13px" width="232px" fill="#323232" font-weight="400" align="top" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-1.95" transform="rotate(0)"><tspan id="SvgjsTspan1152" dy="16" x="0"><tspan id="SvgjsTspan1153" style="text-decoration:;">results of previous layers as input</tspan></tspan></text></g></g><g id="SvgjsG1154" transform="translate(544.471981048584,720.2400054931641)"><path id="SvgjsPath1155" d="M 0 0L 251 0L 251 37.79194641113281L 0 37.79194641113281Z" stroke="none" fill="none"></path><g id="SvgjsG1156"><text id="SvgjsText1157" font-family="微软雅黑" text-anchor="start" font-size="13px" width="251px" fill="#323232" font-weight="400" align="top" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-1.95" transform="rotate(0)"><tspan id="SvgjsTspan1158" dy="16" x="0"><tspan id="SvgjsTspan1159" style="text-decoration:;">accept two operations from all inputs, </tspan></tspan><tspan id="SvgjsTspan1160" dy="16" x="0"><tspan id="SvgjsTspan1161" style="text-decoration:;">then add the results as output</tspan></tspan></text></g></g><g id="SvgjsG1162" transform="translate(545.471981048584,763.2400054931641)"><path id="SvgjsPath1163" d="M 0 0L 251 0L 251 37.79194641113281L 0 37.79194641113281Z" stroke="none" fill="none"></path><g id="SvgjsG1164"><text id="SvgjsText1165" font-family="微软雅黑" text-anchor="start" font-size="13px" width="251px" fill="#323232" font-weight="400" align="top" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-1.95" transform="rotate(0)"><tspan id="SvgjsTspan1166" dy="16" x="0"><tspan id="SvgjsTspan1167" style="text-decoration:;">concat all inputs in channels</tspan></tspan></text></g></g><g id="SvgjsG1168"><path id="SvgjsPath1169" d="M472.471981048584 599.6359786987305L522.471981048584 599.6359786987305" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1170)"></path></g><g id="SvgjsG1172" transform="translate(545.471981048584,586.1359786987305)"><path id="SvgjsPath1173" d="M 0 0L 255 0L 255 33L 0 33Z" stroke="none" fill="none"></path><g id="SvgjsG1174"><text id="SvgjsText1175" font-family="微软雅黑" text-anchor="start" font-size="13px" width="255px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-0.95" transform="rotate(0)"><tspan id="SvgjsTspan1176" dy="16" x="0"><tspan id="SvgjsTspan1177" style="text-decoration:;">pass the result directly to the output </tspan></tspan><tspan id="SvgjsTspan1178" dy="16" x="0"><tspan id="SvgjsTspan1179" style="text-decoration:;">node</tspan></tspan></text></g></g></svg>
\ No newline at end of file
<svg id="SvgjsSvg1006" width="837.0800476074219" height="917.6933898925781" xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:svgjs="http://svgjs.com/svgjs"><defs id="SvgjsDefs1007"><marker id="SvgjsMarker1022" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1023" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1036" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1037" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1040" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1041" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1044" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1045" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1053" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1054" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1089" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1090" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1141" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1142" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1145" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1146" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1149" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1150" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1153" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1154" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1157" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1158" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1161" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1162" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1165" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1166" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1169" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1170" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1173" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1174" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1177" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1178" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1181" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1182" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1191" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1192" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1205" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1206" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1225" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1226" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1237" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1238" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1241" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1242" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker></defs><g id="SvgjsG1008" transform="translate(74.002685546875,148.85643214220443)"><path id="SvgjsPath1009" d="M 0 133.86033519553072C 0 -44.62011173184357 267.72067039106145 -44.62011173184357 267.72067039106145 133.86033519553072C 267.72067039106145 312.340782122905 0 312.340782122905 0 133.86033519553072Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffffff"></path><g id="SvgjsG1010"><text id="SvgjsText1011" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="248px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="124.41033519553072" transform="rotate(0)"></text></g></g><g id="SvgjsG1012" transform="translate(131.37140063067386,187.10224219807034)"><path id="SvgjsPath1013" d="M 0 31.871508379888265C 0 -10.623836126629422 63.74301675977653 -10.623836126629422 63.74301675977653 31.871508379888265C 63.74301675977653 74.36685288640595 0 74.36685288640595 0 31.871508379888265Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1014"><text id="SvgjsText1015" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="44px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="22.421508379888266" transform="rotate(0)"></text></g></g><g id="SvgjsG1016" transform="translate(209.68424979268502,311.8564321422044)"><path id="SvgjsPath1017" d="M 0 31.871508379888265C 0 -10.623836126629422 63.74301675977653 -10.623836126629422 63.74301675977653 31.871508379888265C 63.74301675977653 74.36685288640595 0 74.36685288640595 0 31.871508379888265Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#cc99ff"></path><g id="SvgjsG1018"><text id="SvgjsText1019" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="44px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="22.421508379888266" transform="rotate(0)"></text></g></g><g id="SvgjsG1020"><path id="SvgjsPath1021" d="M109.51665202732192 96.04078968410386C 131.00715482061798 132.46537068969045 142.81869506835938 152.7813491821289 163.24290901056213 187.10224219807034" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1022)"></path></g><g id="SvgjsG1024" transform="translate(301.6563168317912,25.923471248349685)"><path id="SvgjsPath1025" d="M 0 35.05865921787709C 0 -11.686219739292364 70.11731843575419 -11.686219739292364 70.11731843575419 35.05865921787709C 70.11731843575419 81.80353817504655 0 81.80353817504655 0 35.05865921787709Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc66"></path><g id="SvgjsG1026"><text id="SvgjsText1027" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="51px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="10.108659217877094" transform="rotate(0)"><tspan id="SvgjsTspan1028" dy="16" x="35.5"><tspan id="SvgjsTspan1029" style="text-decoration:;">Layer </tspan></tspan><tspan id="SvgjsTspan1030" dy="16" x="35.5"><tspan id="SvgjsTspan1031" style="text-decoration:;">N-1's</tspan></tspan><tspan id="SvgjsTspan1032" dy="16" x="35.5"><tspan id="SvgjsTspan1033" style="text-decoration:;">output</tspan></tspan></text></g></g><g id="SvgjsG1034"><path id="SvgjsPath1035" d="M336.7149760496683 96.04078968410386C 344.9377251996825 190.02724254125366 249.77850732258764 217.86997928505463 241.5557581725733 311.8564321422044" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1036)"></path></g><g id="SvgjsG1038"><path id="SvgjsPath1039" d="M132.28201515581355 94.21956063382453C 144.66637269771297 131.37263325952284 152.81869506835938 149.7813491821289 163.24290901056213 187.10224219807034" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1040)"></path></g><g id="SvgjsG1042"><path id="SvgjsPath1043" d="M163.24290901056213 93.30894610868489C 163.24290901056213 130.82626454443908 166.51275876473073 149.7276884857523 163.24290901056213 187.10224219807034" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1044)"></path></g><g id="SvgjsG1046" transform="matrix(6.123233995736766e-17,1,-1,6.123233995736766e-17,175.99151428052167,25.012857064188523)"><path id="SvgjsPath1047" d="M 61.9217877094972 0Q 55.729608938547486 0 55.729608938547486 6.73854748603352L 55.729608938547486 26.95418994413408Q 55.729608938547486 33.6927374301676 39.9217877094972 33.6927374301676Q 55.729608938547486 33.6927374301676 55.729608938547486 40.431284916201115L 55.729608938547486 60.646927374301676Q 55.729608938547486 67.3854748603352 61.9217877094972 67.3854748603352" stroke="rgba(50,50,50,1)" stroke-width="2" fill="none"></path><path id="SvgjsPath1048" d="M 0 0L 61.9217877094972 0L 61.9217877094972 67.3854748603352L 0 67.3854748603352Z" stroke="none" fill="none"></path><g id="SvgjsG1049"><text id="SvgjsText1050" font-family="微软雅黑" text-anchor="end" font-size="13px" width="35px" fill="#323232" font-weight="400" align="middle" anchor="end" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="24.242737430167598" transform="rotate(0)"></text></g></g><g id="SvgjsG1051"><path id="SvgjsPath1052" d="M427.321121301065 258.1301751589642L472.8518475580481 258.1301751589642" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1053)"></path></g><g id="SvgjsG1055" transform="translate(489.24290901056213,239.00727013103125)"><path id="SvgjsPath1056" d="M 0 0L 236.75977653631284 0L 236.75977653631284 46.44134078212291L 0 46.44134078212291Z" stroke="none" fill="none"></path><g id="SvgjsG1057"><text id="SvgjsText1058" font-family="微软雅黑" text-anchor="start" font-size="13px" width="237px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-1.7293296089385464" transform="rotate(0)"><tspan id="SvgjsTspan1059" dy="16" x="0"><tspan id="SvgjsTspan1060" style="text-decoration:;">choose an operation from: MaxPool, </tspan></tspan><tspan id="SvgjsTspan1061" dy="16" x="0"><tspan id="SvgjsTspan1062" style="text-decoration:;">AvgPool, SepConvBN3x3, SepConvBN</tspan></tspan><tspan id="SvgjsTspan1063" dy="16" x="0"><tspan id="SvgjsTspan1064" style="text-decoration:;">5x5 and SkipConnect</tspan></tspan></text></g></g><g id="SvgjsG1065" transform="translate(424.58927772564596,293.6441416394111)"><path id="SvgjsPath1066" d="M 0 25.952513966480446C 0 -8.650837988826815 51.90502793296089 -8.650837988826815 51.90502793296089 25.952513966480446C 51.90502793296089 60.555865921787706 0 60.555865921787706 0 25.952513966480446Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1067"><text id="SvgjsText1068" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="32px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.502513966480446" transform="rotate(0)"></text></g></g><g id="SvgjsG1069" transform="translate(489.24290901056213,301.3843651030983)"><path id="SvgjsPath1070" d="M 0 0L 236.75977653631284 0L 236.75977653631284 36.42458100558659L 0 36.42458100558659Z" stroke="none" fill="none"></path><g id="SvgjsG1071"><text id="SvgjsText1072" font-family="微软雅黑" text-anchor="start" font-size="13px" width="237px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="8.762290502793295" transform="rotate(0)"><tspan id="SvgjsTspan1073" dy="16" x="0"><tspan id="SvgjsTspan1074" style="text-decoration:;">accept none/one/multiple input(s)</tspan></tspan></text></g></g><g id="SvgjsG1075" transform="translate(425.9551995133554,367.40391817572396)"><path id="SvgjsPath1076" d="M 0 24.58659217877095C 0 -8.195530726256983 49.1731843575419 -8.195530726256983 49.1731843575419 24.58659217877095C 49.1731843575419 57.36871508379888 0 57.36871508379888 0 24.58659217877095Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#cc99ff"></path><g id="SvgjsG1077"><text id="SvgjsText1078" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="30px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="15.13659217877095" transform="rotate(0)"></text></g></g><g id="SvgjsG1079" transform="translate(489.24290901056213,373.7782198517016)"><path id="SvgjsPath1080" d="M 0 0L 236.75977653631284 0L 236.75977653631284 36.42458100558659L 0 36.42458100558659Z" stroke="none" fill="none"></path><g id="SvgjsG1081"><text id="SvgjsText1082" font-family="微软雅黑" text-anchor="start" font-size="13px" width="237px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="0.7622905027932945" transform="rotate(0)"><tspan id="SvgjsTspan1083" dy="16" x="0"><tspan id="SvgjsTspan1084" style="text-decoration:;">add all input as the output of this </tspan></tspan><tspan id="SvgjsTspan1085" dy="16" x="0"><tspan id="SvgjsTspan1086" style="text-decoration:;">layer</tspan></tspan></text></g></g><g id="SvgjsG1087"><path id="SvgjsPath1088" d="M426.41050677592534 208.04637627628261L471.9412330329085 208.04637627628261" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1089)"></path></g><g id="SvgjsG1091" transform="translate(489.24290901056213,184.37039862265135)"><path id="SvgjsPath1092" d="M 0 0L 205.79888268156424 0L 205.79888268156424 43.70949720670391L 0 43.70949720670391Z" stroke="none" fill="none"></path><g id="SvgjsG1093"><text id="SvgjsText1094" font-family="微软雅黑" text-anchor="start" font-size="13px" width="206px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="4.404748603351954" transform="rotate(0)"><tspan id="SvgjsTspan1095" dy="16" x="0"><tspan id="SvgjsTspan1096" style="text-decoration:;">(suppose this layer is the Nth </tspan></tspan><tspan id="SvgjsTspan1097" dy="16" x="0"><tspan id="SvgjsTspan1098" style="text-decoration:;">layer) outputs of all N-2 layers</tspan></tspan></text></g></g><g id="SvgjsG1099" transform="translate(100.41050677592528,33.208387449467)"><path id="SvgjsPath1100" d="M 0 0L 109.27374301675977 0L 109.27374301675977 36.42458100558659L 0 36.42458100558659Z" stroke="none" fill="none"></path><g id="SvgjsG1101"><text id="SvgjsText1102" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="110px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="8.762290502793295" transform="rotate(0)"><tspan id="SvgjsTspan1103" dy="16" x="55"><tspan id="SvgjsTspan1104" style="text-decoration:;">x (N-2)</tspan></tspan></text></g></g><g id="SvgjsG1105" transform="translate(150.49430565860683,439.196758792387)"><path id="SvgjsPath1106" d="M 0 0L 109.27374301675977 0L 109.27374301675977 36.42458100558659L 0 36.42458100558659Z" stroke="none" fill="none"></path><g id="SvgjsG1107"><text id="SvgjsText1108" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="110px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="8.762290502793295" transform="rotate(0)"><tspan id="SvgjsTspan1109" dy="16" x="55"><tspan id="SvgjsTspan1110" style="text-decoration:;">ENASMacroLayer</tspan></tspan></text></g></g><g id="SvgjsG1111" transform="translate(25.002685546875,608.5613342285156)"><path id="SvgjsPath1112" d="M 0 0L 67 0L 67 46.9L 0 46.9Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc66"></path><g id="SvgjsG1113"><text id="SvgjsText1114" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="47px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="14" transform="rotate(0)"><tspan id="SvgjsTspan1115" dy="16" x="33.5"><tspan id="SvgjsTspan1116" style="text-decoration:;">input</tspan></tspan></text></g></g><g id="SvgjsG1117" transform="translate(145.49430565860683,601.7863342285157)"><path id="SvgjsPath1118" d="M 0 29.224999999999966C 0 -9.741666666666655 59.84357541899442 -9.741666666666655 59.84357541899442 29.224999999999966C 59.84357541899442 68.19166666666659 0 68.19166666666659 0 29.224999999999966Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00ffff"></path><g id="SvgjsG1119"><text id="SvgjsText1120" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="40px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="19.774999999999967" transform="rotate(0)"></text></g></g><g id="SvgjsG1121" transform="translate(264.49430565860683,602.7863342285157)"><path id="SvgjsPath1122" d="M 0 29.224999999999966C 0 -9.741666666666655 59.84357541899442 -9.741666666666655 59.84357541899442 29.224999999999966C 59.84357541899442 68.19166666666659 0 68.19166666666659 0 29.224999999999966Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00ffff"></path><g id="SvgjsG1123"><text id="SvgjsText1124" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="40px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="19.774999999999967" transform="rotate(0)"></text></g></g><g id="SvgjsG1125" transform="translate(384.87978051894197,602.7863342285157)"><path id="SvgjsPath1126" d="M 0 29.224999999999966C 0 -9.741666666666655 59.84357541899442 -9.741666666666655 59.84357541899442 29.224999999999966C 59.84357541899442 68.19166666666659 0 68.19166666666659 0 29.224999999999966Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00ffff"></path><g id="SvgjsG1127"><text id="SvgjsText1128" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="40px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="19.774999999999967" transform="rotate(0)"></text></g></g><g id="SvgjsG1129" transform="translate(504.87978051894197,603.7863342285157)"><path id="SvgjsPath1130" d="M 0 29.224999999999966C 0 -9.741666666666655 59.84357541899442 -9.741666666666655 59.84357541899442 29.224999999999966C 59.84357541899442 68.19166666666659 0 68.19166666666659 0 29.224999999999966Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00ffff"></path><g id="SvgjsG1131"><text id="SvgjsText1132" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="40px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="19.774999999999967" transform="rotate(0)"></text></g></g><g id="SvgjsG1133" transform="translate(621.002685546875,610.5613342285156)"><path id="SvgjsPath1134" d="M 0 0L 83 0L 83 44.899999999999864L 0 44.899999999999864Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#cc99ff"></path><g id="SvgjsG1135"><text id="SvgjsText1136" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="63px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="12.999999999999932" transform="rotate(0)"><tspan id="SvgjsTspan1137" dy="16" x="41.5"><tspan id="SvgjsTspan1138" style="text-decoration:;">softmax</tspan></tspan></text></g></g><g id="SvgjsG1139"><path id="SvgjsPath1140" d="M92.002685546875 632.0113342285157L145.49430565860683 631.0113342285156" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1141)"></path></g><g id="SvgjsG1143"><path id="SvgjsPath1144" d="M205.33788107760125 631.0113342285156L264.49430565860683 632.0113342285156" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1145)"></path></g><g id="SvgjsG1147"><path id="SvgjsPath1148" d="M324.3378810776012 632.0113342285156L384.87978051894197 632.0113342285156" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1149)"></path></g><g id="SvgjsG1151"><path id="SvgjsPath1152" d="M444.7233559379365 632.0113342285156L504.87978051894197 633.0113342285156" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1153)"></path></g><g id="SvgjsG1155"><path id="SvgjsPath1156" d="M564.7233559379364 633.0113342285156L621.002685546875 633.0113342285156" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1157)"></path></g><g id="SvgjsG1159"><path id="SvgjsPath1160" d="M205.33788107760125 631.0113342285156C 232.002685546875 616.6213397979736 235.002685546875 606.6213397979736 264.49430565860683 632.0113342285156" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1161)"></path></g><g id="SvgjsG1163"><path id="SvgjsPath1164" d="M205.33788107760125 631.0113342285156C 272.002685546875 564.6213397979736 290.002685546875 572.6213397979736 384.87978051894197 632.0113342285156" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1165)"></path></g><g id="SvgjsG1167"><path id="SvgjsPath1168" d="M205.33788107760125 631.0113342285156C 285.002685546875 531.6213397979736 416.002685546875 545.6213397979736 504.87978051894197 633.0113342285156" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1169)"></path></g><g id="SvgjsG1171"><path id="SvgjsPath1172" d="M324.3378810776012 632.0113342285156C 353.002685546875 653.6213397979736 361.002685546875 652.6213397979736 384.87978051894197 632.0113342285156" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1173)"></path></g><g id="SvgjsG1175"><path id="SvgjsPath1176" d="M324.3378810776012 632.0113342285156C 393.002685546875 696.6213397979736 435.002685546875 690.6213397979736 504.87978051894197 633.0113342285156" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1177)"></path></g><g id="SvgjsG1179"><path id="SvgjsPath1180" d="M444.7233559379365 632.0113342285156C 465.002685546875 613.6213397979736 475.002685546875 615.6213397979736 504.87978051894197 633.0113342285156" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1181)"></path></g><g id="SvgjsG1183" transform="translate(254.42726655246156,734.6213397979736)"><path id="SvgjsPath1184" d="M 0 0L 177.28770949720672 0L 177.28770949720672 43L 0 43Z" stroke="none" fill="none"></path><g id="SvgjsG1185"><text id="SvgjsText1186" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="178px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="12.05" transform="rotate(0)"><tspan id="SvgjsTspan1187" dy="16" x="89"><tspan id="SvgjsTspan1188" style="text-decoration:;">GeneralModel</tspan></tspan></text></g></g><g id="SvgjsG1189"><path id="SvgjsPath1190" d="M513.16190342397 711.1301751589642L558.6926296809531 711.1301751589642" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1191)"></path></g><g id="SvgjsG1193" transform="translate(575.0836911334671,692.0072701310312)"><path id="SvgjsPath1194" d="M 0 0L 236.75977653631284 0L 236.75977653631284 46.44134078212291L 0 46.44134078212291Z" stroke="none" fill="none"></path><g id="SvgjsG1195"><text id="SvgjsText1196" font-family="微软雅黑" text-anchor="start" font-size="13px" width="237px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-1.7293296089385464" transform="rotate(0)"><tspan id="SvgjsTspan1197" dy="16" x="0"><tspan id="SvgjsTspan1198" style="text-decoration:;">choose an operation from: MaxPool, </tspan></tspan><tspan id="SvgjsTspan1199" dy="16" x="0"><tspan id="SvgjsTspan1200" style="text-decoration:;">AvgPool, SepConvBN3x3, SepConvBN</tspan></tspan><tspan id="SvgjsTspan1201" dy="16" x="0"><tspan id="SvgjsTspan1202" style="text-decoration:;">5x5 and SkipConnect</tspan></tspan></text></g></g><g id="SvgjsG1203"><path id="SvgjsPath1204" d="M512.002685546875 764.0113342285156L558.002685546875 764.0113342285156" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1205)"></path></g><g id="SvgjsG1207" transform="translate(575.0836911334671,746.4613342285156)"><path id="SvgjsPath1208" d="M 0 0L 206.91899441340786 0L 206.91899441340786 30L 0 30Z" stroke="none" fill="none"></path><g id="SvgjsG1209"><text id="SvgjsText1210" font-family="微软雅黑" text-anchor="start" font-size="13px" width="207px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="5.55" transform="rotate(0)"><tspan id="SvgjsTspan1211" dy="16" x="0"><tspan id="SvgjsTspan1212" style="text-decoration:;">pass tensor without operation</tspan></tspan></text></g></g><g id="SvgjsG1213" transform="translate(515.2401157144727,840.7024446654088)"><path id="SvgjsPath1214" d="M 0 25.766944781553388C 0 -8.588981593851129 52.76256983240228 -8.588981593851129 52.76256983240228 25.766944781553388C 52.76256983240228 60.122871156957906 0 60.122871156957906 0 25.766944781553388Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00ffff"></path><g id="SvgjsG1215"><text id="SvgjsText1216" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="33px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.31694478155339" transform="rotate(0)"></text></g></g><g id="SvgjsG1217" transform="translate(586.4943056586069,848.257098944169)"><path id="SvgjsPath1218" d="M 0 0L 109.27374301675977 0L 109.27374301675977 36.42458100558659L 0 36.42458100558659Z" stroke="none" fill="none"></path><g id="SvgjsG1219"><text id="SvgjsText1220" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="110px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="8.762290502793295" transform="rotate(0)"><tspan id="SvgjsTspan1221" dy="16" x="55"><tspan id="SvgjsTspan1222" style="text-decoration:;">ENASMacroLayer</tspan></tspan></text></g></g><g id="SvgjsG1223"><path id="SvgjsPath1224" d="M515.2401157144727 810.0463762762827L560.7708419714559 810.0463762762827" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1225)"></path></g><g id="SvgjsG1227" transform="translate(578.0725179491095,786.3703986226515)"><path id="SvgjsPath1228" d="M 0 0L 205.79888268156424 0L 205.79888268156424 43.70949720670391L 0 43.70949720670391Z" stroke="none" fill="none"></path><g id="SvgjsG1229"><text id="SvgjsText1230" font-family="微软雅黑" text-anchor="start" font-size="13px" width="206px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="4.404748603351954" transform="rotate(0)"><tspan id="SvgjsTspan1231" dy="16" x="0"><tspan id="SvgjsTspan1232" style="text-decoration:;">(suppose this layer is the Nth </tspan></tspan><tspan id="SvgjsTspan1233" dy="16" x="0"><tspan id="SvgjsTspan1234" style="text-decoration:;">layer) outputs of all N-1 layers</tspan></tspan></text></g></g><g id="SvgjsG1235"><path id="SvgjsPath1236" d="M163.24290901056213 250.84525895784685C 166.7038178979269 290.40362855551774 245.01666705993807 272.29806254453354 241.5557581725733 311.8564321422044" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1237)"></path></g><g id="SvgjsG1239"><path id="SvgjsPath1240" d="M424.41050677592534 163.04637627628261L469.9412330329085 163.04637627628261" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1241)"></path></g><g id="SvgjsG1243" transform="translate(489.24290901056213,136.37039862265135)"><path id="SvgjsPath1244" d="M 0 0L 205.79888268156424 0L 205.79888268156424 43.70949720670391L 0 43.70949720670391Z" stroke="none" fill="none"></path><g id="SvgjsG1245"><text id="SvgjsText1246" font-family="微软雅黑" text-anchor="start" font-size="13px" width="206px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="4.404748603351954" transform="rotate(0)"><tspan id="SvgjsTspan1247" dy="16" x="0"><tspan id="SvgjsTspan1248" style="text-decoration:;">pass the tensor input directly to </tspan></tspan><tspan id="SvgjsTspan1249" dy="16" x="0"><tspan id="SvgjsTspan1250" style="text-decoration:;">the next node</tspan></tspan></text></g></g></svg>
\ No newline at end of file
<svg id="SvgjsSvg1006" width="704.9999694824219" height="824.0800323486328" xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:svgjs="http://svgjs.com/svgjs"><defs id="SvgjsDefs1007"><marker id="SvgjsMarker1046" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1047" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1050" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1051" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1054" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1055" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1058" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1059" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1062" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1063" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1066" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1067" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1070" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1071" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1074" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1075" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1078" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1079" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1082" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1083" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1086" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1087" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1090" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1091" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1094" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1095" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1098" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1099" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1134" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1135" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1144" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1145" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1148" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1149" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1152" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1153" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1156" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1157" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1160" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1161" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker></defs><g id="SvgjsG1008" transform="translate(67.99998474121094,25.008005142211914)"><path id="SvgjsPath1009" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc66"></path><g id="SvgjsG1010"><text id="SvgjsText1011" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1012" dy="16" x="35"><tspan id="SvgjsTspan1013" style="text-decoration:;">input_0</tspan></tspan></text></g></g><g id="SvgjsG1014" transform="translate(238.99998474121094,111.00800514221191)"><path id="SvgjsPath1015" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc66"></path><g id="SvgjsG1016"><text id="SvgjsText1017" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1018" dy="16" x="35"><tspan id="SvgjsTspan1019" style="text-decoration:;">input_1</tspan></tspan></text></g></g><g id="SvgjsG1020" transform="translate(238.99998474121094,279.0080051422119)"><path id="SvgjsPath1021" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1022"><text id="SvgjsText1023" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1024" dy="16" x="35"><tspan id="SvgjsTspan1025" style="text-decoration:;">node 0</tspan></tspan></text></g></g><g id="SvgjsG1026" transform="translate(238.99998474121094,492.0080051422119)"><path id="SvgjsPath1027" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1028"><text id="SvgjsText1029" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1030" dy="16" x="35"><tspan id="SvgjsTspan1031" style="text-decoration:;">node 2</tspan></tspan></text></g></g><g id="SvgjsG1032" transform="translate(67.99998474121094,372.0080051422119)"><path id="SvgjsPath1033" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1034"><text id="SvgjsText1035" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1036" dy="16" x="35"><tspan id="SvgjsTspan1037" style="text-decoration:;">node 1</tspan></tspan></text></g></g><g id="SvgjsG1038" transform="translate(67.99998474121094,594.0080051422119)"><path id="SvgjsPath1039" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1040"><text id="SvgjsText1041" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1042" dy="16" x="35"><tspan id="SvgjsTspan1043" style="text-decoration:;">node 3</tspan></tspan></text></g></g><g id="SvgjsG1044"><path id="SvgjsPath1045" d="M102.99998474121094 95.00800514221191C 111.75708566874948 195.1021267645375 282.7570856687495 178.91388351988633 273.99998474121094 279.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1046)"></path></g><g id="SvgjsG1048"><path id="SvgjsPath1049" d="M273.99998474121094 181.0080051422119C 277.41648985691916 220.05883730740834 277.41648985691916 239.9571729770155 273.99998474121094 279.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1050)"></path></g><g id="SvgjsG1052"><path id="SvgjsPath1053" d="M273.99998474121094 349.0080051422119C 278.98529322637694 405.9903418730597 278.98529322637705 435.0256684113641 273.99998474121094 492.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1054)"></path></g><g id="SvgjsG1056"><path id="SvgjsPath1057" d="M102.99998474121094 95.00800514221191C 112.65684103765147 205.38637769077735 112.65684103765147 261.6296325936465 102.99998474121094 372.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1058)"></path></g><g id="SvgjsG1060"><path id="SvgjsPath1061" d="M273.99998474121094 349.0080051422119C 280.01512026970283 417.7613188410721 109.01512026970283 303.2546914433517 102.99998474121094 372.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1062)"></path></g><g id="SvgjsG1064"><path id="SvgjsPath1065" d="M102.99998474121094 442.0080051422119C 108.29905390026858 502.57664278619006 108.29905390026858 533.4393674982338 102.99998474121094 594.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1066)"></path></g><g id="SvgjsG1068"><path id="SvgjsPath1069" d="M273.99998474121094 349.0080051422119C 284.41593834944956 468.0628996675149 113.41593834944956 474.95311061690893 102.99998474121094 594.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1070)"></path></g><g id="SvgjsG1072"><path id="SvgjsPath1073" d="M273.99998474121094 181.0080051422119C 212.99998474121094 264.0080051422119 111.93738864405282 269.8530110818261 102.99998474121094 372.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1074)"></path></g><g id="SvgjsG1076"><path id="SvgjsPath1077" d="M102.99998474121094 442.0080051422119C 109.2110530012738 513.0008402107513 280.21105300127385 421.0151700736725 273.99998474121094 492.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1078)"></path></g><g id="SvgjsG1080"><path id="SvgjsPath1081" d="M273.99998474121094 562.0080051422119C 280.06492236717565 631.3305594199732 109.0649223671756 524.6854508644507 102.99998474121094 594.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1082)"></path></g><g id="SvgjsG1084"><path id="SvgjsPath1085" d="M102.99998474121094 95.00800514221191C 24.999984741210938 329.0080051422119 30.999984741210938 425.0080051422119 102.99998474121094 594.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1086)"></path></g><g id="SvgjsG1088"><path id="SvgjsPath1089" d="M273.99998474121094 181.0080051422119C 222.99998474121094 348.0080051422119 118.5834705403015 415.88794739926834 102.99998474121094 594.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1090)"></path></g><g id="SvgjsG1092"><path id="SvgjsPath1093" d="M273.99998474121094 181.0080051422119C 319.99998474121094 315.0080051422119 358.99998474121094 344.0080051422119 273.99998474121094 492.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1094)"></path></g><g id="SvgjsG1096"><path id="SvgjsPath1097" d="M371.99998474121094 529.0080051422119L417.99998474121094 529.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1098)"></path></g><g id="SvgjsG1100" transform="translate(435.99998474121094,504.0080051422119)"><path id="SvgjsPath1101" d="M 0 0L 238 0L 238 46L 0 46Z" stroke="none" fill="none"></path><g id="SvgjsG1102"><text id="SvgjsText1103" font-family="微软雅黑" text-anchor="start" font-size="13px" width="238px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-1.95" transform="rotate(0)"><tspan id="SvgjsTspan1104" dy="16" x="0"><tspan id="SvgjsTspan1105" style="text-decoration:;">choose one operation from MaxPool, </tspan></tspan><tspan id="SvgjsTspan1106" dy="16" x="0"><tspan id="SvgjsTspan1107" style="text-decoration:;">AvgPool, SepConvBN3x3, SepConvBN</tspan></tspan><tspan id="SvgjsTspan1108" dy="16" x="0"><tspan id="SvgjsTspan1109" style="text-decoration:;">5x5, SkipConnect</tspan></tspan></text></g></g><g id="SvgjsG1110" transform="translate(371.99998474121094,562.0080051422119)"><path id="SvgjsPath1111" d="M 0 25.5C 0 -8.5 51 -8.5 51 25.5C 51 59.5 0 59.5 0 25.5Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1112"><text id="SvgjsText1113" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="31px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.05" transform="rotate(0)"></text></g></g><g id="SvgjsG1114" transform="translate(371.99998474121094,637.0080051422119)"><path id="SvgjsPath1115" d="M 0 25.5C 0 -8.5 51 -8.5 51 25.5C 51 59.5 0 59.5 0 25.5Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc66"></path><g id="SvgjsG1116"><text id="SvgjsText1117" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="31px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.05" transform="rotate(0)"></text></g></g><g id="SvgjsG1118" transform="translate(435.99998474121094,566.5080051422119)"><path id="SvgjsPath1119" d="M 0 0L 244 0L 244 42L 0 42Z" stroke="none" fill="none"></path><g id="SvgjsG1120"><text id="SvgjsText1121" font-family="微软雅黑" text-anchor="start" font-size="13px" width="244px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="3.55" transform="rotate(0)"><tspan id="SvgjsTspan1122" dy="16" x="0"><tspan id="SvgjsTspan1123" style="text-decoration:;">accept two operations from all inputs, </tspan></tspan><tspan id="SvgjsTspan1124" dy="16" x="0"><tspan id="SvgjsTspan1125" style="text-decoration:;">then add the results as output</tspan></tspan></text></g></g><g id="SvgjsG1126" transform="translate(435.99998474121094,642.5080051422119)"><path id="SvgjsPath1127" d="M 0 0L 224 0L 224 40L 0 40Z" stroke="none" fill="none"></path><g id="SvgjsG1128"><text id="SvgjsText1129" font-family="微软雅黑" text-anchor="start" font-size="13px" width="224px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="10.55" transform="rotate(0)"><tspan id="SvgjsTspan1130" dy="16" x="0"><tspan id="SvgjsTspan1131" style="text-decoration:;">results of previous layers as input</tspan></tspan></text></g></g><g id="SvgjsG1132"><path id="SvgjsPath1133" d="M102.99998474121094 95.00800514221191C 118.06961340198137 267.2546489180095 289.0696134019814 319.76136136641435 273.99998474121094 492.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1134)"></path></g><g id="SvgjsG1136" transform="translate(150.99998474121094,729.0880031585693)"><path id="SvgjsPath1137" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke-dasharray="3,4" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#cc99ff"></path><g id="SvgjsG1138"><text id="SvgjsText1139" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1140" dy="16" x="35"><tspan id="SvgjsTspan1141" style="text-decoration:;">output</tspan></tspan></text></g></g><g id="SvgjsG1142"><path id="SvgjsPath1143" d="M102.99998474121094 442.0080051422119C 113.41815075849257 561.0881876185914 196.41815075849257 610.0078206821898 185.99998474121094 729.0880031585693" stroke-dasharray="3,3" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1144)"></path></g><g id="SvgjsG1146"><path id="SvgjsPath1147" d="M102.99998474121094 664.0080051422119C 106.67699141184806 706.0363837051965 189.67699141184806 687.0596245955848 185.99998474121094 729.0880031585693" stroke-dasharray="3,3" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1148)"></path></g><g id="SvgjsG1150"><path id="SvgjsPath1151" d="M273.99998474121094 562.0080051422119C 280.58330524633993 637.2557028416771 192.58330524633993 653.8403054591041 185.99998474121094 729.0880031585693" stroke-dasharray="3,3" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1152)"></path></g><g id="SvgjsG1154"><path id="SvgjsPath1155" d="M273.99998474121094 349.0080051422119C 233.99998474121094 500.92799377441406 199.60096408433088 573.6280978979321 185.99998474121094 729.0880031585693" stroke-dasharray="3,3" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1156)"></path></g><g id="SvgjsG1158"><path id="SvgjsPath1159" d="M369.99998474121094 477.92799377441406L394.49998474121094 477.92799377441406L394.49998474121094 477.92799377441406L418.99998474121094 477.92799377441406" stroke-dasharray="3,3" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1160)"></path></g><g id="SvgjsG1162" transform="translate(435.99998474121094,448.0080051422119)"><path id="SvgjsPath1163" d="M 0 0L 243 0L 243 50.91998863220215L 0 50.91998863220215Z" stroke="none" fill="none"></path><g id="SvgjsG1164"><text id="SvgjsText1165" font-family="微软雅黑" text-anchor="start" font-size="13px" width="243px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.009994316101075" transform="rotate(0)"><tspan id="SvgjsTspan1166" dy="16" x="0"><tspan id="SvgjsTspan1167" style="text-decoration:;">pass the result directly to the output</tspan></tspan></text></g></g><g id="SvgjsG1168" transform="translate(371.99998474121094,709.0080051422119)"><path id="SvgjsPath1169" d="M 0 25.5C 0 -8.5 51 -8.5 51 25.5C 51 59.5 0 59.5 0 25.5Z" stroke-dasharray="3,4" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#cc99ff"></path><g id="SvgjsG1170"><text id="SvgjsText1171" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="31px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.05" transform="rotate(0)"></text></g></g><g id="SvgjsG1172" transform="translate(438.99998474121094,729.0880031585693)"><path id="SvgjsPath1173" d="M 0 0L 240 0L 240 19.920001983642578L 0 19.920001983642578Z" stroke="none" fill="none"></path><g id="SvgjsG1174"><text id="SvgjsText1175" font-family="微软雅黑" text-anchor="start" font-size="13px" width="240px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-30.48999900817871" transform="rotate(0)"><tspan id="SvgjsTspan1176" dy="16" x="0"><tspan id="SvgjsTspan1177" style="text-decoration:;">(the node does not exist) </tspan><tspan id="SvgjsTspan1178" style="text-decoration:;font-weight: bold;">Only accept </tspan></tspan><tspan id="SvgjsTspan1179" dy="16" x="0"><tspan id="SvgjsTspan1180" style="text-decoration:;font-weight: bold;">input whose starting point is not </tspan></tspan><tspan id="SvgjsTspan1181" dy="16" x="0"><tspan id="SvgjsTspan1182" style="text-decoration:;font-weight: bold;">served as input for any other node.</tspan><tspan id="SvgjsTspan1183" style="text-decoration:;"> </tspan></tspan><tspan id="SvgjsTspan1184" dy="16" x="0"><tspan id="SvgjsTspan1185" style="text-decoration:;">It calculates inputs' average value as </tspan></tspan><tspan id="SvgjsTspan1186" dy="16" x="0"><tspan id="SvgjsTspan1187" style="text-decoration:;">the output of this cell.</tspan></tspan></text></g></g></svg>
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment