the source code of NNI for DCU

docs/en_US/hyperparameter_tune.rst

+#############################
+Auto (Hyper-parameter) Tuning
+#############################
+
+Auto tuning is one of the key features provided by NNI; a main application scenario being
+hyper-parameter tuning. Tuning specifically applies to trial code. We provide a lot of popular
+auto tuning algorithms (called Tuner), and some early stop algorithms (called Assessor).
+NNI supports running trials on various training platforms, for example, on a local machine,
+on several servers in a distributed manner, or on platforms such as OpenPAI, Kubernetes, etc.
+
+Other key features of NNI, such as model compression, feature engineering, can also be further
+enhanced by auto tuning, which we'll described when introducing those features.
+
+NNI has high extensibility, advanced users can customize their own Tuner, Assessor, and Training Service
+according to their needs.
+
+..  toctree::
+    :maxdepth: 2
+
+    Write Trial <TrialExample/Trials>
+    Tuners <builtin_tuner>
+    Assessors <builtin_assessor>
+    Training Platform <training_services>
+    Examples <examples>
+    WebUI <Tutorial/WebUI>
+    How to Debug <Tutorial/HowToDebug>
+    Advanced <hpo_advanced>
+    HPO Benchmarks <hpo_benchmark>

docs/en_US/installation.rst

+############
+Installation
+############
+
+Currently we support installation on Linux, Mac and Windows. We also allow you to use docker.
+
+..  toctree::
+    :maxdepth: 2
+
+    Linux & Mac <Tutorial/InstallationLinux>
+    Windows <Tutorial/InstallationWin>
+    Use Docker <Tutorial/HowToUseDocker>
\ No newline at end of file

docs/en_US/model_compression.rst

+#################
+Model Compression
+#################
+
+Deep neural networks (DNNs) have achieved great success in many tasks. However, typical neural networks are both
+computationally expensive and energy intensive, can be difficult to be deployed on devices with low computation
+resources or with strict latency requirements. Therefore, a natural thought is to perform model compression to
+reduce model size and accelerate model training/inference without losing performance significantly. Model compression
+techniques can be divided into two categories: pruning and quantization. The pruning methods explore the redundancy
+in the model weights and try to remove/prune the redundant and uncritical weights. Quantization refers to compressing
+models by reducing the number of bits required to represent weights or activations.
+
+NNI provides an easy-to-use toolkit to help user design and use model pruning and quantization algorithms.
+It supports Tensorflow and PyTorch with unified interface.
+For users to compress their models, they only need to add several lines in their code.
+There are some popular model compression algorithms built-in in NNI.
+Users could further use NNI's auto tuning power to find the best compressed model,
+which is detailed in Auto Model Compression.
+On the other hand, users could easily customize their new compression algorithms using NNI's interface.
+
+For details, please refer to the following tutorials:
+
+..  toctree::
+    :maxdepth: 2
+
+    Overview <Compression/Overview>
+    Quick Start <Compression/QuickStart>
+    Tutorial <Compression/Tutorial>
+    Pruning <Compression/pruning>
+    Pruning V2 <Compression/v2_pruning>
+    Quantization <Compression/quantization>
+    Utilities <Compression/CompressionUtils>
+    Advanced Usage <Compression/advanced>
+    API Reference <Compression/CompressionReference>

docs/en_US/nas.rst

+#############################################
+Retiarii for Neural Architecture Search (NAS)
+#############################################
+
+Automatic neural architecture search is taking an increasingly important role on finding better models.
+Recent research works have proved the feasibility of automatic NAS, and also found some models that could beat manually tuned models.
+Some of representative works are NASNet, ENAS, DARTS, Network Morphism, and Evolution. Moreover, new innovations keep emerging.
+
+However, it takes great efforts to implement NAS algorithms, and it is hard to reuse code base of existing algorithms in a new one.
+To facilitate NAS innovations (e.g., design and implement new NAS models, compare different NAS models side-by-side),
+an easy-to-use and flexible programming interface is crucial.
+
+Thus, we design `Retiarii <https://www.usenix.org/system/files/osdi20-zhang_quanlu.pdf>`__. It is a deep learning framework that supports the exploratory training on a neural network model space, rather than on a single neural network model.
+Exploratory training with Retiarii allows user to express various search spaces for *Neural Architecture Search* and *Hyper-Parameter Tuning* with high flexibility.
+
+Some frequently used terminologies in this document:
+
+* *Model search space*: it means a set of models from which the best model is explored/searched. Sometimes we use *search space* or *model space* in short.
+* *Exploration strategy*: the algorithm that is used to explore a model search space.
+* *Model evaluator*: it is used to train a model and evaluate the model's performance.
+
+Follow the instructions below to start your journey with Retiarii.
+
+..  toctree::
+    :maxdepth: 2
+
+    Overview <NAS/Overview>
+    Quick Start <NAS/QuickStart>
+    Construct Model Space <NAS/construct_space>
+    Multi-trial NAS <NAS/multi_trial_nas>
+    One-shot NAS <NAS/one_shot_nas>
+    Hardware-aware NAS <NAS/HardwareAwareNAS>
+    NAS Benchmarks <NAS/Benchmarks>
+    NAS API References <NAS/ApiReference>

docs/en_US/reference.rst

+References
+==================
+
+..  toctree::
+    :maxdepth: 2
+
+    nnictl Commands <Tutorial/Nnictl>
+    Experiment Configuration <reference/experiment_config>
+    Experiment Configuration (legacy) <Tutorial/ExperimentConfig>
+    Search Space <Tutorial/SearchSpaceSpec>
+    NNI Annotation <Tutorial/AnnotationSpec>
+    SDK API References <sdk_reference>
+    Supported Framework Library <SupportedFramework_Library>
+    Launch from Python <Tutorial/HowToLaunchFromPython>
+    Shared Storage <Tutorial/HowToUseSharedStorage>
+    Tensorboard <Tutorial/Tensorboard>

docs/en_US/reference/experiment_config.rst

+===========================
+Experiment Config Reference
+===========================
+
+A config file is needed when creating an experiment. This document describes the rules to write a config file and provides some examples.
+
+.. Note::
+
+    1. This document lists field names with ``camelCase``. If users use these fields in the pythonic way with NNI Python APIs (e.g., ``nni.experiment``), the field names should be converted to ``snake_case``.
+
+    2. In this document, the type of fields are formatted as `Python type hint <https://docs.python.org/3.10/library/typing.html>`_. Therefore JSON objects are called `dict` and arrays are called `list`.
+
+    .. _path: 
+
+    3. Some fields take a path to a file or directory. Unless otherwise noted, both absolute path and relative path are supported, and ``~`` will be expanded to the home directory.
+
+       - When written in the YAML file, relative paths are relative to the directory containing that file.
+       - When assigned in Python code, relative paths are relative to the current working directory.
+       - All relative paths are converted to absolute when loading YAML file into Python class, and when saving Python class to YAML file.
+
+    4. Setting a field to ``None`` or ``null`` is equivalent to not setting the field.
+
+.. contents:: Contents
+   :local:
+   :depth: 3
+ 
+
+Examples
+========
+
+Local Mode
+^^^^^^^^^^
+
+.. code-block:: yaml
+
+    experimentName: MNIST
+    searchSpaceFile: search_space.json
+    trialCommand: python mnist.py
+    trialCodeDirectory: .
+    trialGpuNumber: 1
+    trialConcurrency: 2
+    maxExperimentDuration: 24h
+    maxTrialNumber: 100
+    tuner:
+      name: TPE
+      classArgs:
+        optimize_mode: maximize
+    trainingService:
+      platform: local
+      useActiveGpu: True
+
+Local Mode (Inline Search Space)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. code-block:: yaml
+
+    searchSpace:
+      batch_size:
+        _type: choice
+        _value: [16, 32, 64]
+      learning_rate:
+        _type: loguniform
+        _value: [0.0001, 0.1]
+    trialCommand: python mnist.py
+    trialGpuNumber: 1
+    trialConcurrency: 2
+    tuner:
+      name: TPE
+      classArgs:
+        optimize_mode: maximize
+    trainingService:
+      platform: local
+      useActiveGpu: True
+
+Remote Mode
+^^^^^^^^^^^
+
+.. code-block:: yaml
+
+    experimentName: MNIST
+    searchSpaceFile: search_space.json
+    trialCommand: python mnist.py
+    trialCodeDirectory: .
+    trialGpuNumber: 1
+    trialConcurrency: 2
+    maxExperimentDuration: 24h
+    maxTrialNumber: 100
+    tuner:
+      name: TPE
+      classArgs:
+        optimize_mode: maximize
+    trainingService:
+      platform: remote
+      machineList:
+        - host: 11.22.33.44
+          user: alice
+          password: xxxxx
+        - host: my.domain.com
+          user: bob
+          sshKeyFile: ~/.ssh/id_rsa
+
+Reference
+=========
+
+ExperimentConfig
+^^^^^^^^^^^^^^^^
+
+.. list-table::
+    :widths: 10 10 80
+    :header-rows: 1
+
+    * - Field Name
+      - Type
+      - Description
+    
+    * - experimentName
+      - ``str``, optional
+      - Mnemonic name of the experiment, which will be shown in WebUI and nnictl.
+
+    * - searchSpaceFile
+      - ``str``, optional
+      - Path_ to the JSON file containing the search space.
+        Search space format is determined by tuner. The common format for built-in tuners is documented  `here <../Tutorial/SearchSpaceSpec.rst>`__.
+        Mutually exclusive to ``searchSpace``.
+
+    * - searchSpace
+      - ``JSON``, optional
+      - Search space object.
+        The format is determined by tuner. Common format for built-in tuners is documented `here <../Tutorial/SearchSpaceSpec.rst>`__.
+        Note that ``None`` means "no such field" so empty search space should be written as ``{}``.
+        Mutually exclusive to ``searchSpaceFile``.
+
+    * - trialCommand
+      - ``str``
+      - Command to launch trial.
+        The command will be executed in bash on Linux and macOS, and in PowerShell on Windows.
+        Note that using ``python3`` on Linux and macOS, and using ``python`` on Windows.
+
+    * - trialCodeDirectory
+      - ``str``, optional
+      - Default: ``"."``. `Path`_ to the directory containing trial source files.
+        All files in this directory will be sent to the training machine, unless in the ``.nniignore`` file.
+        (See :ref:`nniignore <nniignore>` for details.)
+
+    * - trialConcurrency
+      - ``int``
+      - Specify how many trials should be run concurrently.
+        The real concurrency also depends on hardware resources and may be less than this value.
+
+    * - trialGpuNumber
+      - ``int`` or ``None``, optional
+      - Default: None. This field might have slightly different meanings for various training services,
+        especially when set to ``0`` or ``None``.
+        See `training service's document <../training_services.rst>`__ for details.
+
+        In local mode, setting the field to ``0`` will prevent trials from accessing GPU (by empty ``HIP_VISIBLE_DEVICES``).
+        And when set to ``None``, trials will be created and scheduled as if they did not use GPU,
+        but they can still use all GPU resources if they want.
+
+    * - maxExperimentDuration
+      - ``str``, optional
+      - Limit the duration of this experiment if specified. The duration is unlimited if not set.
+        Format: ``number + s|m|h|d``.
+        Examples: ``"10m"``, ``"0.5h"``.
+        When time runs out, the experiment will stop creating trials but continue to serve WebUI.
+
+    * - maxTrialNumber
+      - ``int``, optional
+      - Limit the number of trials to create if specified. The trial number is unlimited if not set.
+        When the budget runs out, the experiment will stop creating trials but continue to serve WebUI.
+
+    * - maxTrialDuration
+      - ``str``, optional
+      - Limit the duration of trial job if specified. The duration is unlimited if not set.
+        Format: ``number + s|m|h|d``.
+        Examples: ``"10m"``, ``"0.5h"``.
+        When time runs out, the current trial job will stop.
+
+    * - nniManagerIp
+      - ``str``, optional
+      - Default: default connection chosen by system. IP of the current machine, used by training machines to access NNI manager. Not used in local mode.
+        Except for the local mode, it is highly recommended to set this field manually.
+
+    * - useAnnotation
+      - ``bool``, optional
+      - Default: ``False``. Enable `annotation <../Tutorial/AnnotationSpec.rst>`__.
+        When using annotation, ``searchSpace`` and ``searchSpaceFile`` should not be specified manually.
+
+    * - debug
+      - ``bool``, optional
+      - Default: ``False``. Enable debug mode.
+        When enabled, logging will be more verbose and some internal validation will be loosened.
+
+    * - logLevel
+      - ``str``, optional
+      - Default: ``info`` or ``debug``, depending on ``debug`` option. Set log level of the whole system.
+        values: ``"trace"``, ``"debug"``, ``"info"``, ``"warning"``, ``"error"``, ``"fatal"``
+        When debug mode is enabled, Loglevel is set to "debug", otherwise, Loglevel is set to "info".
+        Most modules of NNI will be affected by this value, including NNI manager, tuner, training service, etc.
+        The exception is trial, whose logging level is directly managed by trial code.
+        For Python modules, "trace" acts as logging level 0 and "fatal" acts as ``logging.CRITICAL``.
+
+    * - experimentWorkingDirectory
+      - ``str``, optional
+      - Default: ``~/nni-experiments``.
+        Specify the :ref:`directory <path>` to place log, checkpoint, metadata, and other run-time stuff.
+        NNI will create a subdirectory named by experiment ID, so it is safe to use the same directory for multiple experiments.
+
+    * - tunerGpuIndices
+      - ``list[int]`` or ``str`` or ``int``, optional
+      - Limit the GPUs visible to tuner, assessor, and advisor.
+        This will be the ``HIP_VISIBLE_DEVICES`` environment variable of tuner process.
+        Because tuner, assessor, and advisor run in the same process, this option will affect them all.
+
+    * - tuner
+      - ``AlgorithmConfig``, optional
+      - Specify the tuner.
+        The built-in tuners can be found `here <../builtin_tuner.rst>`__ and you can follow `this tutorial <../Tuner/CustomizeTuner.rst>`__ to customize a new tuner.
+
+    * - assessor
+      - ``AlgorithmConfig``, optional
+      - Specify the assessor.
+        The built-in assessors can be found `here <../builtin_assessor.rst>`__ and you can follow `this tutorial <../Assessor/CustomizeAssessor.rst>`__ to customize a new assessor.
+
+    * - advisor
+      - ``AlgorithmConfig``, optional
+      - Specify the advisor.
+        NNI provides two built-in advisors: `BOHB <../Tuner/BohbAdvisor.rst>`__ and `Hyperband <../Tuner/HyperbandAdvisor.rst>`__, and you can follow `this tutorial <../Tuner/CustomizeAdvisor.rst>`__ to customize a new advisor.
+
+    * - trainingService
+      - ``TrainingServiceConfig``
+      - Specify the `training service <../TrainingService/Overview.rst>`__.
+
+    * - sharedStorage
+      - ``SharedStorageConfig``, optional
+      - Configure the shared storage, detailed usage can be found `here <../Tutorial/HowToUseSharedStorage.rst>`__.
+
+AlgorithmConfig
+^^^^^^^^^^^^^^^
+
+``AlgorithmConfig`` describes a tuner / assessor / advisor algorithm.
+
+For customized algorithms, there are two ways to describe them:
+
+  1. `Register the algorithm <../Tutorial/InstallCustomizedAlgos.rst>`__ to use it like built-in. (preferred)
+
+  2. Specify code directory and class name directly.
+
+.. list-table::
+    :widths: 10 10 80
+    :header-rows: 1
+
+    * - Field Name
+      - Type
+      - Description
+    
+    * - name
+      - ``str`` or ``None``, optional
+      - Default: None. Name of the built-in or registered algorithm.
+        ``str`` for the built-in and registered algorithm, ``None`` for other customized algorithms.
+
+    * - className
+      - ``str`` or ``None``, optional
+      - Default: None. Qualified class name of not registered customized algorithm.
+        ``None`` for the built-in and registered algorithm, ``str`` for other customized algorithms.
+        example: ``"my_tuner.MyTuner"``
+
+    * - codeDirectory
+      - ``str`` or ``None``, optional
+      - Default: None. Path_ to the directory containing the customized algorithm class.
+        ``None`` for the built-in and registered algorithm, ``str`` for other customized algorithms.
+
+    * - classArgs
+      - ``dict[str, Any]``, optional
+      - Keyword arguments passed to algorithm class' constructor.
+        See algorithm's document for supported value.
+
+TrainingServiceConfig
+^^^^^^^^^^^^^^^^^^^^^
+
+One of the following:
+
+- `LocalConfig`_
+- `RemoteConfig`_
+- `OpenpaiConfig`_
+- `AmlConfig`_
+- `DlcConfig`_
+- `HybridConfig`_
+
+For `Kubeflow <../TrainingService/KubeflowMode.rst>`_, `FrameworkController <../TrainingService/FrameworkControllerMode.rst>`_, and `AdaptDL <../TrainingService/AdaptDLMode.rst>`_ training platforms, it is suggested to use `v1 config schema <../Tutorial/ExperimentConfig.rst>`_ for now.
+
+LocalConfig
+-----------
+
+Detailed usage can be found `here <../TrainingService/LocalMode.rst>`__.
+
+.. list-table::
+    :widths: 10 10 80
+    :header-rows: 1
+
+    * - Field Name
+      - Type
+      - Description
+
+    * - platform
+      - ``"local"``
+      -
+    
+    * - useActiveGpu
+      - ``bool``, optional
+      - Default: ``False``. Specify whether NNI should submit trials to GPUs occupied by other tasks.
+        Must be set when ``trialGpuNumber`` greater than zero.
+        Following processes can make GPU "active":
+
+          - non-NNI HIP programs
+          - graphical desktop
+          - trials submitted by other NNI instances, if you have more than one NNI experiments running at same time
+          - other users' HIP programs, if you are using a shared server
+          
+        If you are using a graphical OS like Windows 10 or Ubuntu desktop, set this field to ``True``, otherwise, the GUI will prevent NNI from launching any trial.
+        When you create multiple NNI experiments and ``useActiveGpu`` is set to ``True``, they will submit multiple trials to the same GPU(s) simultaneously.
+
+    * - maxTrialNumberPerGpu
+      - ``int``, optional
+      - Default: ``1``. Specify how many trials can share one GPU.
+
+    * - gpuIndices
+      - ``list[int]`` or ``str`` or ``int``, optional
+      - Limit the GPUs visible to trial processes.
+        If ``trialGpuNumber`` is less than the length of this value, only a subset will be visible to each trial.
+        This will be used as ``HIP_VISIBLE_DEVICES`` environment variable.
+
+RemoteConfig
+------------
+
+Detailed usage can be found `here <../TrainingService/RemoteMachineMode.rst>`__.
+
+.. list-table::
+    :widths: 10 10 80
+    :header-rows: 1
+
+    * - Field Name
+      - Type
+      - Description
+
+    * - platform
+      - ``"remote"``
+      -
+
+    * - machineList
+      - ``List[RemoteMachineConfig]``
+      - List of training machines.
+
+    * - reuseMode
+      - ``bool``, optional
+      - Default: ``True``. Enable `reuse mode <../TrainingService/Overview.rst#training-service-under-reuse-mode>`__.
+
+RemoteMachineConfig
+"""""""""""""""""""
+
+.. list-table::
+    :widths: 10 10 80
+    :header-rows: 1
+
+    * - Field Name
+      - Type
+      - Description
+
+    * - host
+      - ``str``
+      - IP or hostname (domain name) of the machine.
+
+    * - port
+      - ``int``, optional
+      - Default: ``22``. SSH service port.
+
+    * - user
+      - ``str``
+      - Login user name.
+
+    * - password
+      - ``str``, optional
+      - If not specified, ``sshKeyFile`` will be used instead.
+    
+    * - sshKeyFile
+      - ``str``, optional
+      - `Path`_ to ``sshKeyFile`` (identity file).
+        Only used when ``password`` is not specified.
+
+    * - sshPassphrase
+      - ``str``, optional
+      - Passphrase of SSH identity file.
+
+    * - useActiveGpu
+      - ``bool``, optional
+      - Default: ``False``. Specify whether NNI should submit trials to GPUs occupied by other tasks.
+        Must be set when ``trialGpuNumber`` greater than zero.
+        Following processes can make GPU "active":
+
+          - non-NNI CUDA programs
+          - graphical desktop
+          - trials submitted by other NNI instances, if you have more than one NNI experiments running at same time
+          - other users' CUDA programs, if you are using a shared server
+  
+        If your remote machine is a graphical OS like Ubuntu desktop, set this field to ``True``, otherwise, the GUI will prevent NNI from launching any trial.
+        When you create multiple NNI experiments and ``useActiveGpu`` is set to ``True``, they will submit multiple trials to the same GPU(s) simultaneously.
+
+    * - maxTrialNumberPerGpu
+      - ``int``, optional
+      - Default: ``1``. Specify how many trials can share one GPU.
+
+    * - gpuIndices
+      - ``list[int]`` or ``str`` or ``int``, optional
+      - Limit the GPUs visible to trial processes.
+        If ``trialGpuNumber`` is less than the length of this value, only a subset will be visible to each trial.
+        This will be used as ``HIP_VISIBLE_DEVICES`` environment variable.
+
+    * - pythonPath
+      - ``str``, optional
+      - Specify a Python environment.
+        This path will be inserted at the front of PATH. Here are some examples: 
+
+          - (linux) pythonPath: ``/opt/python3.7/bin``
+          - (windows) pythonPath: ``C:/Python37``
+
+        If you are working on Anaconda, there is some difference. On Windows, you also have to add ``../script`` and ``../Library/bin`` separated by ``;``. Examples are as below:
+
+          - (linux anaconda) pythonPath: ``/home/yourname/anaconda3/envs/myenv/bin/``
+          - (windows anaconda) pythonPath: ``C:/Users/yourname/.conda/envs/myenv``; ``C:/Users/yourname/.conda/envs/myenv/Scripts``; ``C:/Users/yourname/.conda/envs/myenv/Library/bin``
+
+        This is useful if preparing steps vary for different machines.
+
+OpenpaiConfig
+-------------
+
+Detailed usage can be found `here <../TrainingService/PaiMode.rst>`__.
+
+.. list-table::
+    :widths: 10 10 80
+    :header-rows: 1
+
+    * - Field Name
+      - Type
+      - Description
+
+    * - platform
+      - ``"openpai"``
+      -
+    
+    * - host
+      - ``str``
+      - Hostname of OpenPAI service.
+        This may include ``https://`` or ``http://`` prefix.
+        HTTPS will be used by default.
+
+    * - username
+      - ``str``
+      - OpenPAI user name.
+
+    * - token
+      - ``str``
+      - OpenPAI user token.
+        This can be found in your OpenPAI user settings page.
+
+    * - trialCpuNumber
+      - ``int``
+      - Specify the CPU number of each trial to be used in OpenPAI container.
+
+    * - trialMemorySize
+      - ``str``
+      - Specify the memory size of each trial to be used in OpenPAI container.
+        format: ``number + tb|gb|mb|kb``.
+        examples: ``"8gb"``, ``"8192mb"``.
+
+    * - storageConfigName
+      - ``str``
+      - Specify the storage name used in OpenPAI.
+
+    * - dockerImage
+      - ``str``, optional
+      - Default: ``"msranni/nni:latest"``. Name and tag of docker image to run the trials.
+
+    * - localStorageMountPoint
+      - ``str``
+      - :ref:`Mount point <path>` of storage service (typically NFS) on the local machine.
+
+    * - containerStorageMountPoint
+      - ``str``
+      - Mount point of storage service (typically NFS) in docker container.
+        This must be an absolute path.
+
+    * - reuseMode
+      - ``bool``, optional
+      - Default: ``True``. Enable `reuse mode <../TrainingService/Overview.rst#training-service-under-reuse-mode>`__.
+
+    * - openpaiConfig
+      - ``JSON``, optional
+      - Embedded OpenPAI config file.
+
+    * - openpaiConfigFile
+      - ``str``, optional
+      - `Path`_ to OpenPAI config file.
+        An example can be found `here <https://github.com/microsoft/pai/blob/master/docs/manual/cluster-user/examples/hello-world-job.yaml>`__.
+
+AmlConfig
+---------
+
+Detailed usage can be found `here <../TrainingService/AMLMode.rst>`__.
+
+.. list-table::
+    :widths: 10 10 80
+    :header-rows: 1
+
+    * - Field Name
+      - Type
+      - Description
+
+    * - platform
+      - ``"aml"``
+      -
+
+    * - dockerImage
+      - ``str``, optional
+      - Default: ``"msranni/nni:latest"``. Name and tag of docker image to run the trials.
+
+    * - subscriptionId
+      - ``str``
+      - Azure subscription ID.
+
+    * - resourceGroup
+      - ``str``
+      - Azure resource group name.
+
+    * - workspaceName
+      - ``str``
+      - Azure workspace name.
+
+    * - computeTarget
+      - ``str``
+      - AML compute cluster name.
+
+DlcConfig
+---------
+
+Detailed usage can be found `here <../TrainingService/DlcMode.rst>`__.
+
+.. list-table::
+    :widths: 10 10 80
+    :header-rows: 1
+
+    * - Field Name
+      - Type
+      - Description
+
+    * - platform
+      - ``"dlc"``
+      -
+    
+    * - type
+      - ``str``, optional
+      - Default: ``"Worker"``. Job spec type.
+
+    * - image
+      - ``str``
+      - Name and tag of docker image to run the trials.
+
+    * - jobType
+      - ``str``, optional
+      - Default: ``"TFJob"``. PAI-DLC training job type, ``"TFJob"`` or ``"PyTorchJob"``.
+
+    * - podCount
+      - ``str``
+      - Pod count to run a single training job.
+
+    * - ecsSpec
+      - ``str``
+      - Training server config spec string.
+
+    * - region
+      - ``str``
+      - The region where PAI-DLC public-cluster locates.
+
+    * - nasDataSourceId
+      - ``str``
+      - The NAS datasource id configurated in PAI-DLC side.
+
+    * - accessKeyId
+      - ``str``
+      - The accessKeyId of your cloud account.
+
+    * - accessKeySecret
+      - ``str``
+      - The accessKeySecret of your cloud account.
+
+    * - localStorageMountPoint
+      - ``str``
+      - The mount point of the NAS on PAI-DSW server, default is /home/admin/workspace/.
+
+    * - containerStorageMountPoint
+      - ``str``
+      - The mount point of the NAS on PAI-DLC side, default is /root/data/.
+
+HybridConfig
+------------
+
+Currently only support `LocalConfig`_, `RemoteConfig`_, `OpenpaiConfig`_ and `AmlConfig`_ . Detailed usage can be found `here <../TrainingService/HybridMode.rst>`__.
+
+SharedStorageConfig
+^^^^^^^^^^^^^^^^^^^
+
+Detailed usage can be found `here <../Tutorial/HowToUseSharedStorage.rst>`__.
+
+nfsConfig
+---------
+
+.. list-table::
+    :widths: 10 10 80
+    :header-rows: 1
+
+    * - Field Name
+      - Type
+      - Description
+
+    * - storageType
+      - ``"NFS"``
+      -
+
+    * - localMountPoint
+      - ``str``
+      - The path that the storage has been or will be mounted in the local machine.
+        If the path does not exist, it will be created automatically. Recommended to use an absolute path, i.e. ``/tmp/nni-shared-storage``.
+
+    * - remoteMountPoint
+      - ``str``
+      - The path that the storage will be mounted in the remote machine.
+        If the path does not exist, it will be created automatically. Recommended to use a relative path. i.e. ``./nni-shared-storage``.
+
+    * - localMounted
+      - ``str``
+      - Specify the object and status to mount the shared storage.
+        values: ``"usermount"``, ``"nnimount"``, ``"nomount"``
+        ``usermount`` means the user has already mounted this storage on localMountPoint. ``nnimount`` means NNI will try to mount this storage on localMountPoint. ``nomount`` means storage will not mount in the local machine, will support partial storages in the future.
+
+    * - nfsServer
+      - ``str``
+      - NFS server host.
+
+    * - exportedDirectory
+      - ``str``
+      - Exported directory of NFS server, detailed `here <https://www.ibm.com/docs/en/aix/7.2?topic=system-nfs-exporting-mounting>`_.
+
+azureBlobConfig
+---------------
+
+.. list-table::
+    :widths: 10 10 80
+    :header-rows: 1
+
+    * - Field Name
+      - Type
+      - Description
+
+    * - storageType
+      - ``"AzureBlob"``
+      -
+
+    * - localMountPoint
+      - ``str``
+      - The path that the storage has been or will be mounted in the local machine.
+        If the path does not exist, it will be created automatically. Recommended to use an absolute path, i.e. ``/tmp/nni-shared-storage``.
+
+    * - remoteMountPoint
+      - ``str``
+      - The path that the storage will be mounted in the remote machine.
+        If the path does not exist, it will be created automatically. Recommended to use a relative path. i.e. ``./nni-shared-storage``.
+        Note that the directory must be empty when using AzureBlob.
+
+    * - localMounted
+      - ``str``
+      - Specify the object and status to mount the shared storage.
+        values: ``"usermount"``, ``"nnimount"``, ``"nomount"``.
+        ``usermount`` means the user has already mounted this storage on localMountPoint. ``nnimount`` means NNI will try to mount this storage on localMountPoint. ``nomount`` means storage will not mount in the local machine, will support partial storages in the future.
+
+    * - storageAccountName
+      - ``str``
+      - Azure storage account name.
+
+    * - storageAccountKey
+      - ``str``
+      - Azure storage account key.
+
+    * - containerName
+      - ``str``
+      - AzureBlob container name.

docs/en_US/sdk_reference.rst

+####################
+Python API Reference
+####################
+
+
+..  toctree::
+    :maxdepth: 1
+
+    Auto Tune <autotune_ref>
+    NAS <NAS/ApiReference>
+    Compression <Compression/CompressionReference>
+    Python API <Tutorial/HowToLaunchFromPython>
\ No newline at end of file

docs/en_US/training_services.rst

+Introduction to NNI Training Services
+=====================================
+
+..  toctree::
+    Overview <./TrainingService/Overview>
+    Local<./TrainingService/LocalMode>
+    Remote<./TrainingService/RemoteMachineMode>
+    OpenPAI<./TrainingService/PaiMode>
+    Kubeflow<./TrainingService/KubeflowMode>
+    AdaptDL<./TrainingService/AdaptDLMode>
+    FrameworkController<./TrainingService/FrameworkControllerMode>
+    DLTS<./TrainingService/DLTSMode>
+    AML<./TrainingService/AMLMode>
+    PAI-DLC<./TrainingService/DLCMode>
+    Hybrid<./TrainingService/HybridMode>

docs/img/NAS_Bench_201.svg

+<svg id="SvgjsSvg1006" width="709" height="537.0000305175781" xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:svgjs="http://svgjs.com/svgjs"><defs id="SvgjsDefs1007"><marker id="SvgjsMarker1034" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1035" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1038" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1039" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1042" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1043" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1046" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1047" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1050" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1051" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1054" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1055" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1080" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1081" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker></defs><g id="SvgjsG1008" transform="translate(25,24.999980449676514)"><path id="SvgjsPath1009" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc00"></path><g id="SvgjsG1010"><text id="SvgjsText1011" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1012" dy="16" x="35"><tspan id="SvgjsTspan1013" style="text-decoration:;">input</tspan></tspan></text></g></g><g id="SvgjsG1014" transform="translate(267,140.9999804496765)"><path id="SvgjsPath1015" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1016"><text id="SvgjsText1017" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1018" dy="16" x="35"><tspan id="SvgjsTspan1019" style="text-decoration:;">node 1</tspan></tspan></text></g></g><g id="SvgjsG1020" transform="translate(25,293.9999804496765)"><path id="SvgjsPath1021" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1022"><text id="SvgjsText1023" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1024" dy="16" x="35"><tspan id="SvgjsTspan1025" style="text-decoration:;">node 2</tspan></tspan></text></g></g><g id="SvgjsG1026" transform="translate(267,441.9999804496765)"><path id="SvgjsPath1027" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1028"><text id="SvgjsText1029" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1030" dy="16" x="35"><tspan id="SvgjsTspan1031" style="text-decoration:;">node 3</tspan></tspan></text></g></g><g id="SvgjsG1032"><path id="SvgjsPath1033" d="M60 94.99998044967651C 45 172.86665678024292 44 214.86665678024292 60 293.9999804496765" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1034)"></path></g><g id="SvgjsG1036"><path id="SvgjsPath1037" d="M60 94.99998044967651C 74.7485647305657 263.5768467107039 316.7485647305658 273.42311418864915 302 441.9999804496765" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1038)"></path></g><g id="SvgjsG1040"><path id="SvgjsPath1041" d="M302 210.9999804496765C 310.0531906298836 303.04837055335383 310.0531906298836 349.9515903459992 302 441.9999804496765" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1042)"></path></g><g id="SvgjsG1044"><path id="SvgjsPath1045" d="M60 363.9999804496765C 68.86407754147842 465.3168503645069 310.8640775414784 340.68311053484615 302 441.9999804496765" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1046)"></path></g><g id="SvgjsG1048"><path id="SvgjsPath1049" d="M60 94.99998044967651C 68.58773798957196 193.15827483289445 310.587737989572 42.841686066458635 302 140.9999804496765" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1050)"></path></g><g id="SvgjsG1052"><path id="SvgjsPath1053" d="M302 210.9999804496765C 310.9190947616536 312.94570006866127 68.91909476165358 192.05426083069176 60 293.9999804496765" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1054)"></path></g><g id="SvgjsG1056" transform="translate(382,279.8666567802429)"><path id="SvgjsPath1057" d="M 0 26C 0 -8.666666666666666 52 -8.666666666666666 52 26C 52 60.666666666666664 0 60.666666666666664 0 26Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1058"><text id="SvgjsText1059" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="32px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.55" transform="rotate(0)"></text></g></g><g id="SvgjsG1060" transform="translate(455,295.8666567802429)"><path id="SvgjsPath1061" d="M 0 0L 220 0L 220 20L 0 20Z" stroke="none" fill="none"></path><g id="SvgjsG1062"><text id="SvgjsText1063" font-family="微软雅黑" text-anchor="start" font-size="13px" width="220px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-7.45" transform="rotate(0)"><tspan id="SvgjsTspan1064" dy="16" x="0"><tspan id="SvgjsTspan1065" style="text-decoration:;">choose none/one/multiple input(s) </tspan></tspan><tspan id="SvgjsTspan1066" dy="16" x="0"><tspan id="SvgjsTspan1067" style="text-decoration:;">then add them as output</tspan></tspan></text></g></g><g id="SvgjsG1068" transform="translate(382,210.9999804496765)"><path id="SvgjsPath1069" d="M 0 26C 0 -8.666666666666666 52 -8.666666666666666 52 26C 52 60.666666666666664 0 60.666666666666664 0 26Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc00"></path><g id="SvgjsG1070"><text id="SvgjsText1071" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="32px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.55" transform="rotate(0)"></text></g></g><g id="SvgjsG1072" transform="translate(455,226.9999804496765)"><path id="SvgjsPath1073" d="M 0 0L 220 0L 220 20L 0 20Z" stroke="none" fill="none"></path><g id="SvgjsG1074"><text id="SvgjsText1075" font-family="微软雅黑" text-anchor="start" font-size="13px" width="220px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="0.55" transform="rotate(0)"><tspan id="SvgjsTspan1076" dy="16" x="0"><tspan id="SvgjsTspan1077" style="text-decoration:;">the output of the previous cell</tspan></tspan></text></g></g><g id="SvgjsG1078"><path id="SvgjsPath1079" d="M376 374.8666567802429L404.5 374.8666567802429L404.5 374.8666567802429L433 374.8666567802429" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1080)"></path></g><g id="SvgjsG1082" transform="translate(463,363.9999804496765)"><path id="SvgjsPath1083" d="M 0 0L 220 0L 220 20L 0 20Z" stroke="none" fill="none"></path><g id="SvgjsG1084"><text id="SvgjsText1085" font-family="微软雅黑" text-anchor="start" font-size="13px" width="220px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-14.95" transform="rotate(0)"><tspan id="SvgjsTspan1086" dy="16" x="0"><tspan id="SvgjsTspan1087" style="text-decoration:;">choose one operation from </tspan></tspan><tspan id="SvgjsTspan1088" dy="16" x="0"><tspan id="SvgjsTspan1089" style="text-decoration:;">MaxPool, AvgPool,, Conv1x1, </tspan></tspan><tspan id="SvgjsTspan1090" dy="16" x="0"><tspan id="SvgjsTspan1091" style="text-decoration:;">Conv3x3, SkipConnect, Zeroize</tspan></tspan></text></g></g></svg>
\ No newline at end of file

docs/img/NAS_Darts_cell.svg

+<svg id="SvgjsSvg1006" width="825.4666442871094" height="869.1333465576172" xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:svgjs="http://svgjs.com/svgjs"><defs id="SvgjsDefs1007"><marker id="SvgjsMarker1046" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1047" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1050" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1051" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1054" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1055" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1058" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1059" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1062" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1063" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1066" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1067" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1070" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1071" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1074" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1075" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1078" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1079" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1082" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1083" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1086" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1087" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1090" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1091" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1094" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1095" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1098" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1099" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1108" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1109" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1112" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1113" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1116" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1117" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1120" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1121" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1134" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1135" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1170" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1171" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker></defs><g id="SvgjsG1008" transform="translate(102.03997802734375,25.008010864257812)"><path id="SvgjsPath1009" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc66"></path><g id="SvgjsG1010"><text id="SvgjsText1011" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1012" dy="16" x="35"><tspan id="SvgjsTspan1013" style="text-decoration:;">input 0</tspan></tspan></text></g></g><g id="SvgjsG1014" transform="translate(346.03997802734375,91.00801086425781)"><path id="SvgjsPath1015" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc66"></path><g id="SvgjsG1016"><text id="SvgjsText1017" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1018" dy="16" x="35"><tspan id="SvgjsTspan1019" style="text-decoration:;">input 1</tspan></tspan></text></g></g><g id="SvgjsG1020" transform="translate(102.03997802734375,247.0080108642578)"><path id="SvgjsPath1021" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1022"><text id="SvgjsText1023" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1024" dy="16" x="35"><tspan id="SvgjsTspan1025" style="text-decoration:;">node 0</tspan></tspan></text></g></g><g id="SvgjsG1026" transform="translate(346.03997802734375,386.24000549316406)"><path id="SvgjsPath1027" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff33"></path><g id="SvgjsG1028"><text id="SvgjsText1029" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1030" dy="16" x="35"><tspan id="SvgjsTspan1031" style="text-decoration:;">node 1</tspan></tspan></text></g></g><g id="SvgjsG1032" transform="translate(102.03997802734375,503.24000549316406)"><path id="SvgjsPath1033" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff33"></path><g id="SvgjsG1034"><text id="SvgjsText1035" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1036" dy="16" x="35"><tspan id="SvgjsTspan1037" style="text-decoration:;">node 2</tspan></tspan></text></g></g><g id="SvgjsG1038" transform="translate(346.03997802734375,660.2400054931641)"><path id="SvgjsPath1039" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff33"></path><g id="SvgjsG1040"><text id="SvgjsText1041" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1042" dy="16" x="35"><tspan id="SvgjsTspan1043" style="text-decoration:;">node 3</tspan></tspan></text></g></g><g id="SvgjsG1044"><path id="SvgjsPath1045" d="M137.03997802734375 95.00801086425781C 142.3390471864014 155.57664850823596 142.3390471864014 186.43937322027966 137.03997802734375 247.0080108642578" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1046)"></path></g><g id="SvgjsG1048"><path id="SvgjsPath1049" d="M137.03997802734375 95.00801086425781C 150.28545233965042 246.404475028805 394.2854523396504 234.84354132861688 381.03997802734375 386.24000549316406" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1050)"></path></g><g id="SvgjsG1052"><path id="SvgjsPath1053" d="M381.03997802734375 161.0080108642578C 388.89208274031125 250.75797842023576 388.89208274031137 296.4900379371861 381.03997802734375 386.24000549316406" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1054)"></path></g><g id="SvgjsG1056"><path id="SvgjsPath1057" d="M381.03997802734375 161.0080108642578C 390.059279443249 264.0990977824248 146.05927944324907 143.91692394609086 137.03997802734375 247.0080108642578" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1058)"></path></g><g id="SvgjsG1060"><path id="SvgjsPath1061" d="M137.03997802734375 317.0080108642578C 143.53245315344782 391.21734113000457 143.53245315344788 429.0306752274173 137.03997802734375 503.24000549316406" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1062)"></path></g><g id="SvgjsG1064"><path id="SvgjsPath1065" d="M137.03997802734375 317.0080108642578C 145.8821637569588 418.0746562244878 389.8821637569588 285.1733601329341 381.03997802734375 386.24000549316406" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1066)"></path></g><g id="SvgjsG1068"><path id="SvgjsPath1069" d="M137.03997802734375 573.2400054931641C 146.07092807311756 676.4642368599839 390.07092807311756 557.0157741263442 381.03997802734375 660.2400054931641" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1070)"></path></g><g id="SvgjsG1072"><path id="SvgjsPath1073" d="M381.03997802734375 456.24000549316406C 388.1518866355526 537.5294928574505 388.1518866355527 578.9505181288777 381.03997802734375 660.2400054931641" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1074)"></path></g><g id="SvgjsG1076"><path id="SvgjsPath1077" d="M137.03997802734375 317.0080108642578C 151.72127748132198 484.81603149573016 395.721277481322 492.4319848616917 381.03997802734375 660.2400054931641" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1078)"></path></g><g id="SvgjsG1080"><path id="SvgjsPath1081" d="M381.03997802734375 456.24000549316406C 389.70275028689764 555.2559455067756 145.7027502868977 404.22406547955256 137.03997802734375 503.24000549316406" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1082)"></path></g><g id="SvgjsG1084"><path id="SvgjsPath1085" d="M381.03997802734375 161.0080108642578C 395.6928772148526 328.491414964573 151.69287721485261 335.7566013928489 137.03997802734375 503.24000549316406" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1086)"></path></g><g id="SvgjsG1088"><path id="SvgjsPath1089" d="M137.03997802734375 95.00801086425781C 24.999984741210938 260.9920015335083 77.99998474121094 362.9920015335083 137.03997802734375 503.24000549316406" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1090)"></path></g><g id="SvgjsG1092"><path id="SvgjsPath1093" d="M137.03997802734375 95.00801086425781C 158.5029026859753 340.33036228264245 402.5029026859754 414.9176540747794 381.03997802734375 660.2400054931641" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1094)"></path></g><g id="SvgjsG1096"><path id="SvgjsPath1097" d="M381.03997802734375 161.0080108642578C 474.99998474121094 357.7520065307617 436.99998474121094 463.7520065307617 381.03997802734375 660.2400054931641" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1098)"></path></g><g id="SvgjsG1100" transform="translate(213.99998474121094,774.1439895629883)"><path id="SvgjsPath1101" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#cc99ff"></path><g id="SvgjsG1102"><text id="SvgjsText1103" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1104" dy="16" x="35"><tspan id="SvgjsTspan1105" style="text-decoration:;">output</tspan></tspan></text></g></g><g id="SvgjsG1106"><path id="SvgjsPath1107" d="M137.03997802734375 317.0080108642578C 153.44780421895462 504.5503224089872 265.40781093282186 586.6016780182589 248.99998474121094 774.1439895629883" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1108)"></path></g><g id="SvgjsG1110"><path id="SvgjsPath1111" d="M137.03997802734375 573.2400054931641C 145.05811339639388 664.8877121320274 257.01812011026107 682.4962829241249 248.99998474121094 774.1439895629883" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1112)"></path></g><g id="SvgjsG1114"><path id="SvgjsPath1115" d="M381.03997802734375 456.24000549316406C 393.0407891056441 593.409903793695 261.0007958195113 636.9740912624574 248.99998474121094 774.1439895629883" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1116)"></path></g><g id="SvgjsG1118"><path id="SvgjsPath1119" d="M381.03997802734375 730.2400054931641C 385.89099114909015 785.6873391961069 253.85099786295734 718.6966558600454 248.99998474121094 774.1439895629883" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1120)"></path></g><g id="SvgjsG1122" transform="translate(543.471981048584,627.2400054931641)"><path id="SvgjsPath1123" d="M 0 0L 255 0L 255 33L 0 33Z" stroke="none" fill="none"></path><g id="SvgjsG1124"><text id="SvgjsText1125" font-family="微软雅黑" text-anchor="start" font-size="13px" width="255px" fill="#323232" font-weight="400" align="top" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-1.95" transform="rotate(0)"><tspan id="SvgjsTspan1126" dy="16" x="0"><tspan id="SvgjsTspan1127" style="text-decoration:;">choose an operation from AvgPool, </tspan></tspan><tspan id="SvgjsTspan1128" dy="16" x="0"><tspan id="SvgjsTspan1129" style="text-decoration:;">MaxPool, 3x3 SepConv, 5x5SepConv, </tspan></tspan><tspan id="SvgjsTspan1130" dy="16" x="0"><tspan id="SvgjsTspan1131" style="text-decoration:;">3x3DilConv, 5x5DilConv, SkipConnect</tspan></tspan></text></g></g><g id="SvgjsG1132"><path id="SvgjsPath1133" d="M473.471981048584 655.6359786987305L523.471981048584 655.6359786987305" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1134)"></path></g><g id="SvgjsG1136" transform="translate(484.471981048584,682.1359786987305)"><path id="SvgjsPath1137" d="M 0 13.104026794433594C 0 -4.368008931477864 29 -4.368008931477864 29 13.104026794433594C 29 30.57606252034505 0 30.57606252034505 0 13.104026794433594Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc00"></path><g id="SvgjsG1138"><text id="SvgjsText1139" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="9px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="3.6540267944335936" transform="rotate(0)"></text></g></g><g id="SvgjsG1140" transform="translate(484.471981048584,720.2400054931641)"><path id="SvgjsPath1141" d="M 0 13.104026794433594C 0 -4.368008931477864 29 -4.368008931477864 29 13.104026794433594C 29 30.57606252034505 0 30.57606252034505 0 13.104026794433594Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1142"><text id="SvgjsText1143" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="9px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="3.6540267944335936" transform="rotate(0)"></text></g></g><g id="SvgjsG1144" transform="translate(484.471981048584,760.2400054931641)"><path id="SvgjsPath1145" d="M 0 13.104026794433594C 0 -4.368008931477864 29 -4.368008931477864 29 13.104026794433594C 29 30.57606252034505 0 30.57606252034505 0 13.104026794433594Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#cc99ff"></path><g id="SvgjsG1146"><text id="SvgjsText1147" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="9px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="3.6540267944335936" transform="rotate(0)"></text></g></g><g id="SvgjsG1148" transform="translate(544.471981048584,684.4480590820312)"><path id="SvgjsPath1149" d="M 0 0L 232 0L 232 35.79194641113281L 0 35.79194641113281Z" stroke="none" fill="none"></path><g id="SvgjsG1150"><text id="SvgjsText1151" font-family="微软雅黑" text-anchor="start" font-size="13px" width="232px" fill="#323232" font-weight="400" align="top" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-1.95" transform="rotate(0)"><tspan id="SvgjsTspan1152" dy="16" x="0"><tspan id="SvgjsTspan1153" style="text-decoration:;">results of previous layers as input</tspan></tspan></text></g></g><g id="SvgjsG1154" transform="translate(544.471981048584,720.2400054931641)"><path id="SvgjsPath1155" d="M 0 0L 251 0L 251 37.79194641113281L 0 37.79194641113281Z" stroke="none" fill="none"></path><g id="SvgjsG1156"><text id="SvgjsText1157" font-family="微软雅黑" text-anchor="start" font-size="13px" width="251px" fill="#323232" font-weight="400" align="top" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-1.95" transform="rotate(0)"><tspan id="SvgjsTspan1158" dy="16" x="0"><tspan id="SvgjsTspan1159" style="text-decoration:;">accept two operations from all inputs, </tspan></tspan><tspan id="SvgjsTspan1160" dy="16" x="0"><tspan id="SvgjsTspan1161" style="text-decoration:;">then add the results as output</tspan></tspan></text></g></g><g id="SvgjsG1162" transform="translate(545.471981048584,763.2400054931641)"><path id="SvgjsPath1163" d="M 0 0L 251 0L 251 37.79194641113281L 0 37.79194641113281Z" stroke="none" fill="none"></path><g id="SvgjsG1164"><text id="SvgjsText1165" font-family="微软雅黑" text-anchor="start" font-size="13px" width="251px" fill="#323232" font-weight="400" align="top" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-1.95" transform="rotate(0)"><tspan id="SvgjsTspan1166" dy="16" x="0"><tspan id="SvgjsTspan1167" style="text-decoration:;">concat all inputs in channels</tspan></tspan></text></g></g><g id="SvgjsG1168"><path id="SvgjsPath1169" d="M472.471981048584 599.6359786987305L522.471981048584 599.6359786987305" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1170)"></path></g><g id="SvgjsG1172" transform="translate(545.471981048584,586.1359786987305)"><path id="SvgjsPath1173" d="M 0 0L 255 0L 255 33L 0 33Z" stroke="none" fill="none"></path><g id="SvgjsG1174"><text id="SvgjsText1175" font-family="微软雅黑" text-anchor="start" font-size="13px" width="255px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-0.95" transform="rotate(0)"><tspan id="SvgjsTspan1176" dy="16" x="0"><tspan id="SvgjsTspan1177" style="text-decoration:;">pass the result directly to the output </tspan></tspan><tspan id="SvgjsTspan1178" dy="16" x="0"><tspan id="SvgjsTspan1179" style="text-decoration:;">node</tspan></tspan></text></g></g></svg>
\ No newline at end of file

docs/img/NAS_ENAS_macro.svg

+<svg id="SvgjsSvg1006" width="837.0800476074219" height="917.6933898925781" xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:svgjs="http://svgjs.com/svgjs"><defs id="SvgjsDefs1007"><marker id="SvgjsMarker1022" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1023" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1036" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1037" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1040" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1041" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1044" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1045" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1053" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1054" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1089" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1090" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1141" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1142" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1145" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1146" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1149" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1150" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1153" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1154" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1157" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1158" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1161" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1162" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1165" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1166" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1169" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1170" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1173" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1174" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1177" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1178" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1181" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1182" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1191" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1192" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1205" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1206" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1225" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1226" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1237" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1238" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1241" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1242" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker></defs><g id="SvgjsG1008" transform="translate(74.002685546875,148.85643214220443)"><path id="SvgjsPath1009" d="M 0 133.86033519553072C 0 -44.62011173184357 267.72067039106145 -44.62011173184357 267.72067039106145 133.86033519553072C 267.72067039106145 312.340782122905 0 312.340782122905 0 133.86033519553072Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffffff"></path><g id="SvgjsG1010"><text id="SvgjsText1011" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="248px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="124.41033519553072" transform="rotate(0)"></text></g></g><g id="SvgjsG1012" transform="translate(131.37140063067386,187.10224219807034)"><path id="SvgjsPath1013" d="M 0 31.871508379888265C 0 -10.623836126629422 63.74301675977653 -10.623836126629422 63.74301675977653 31.871508379888265C 63.74301675977653 74.36685288640595 0 74.36685288640595 0 31.871508379888265Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1014"><text id="SvgjsText1015" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="44px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="22.421508379888266" transform="rotate(0)"></text></g></g><g id="SvgjsG1016" transform="translate(209.68424979268502,311.8564321422044)"><path id="SvgjsPath1017" d="M 0 31.871508379888265C 0 -10.623836126629422 63.74301675977653 -10.623836126629422 63.74301675977653 31.871508379888265C 63.74301675977653 74.36685288640595 0 74.36685288640595 0 31.871508379888265Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#cc99ff"></path><g id="SvgjsG1018"><text id="SvgjsText1019" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="44px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="22.421508379888266" transform="rotate(0)"></text></g></g><g id="SvgjsG1020"><path id="SvgjsPath1021" d="M109.51665202732192 96.04078968410386C 131.00715482061798 132.46537068969045 142.81869506835938 152.7813491821289 163.24290901056213 187.10224219807034" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1022)"></path></g><g id="SvgjsG1024" transform="translate(301.6563168317912,25.923471248349685)"><path id="SvgjsPath1025" d="M 0 35.05865921787709C 0 -11.686219739292364 70.11731843575419 -11.686219739292364 70.11731843575419 35.05865921787709C 70.11731843575419 81.80353817504655 0 81.80353817504655 0 35.05865921787709Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc66"></path><g id="SvgjsG1026"><text id="SvgjsText1027" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="51px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="10.108659217877094" transform="rotate(0)"><tspan id="SvgjsTspan1028" dy="16" x="35.5"><tspan id="SvgjsTspan1029" style="text-decoration:;">Layer </tspan></tspan><tspan id="SvgjsTspan1030" dy="16" x="35.5"><tspan id="SvgjsTspan1031" style="text-decoration:;">N-1's</tspan></tspan><tspan id="SvgjsTspan1032" dy="16" x="35.5"><tspan id="SvgjsTspan1033" style="text-decoration:;">output</tspan></tspan></text></g></g><g id="SvgjsG1034"><path id="SvgjsPath1035" d="M336.7149760496683 96.04078968410386C 344.9377251996825 190.02724254125366 249.77850732258764 217.86997928505463 241.5557581725733 311.8564321422044" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1036)"></path></g><g id="SvgjsG1038"><path id="SvgjsPath1039" d="M132.28201515581355 94.21956063382453C 144.66637269771297 131.37263325952284 152.81869506835938 149.7813491821289 163.24290901056213 187.10224219807034" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1040)"></path></g><g id="SvgjsG1042"><path id="SvgjsPath1043" d="M163.24290901056213 93.30894610868489C 163.24290901056213 130.82626454443908 166.51275876473073 149.7276884857523 163.24290901056213 187.10224219807034" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1044)"></path></g><g id="SvgjsG1046" transform="matrix(6.123233995736766e-17,1,-1,6.123233995736766e-17,175.99151428052167,25.012857064188523)"><path id="SvgjsPath1047" d="M 61.9217877094972 0Q 55.729608938547486 0 55.729608938547486 6.73854748603352L 55.729608938547486 26.95418994413408Q 55.729608938547486 33.6927374301676 39.9217877094972 33.6927374301676Q 55.729608938547486 33.6927374301676 55.729608938547486 40.431284916201115L 55.729608938547486 60.646927374301676Q 55.729608938547486 67.3854748603352 61.9217877094972 67.3854748603352" stroke="rgba(50,50,50,1)" stroke-width="2" fill="none"></path><path id="SvgjsPath1048" d="M 0 0L 61.9217877094972 0L 61.9217877094972 67.3854748603352L 0 67.3854748603352Z" stroke="none" fill="none"></path><g id="SvgjsG1049"><text id="SvgjsText1050" font-family="微软雅黑" text-anchor="end" font-size="13px" width="35px" fill="#323232" font-weight="400" align="middle" anchor="end" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="24.242737430167598" transform="rotate(0)"></text></g></g><g id="SvgjsG1051"><path id="SvgjsPath1052" d="M427.321121301065 258.1301751589642L472.8518475580481 258.1301751589642" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1053)"></path></g><g id="SvgjsG1055" transform="translate(489.24290901056213,239.00727013103125)"><path id="SvgjsPath1056" d="M 0 0L 236.75977653631284 0L 236.75977653631284 46.44134078212291L 0 46.44134078212291Z" stroke="none" fill="none"></path><g id="SvgjsG1057"><text id="SvgjsText1058" font-family="微软雅黑" text-anchor="start" font-size="13px" width="237px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-1.7293296089385464" transform="rotate(0)"><tspan id="SvgjsTspan1059" dy="16" x="0"><tspan id="SvgjsTspan1060" style="text-decoration:;">choose an operation from: MaxPool, </tspan></tspan><tspan id="SvgjsTspan1061" dy="16" x="0"><tspan id="SvgjsTspan1062" style="text-decoration:;">AvgPool, SepConvBN3x3, SepConvBN</tspan></tspan><tspan id="SvgjsTspan1063" dy="16" x="0"><tspan id="SvgjsTspan1064" style="text-decoration:;">5x5 and SkipConnect</tspan></tspan></text></g></g><g id="SvgjsG1065" transform="translate(424.58927772564596,293.6441416394111)"><path id="SvgjsPath1066" d="M 0 25.952513966480446C 0 -8.650837988826815 51.90502793296089 -8.650837988826815 51.90502793296089 25.952513966480446C 51.90502793296089 60.555865921787706 0 60.555865921787706 0 25.952513966480446Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1067"><text id="SvgjsText1068" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="32px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.502513966480446" transform="rotate(0)"></text></g></g><g id="SvgjsG1069" transform="translate(489.24290901056213,301.3843651030983)"><path id="SvgjsPath1070" d="M 0 0L 236.75977653631284 0L 236.75977653631284 36.42458100558659L 0 36.42458100558659Z" stroke="none" fill="none"></path><g id="SvgjsG1071"><text id="SvgjsText1072" font-family="微软雅黑" text-anchor="start" font-size="13px" width="237px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="8.762290502793295" transform="rotate(0)"><tspan id="SvgjsTspan1073" dy="16" x="0"><tspan id="SvgjsTspan1074" style="text-decoration:;">accept none/one/multiple input(s)</tspan></tspan></text></g></g><g id="SvgjsG1075" transform="translate(425.9551995133554,367.40391817572396)"><path id="SvgjsPath1076" d="M 0 24.58659217877095C 0 -8.195530726256983 49.1731843575419 -8.195530726256983 49.1731843575419 24.58659217877095C 49.1731843575419 57.36871508379888 0 57.36871508379888 0 24.58659217877095Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#cc99ff"></path><g id="SvgjsG1077"><text id="SvgjsText1078" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="30px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="15.13659217877095" transform="rotate(0)"></text></g></g><g id="SvgjsG1079" transform="translate(489.24290901056213,373.7782198517016)"><path id="SvgjsPath1080" d="M 0 0L 236.75977653631284 0L 236.75977653631284 36.42458100558659L 0 36.42458100558659Z" stroke="none" fill="none"></path><g id="SvgjsG1081"><text id="SvgjsText1082" font-family="微软雅黑" text-anchor="start" font-size="13px" width="237px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="0.7622905027932945" transform="rotate(0)"><tspan id="SvgjsTspan1083" dy="16" x="0"><tspan id="SvgjsTspan1084" style="text-decoration:;">add all input as the output of this </tspan></tspan><tspan id="SvgjsTspan1085" dy="16" x="0"><tspan id="SvgjsTspan1086" style="text-decoration:;">layer</tspan></tspan></text></g></g><g id="SvgjsG1087"><path id="SvgjsPath1088" d="M426.41050677592534 208.04637627628261L471.9412330329085 208.04637627628261" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1089)"></path></g><g id="SvgjsG1091" transform="translate(489.24290901056213,184.37039862265135)"><path id="SvgjsPath1092" d="M 0 0L 205.79888268156424 0L 205.79888268156424 43.70949720670391L 0 43.70949720670391Z" stroke="none" fill="none"></path><g id="SvgjsG1093"><text id="SvgjsText1094" font-family="微软雅黑" text-anchor="start" font-size="13px" width="206px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="4.404748603351954" transform="rotate(0)"><tspan id="SvgjsTspan1095" dy="16" x="0"><tspan id="SvgjsTspan1096" style="text-decoration:;">(suppose this layer is the Nth </tspan></tspan><tspan id="SvgjsTspan1097" dy="16" x="0"><tspan id="SvgjsTspan1098" style="text-decoration:;">layer) outputs of all N-2 layers</tspan></tspan></text></g></g><g id="SvgjsG1099" transform="translate(100.41050677592528,33.208387449467)"><path id="SvgjsPath1100" d="M 0 0L 109.27374301675977 0L 109.27374301675977 36.42458100558659L 0 36.42458100558659Z" stroke="none" fill="none"></path><g id="SvgjsG1101"><text id="SvgjsText1102" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="110px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="8.762290502793295" transform="rotate(0)"><tspan id="SvgjsTspan1103" dy="16" x="55"><tspan id="SvgjsTspan1104" style="text-decoration:;">x (N-2)</tspan></tspan></text></g></g><g id="SvgjsG1105" transform="translate(150.49430565860683,439.196758792387)"><path id="SvgjsPath1106" d="M 0 0L 109.27374301675977 0L 109.27374301675977 36.42458100558659L 0 36.42458100558659Z" stroke="none" fill="none"></path><g id="SvgjsG1107"><text id="SvgjsText1108" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="110px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="8.762290502793295" transform="rotate(0)"><tspan id="SvgjsTspan1109" dy="16" x="55"><tspan id="SvgjsTspan1110" style="text-decoration:;">ENASMacroLayer</tspan></tspan></text></g></g><g id="SvgjsG1111" transform="translate(25.002685546875,608.5613342285156)"><path id="SvgjsPath1112" d="M 0 0L 67 0L 67 46.9L 0 46.9Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc66"></path><g id="SvgjsG1113"><text id="SvgjsText1114" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="47px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="14" transform="rotate(0)"><tspan id="SvgjsTspan1115" dy="16" x="33.5"><tspan id="SvgjsTspan1116" style="text-decoration:;">input</tspan></tspan></text></g></g><g id="SvgjsG1117" transform="translate(145.49430565860683,601.7863342285157)"><path id="SvgjsPath1118" d="M 0 29.224999999999966C 0 -9.741666666666655 59.84357541899442 -9.741666666666655 59.84357541899442 29.224999999999966C 59.84357541899442 68.19166666666659 0 68.19166666666659 0 29.224999999999966Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00ffff"></path><g id="SvgjsG1119"><text id="SvgjsText1120" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="40px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="19.774999999999967" transform="rotate(0)"></text></g></g><g id="SvgjsG1121" transform="translate(264.49430565860683,602.7863342285157)"><path id="SvgjsPath1122" d="M 0 29.224999999999966C 0 -9.741666666666655 59.84357541899442 -9.741666666666655 59.84357541899442 29.224999999999966C 59.84357541899442 68.19166666666659 0 68.19166666666659 0 29.224999999999966Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00ffff"></path><g id="SvgjsG1123"><text id="SvgjsText1124" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="40px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="19.774999999999967" transform="rotate(0)"></text></g></g><g id="SvgjsG1125" transform="translate(384.87978051894197,602.7863342285157)"><path id="SvgjsPath1126" d="M 0 29.224999999999966C 0 -9.741666666666655 59.84357541899442 -9.741666666666655 59.84357541899442 29.224999999999966C 59.84357541899442 68.19166666666659 0 68.19166666666659 0 29.224999999999966Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00ffff"></path><g id="SvgjsG1127"><text id="SvgjsText1128" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="40px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="19.774999999999967" transform="rotate(0)"></text></g></g><g id="SvgjsG1129" transform="translate(504.87978051894197,603.7863342285157)"><path id="SvgjsPath1130" d="M 0 29.224999999999966C 0 -9.741666666666655 59.84357541899442 -9.741666666666655 59.84357541899442 29.224999999999966C 59.84357541899442 68.19166666666659 0 68.19166666666659 0 29.224999999999966Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00ffff"></path><g id="SvgjsG1131"><text id="SvgjsText1132" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="40px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="19.774999999999967" transform="rotate(0)"></text></g></g><g id="SvgjsG1133" transform="translate(621.002685546875,610.5613342285156)"><path id="SvgjsPath1134" d="M 0 0L 83 0L 83 44.899999999999864L 0 44.899999999999864Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#cc99ff"></path><g id="SvgjsG1135"><text id="SvgjsText1136" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="63px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="12.999999999999932" transform="rotate(0)"><tspan id="SvgjsTspan1137" dy="16" x="41.5"><tspan id="SvgjsTspan1138" style="text-decoration:;">softmax</tspan></tspan></text></g></g><g id="SvgjsG1139"><path id="SvgjsPath1140" d="M92.002685546875 632.0113342285157L145.49430565860683 631.0113342285156" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1141)"></path></g><g id="SvgjsG1143"><path id="SvgjsPath1144" d="M205.33788107760125 631.0113342285156L264.49430565860683 632.0113342285156" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1145)"></path></g><g id="SvgjsG1147"><path id="SvgjsPath1148" d="M324.3378810776012 632.0113342285156L384.87978051894197 632.0113342285156" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1149)"></path></g><g id="SvgjsG1151"><path id="SvgjsPath1152" d="M444.7233559379365 632.0113342285156L504.87978051894197 633.0113342285156" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1153)"></path></g><g id="SvgjsG1155"><path id="SvgjsPath1156" d="M564.7233559379364 633.0113342285156L621.002685546875 633.0113342285156" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1157)"></path></g><g id="SvgjsG1159"><path id="SvgjsPath1160" d="M205.33788107760125 631.0113342285156C 232.002685546875 616.6213397979736 235.002685546875 606.6213397979736 264.49430565860683 632.0113342285156" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1161)"></path></g><g id="SvgjsG1163"><path id="SvgjsPath1164" d="M205.33788107760125 631.0113342285156C 272.002685546875 564.6213397979736 290.002685546875 572.6213397979736 384.87978051894197 632.0113342285156" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1165)"></path></g><g id="SvgjsG1167"><path id="SvgjsPath1168" d="M205.33788107760125 631.0113342285156C 285.002685546875 531.6213397979736 416.002685546875 545.6213397979736 504.87978051894197 633.0113342285156" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1169)"></path></g><g id="SvgjsG1171"><path id="SvgjsPath1172" d="M324.3378810776012 632.0113342285156C 353.002685546875 653.6213397979736 361.002685546875 652.6213397979736 384.87978051894197 632.0113342285156" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1173)"></path></g><g id="SvgjsG1175"><path id="SvgjsPath1176" d="M324.3378810776012 632.0113342285156C 393.002685546875 696.6213397979736 435.002685546875 690.6213397979736 504.87978051894197 633.0113342285156" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1177)"></path></g><g id="SvgjsG1179"><path id="SvgjsPath1180" d="M444.7233559379365 632.0113342285156C 465.002685546875 613.6213397979736 475.002685546875 615.6213397979736 504.87978051894197 633.0113342285156" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1181)"></path></g><g id="SvgjsG1183" transform="translate(254.42726655246156,734.6213397979736)"><path id="SvgjsPath1184" d="M 0 0L 177.28770949720672 0L 177.28770949720672 43L 0 43Z" stroke="none" fill="none"></path><g id="SvgjsG1185"><text id="SvgjsText1186" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="178px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="12.05" transform="rotate(0)"><tspan id="SvgjsTspan1187" dy="16" x="89"><tspan id="SvgjsTspan1188" style="text-decoration:;">GeneralModel</tspan></tspan></text></g></g><g id="SvgjsG1189"><path id="SvgjsPath1190" d="M513.16190342397 711.1301751589642L558.6926296809531 711.1301751589642" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1191)"></path></g><g id="SvgjsG1193" transform="translate(575.0836911334671,692.0072701310312)"><path id="SvgjsPath1194" d="M 0 0L 236.75977653631284 0L 236.75977653631284 46.44134078212291L 0 46.44134078212291Z" stroke="none" fill="none"></path><g id="SvgjsG1195"><text id="SvgjsText1196" font-family="微软雅黑" text-anchor="start" font-size="13px" width="237px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-1.7293296089385464" transform="rotate(0)"><tspan id="SvgjsTspan1197" dy="16" x="0"><tspan id="SvgjsTspan1198" style="text-decoration:;">choose an operation from: MaxPool, </tspan></tspan><tspan id="SvgjsTspan1199" dy="16" x="0"><tspan id="SvgjsTspan1200" style="text-decoration:;">AvgPool, SepConvBN3x3, SepConvBN</tspan></tspan><tspan id="SvgjsTspan1201" dy="16" x="0"><tspan id="SvgjsTspan1202" style="text-decoration:;">5x5 and SkipConnect</tspan></tspan></text></g></g><g id="SvgjsG1203"><path id="SvgjsPath1204" d="M512.002685546875 764.0113342285156L558.002685546875 764.0113342285156" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1205)"></path></g><g id="SvgjsG1207" transform="translate(575.0836911334671,746.4613342285156)"><path id="SvgjsPath1208" d="M 0 0L 206.91899441340786 0L 206.91899441340786 30L 0 30Z" stroke="none" fill="none"></path><g id="SvgjsG1209"><text id="SvgjsText1210" font-family="微软雅黑" text-anchor="start" font-size="13px" width="207px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="5.55" transform="rotate(0)"><tspan id="SvgjsTspan1211" dy="16" x="0"><tspan id="SvgjsTspan1212" style="text-decoration:;">pass tensor without operation</tspan></tspan></text></g></g><g id="SvgjsG1213" transform="translate(515.2401157144727,840.7024446654088)"><path id="SvgjsPath1214" d="M 0 25.766944781553388C 0 -8.588981593851129 52.76256983240228 -8.588981593851129 52.76256983240228 25.766944781553388C 52.76256983240228 60.122871156957906 0 60.122871156957906 0 25.766944781553388Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00ffff"></path><g id="SvgjsG1215"><text id="SvgjsText1216" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="33px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.31694478155339" transform="rotate(0)"></text></g></g><g id="SvgjsG1217" transform="translate(586.4943056586069,848.257098944169)"><path id="SvgjsPath1218" d="M 0 0L 109.27374301675977 0L 109.27374301675977 36.42458100558659L 0 36.42458100558659Z" stroke="none" fill="none"></path><g id="SvgjsG1219"><text id="SvgjsText1220" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="110px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="8.762290502793295" transform="rotate(0)"><tspan id="SvgjsTspan1221" dy="16" x="55"><tspan id="SvgjsTspan1222" style="text-decoration:;">ENASMacroLayer</tspan></tspan></text></g></g><g id="SvgjsG1223"><path id="SvgjsPath1224" d="M515.2401157144727 810.0463762762827L560.7708419714559 810.0463762762827" stroke-dasharray="10,5,3,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1225)"></path></g><g id="SvgjsG1227" transform="translate(578.0725179491095,786.3703986226515)"><path id="SvgjsPath1228" d="M 0 0L 205.79888268156424 0L 205.79888268156424 43.70949720670391L 0 43.70949720670391Z" stroke="none" fill="none"></path><g id="SvgjsG1229"><text id="SvgjsText1230" font-family="微软雅黑" text-anchor="start" font-size="13px" width="206px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="4.404748603351954" transform="rotate(0)"><tspan id="SvgjsTspan1231" dy="16" x="0"><tspan id="SvgjsTspan1232" style="text-decoration:;">(suppose this layer is the Nth </tspan></tspan><tspan id="SvgjsTspan1233" dy="16" x="0"><tspan id="SvgjsTspan1234" style="text-decoration:;">layer) outputs of all N-1 layers</tspan></tspan></text></g></g><g id="SvgjsG1235"><path id="SvgjsPath1236" d="M163.24290901056213 250.84525895784685C 166.7038178979269 290.40362855551774 245.01666705993807 272.29806254453354 241.5557581725733 311.8564321422044" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1237)"></path></g><g id="SvgjsG1239"><path id="SvgjsPath1240" d="M424.41050677592534 163.04637627628261L469.9412330329085 163.04637627628261" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1241)"></path></g><g id="SvgjsG1243" transform="translate(489.24290901056213,136.37039862265135)"><path id="SvgjsPath1244" d="M 0 0L 205.79888268156424 0L 205.79888268156424 43.70949720670391L 0 43.70949720670391Z" stroke="none" fill="none"></path><g id="SvgjsG1245"><text id="SvgjsText1246" font-family="微软雅黑" text-anchor="start" font-size="13px" width="206px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="4.404748603351954" transform="rotate(0)"><tspan id="SvgjsTspan1247" dy="16" x="0"><tspan id="SvgjsTspan1248" style="text-decoration:;">pass the tensor input directly to </tspan></tspan><tspan id="SvgjsTspan1249" dy="16" x="0"><tspan id="SvgjsTspan1250" style="text-decoration:;">the next node</tspan></tspan></text></g></g></svg>
\ No newline at end of file

docs/img/NAS_ENAS_micro.svg

+<svg id="SvgjsSvg1006" width="704.9999694824219" height="824.0800323486328" xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:svgjs="http://svgjs.com/svgjs"><defs id="SvgjsDefs1007"><marker id="SvgjsMarker1046" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1047" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1050" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1051" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1054" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1055" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1058" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1059" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1062" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1063" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1066" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1067" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1070" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1071" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1074" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1075" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1078" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1079" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1082" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1083" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1086" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1087" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1090" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1091" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1094" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1095" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1098" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1099" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1134" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1135" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1144" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1145" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1148" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1149" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1152" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1153" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1156" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1157" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker><marker id="SvgjsMarker1160" markerWidth="16" markerHeight="12" refX="16" refY="6" viewBox="0 0 16 12" orient="auto" markerUnits="userSpaceOnUse" stroke-dasharray="0,0"><path id="SvgjsPath1161" d="M0,2 L14,6 L0,11 L0,2" fill="#323232" stroke="#323232" stroke-width="2"></path></marker></defs><g id="SvgjsG1008" transform="translate(67.99998474121094,25.008005142211914)"><path id="SvgjsPath1009" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc66"></path><g id="SvgjsG1010"><text id="SvgjsText1011" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1012" dy="16" x="35"><tspan id="SvgjsTspan1013" style="text-decoration:;">input_0</tspan></tspan></text></g></g><g id="SvgjsG1014" transform="translate(238.99998474121094,111.00800514221191)"><path id="SvgjsPath1015" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc66"></path><g id="SvgjsG1016"><text id="SvgjsText1017" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1018" dy="16" x="35"><tspan id="SvgjsTspan1019" style="text-decoration:;">input_1</tspan></tspan></text></g></g><g id="SvgjsG1020" transform="translate(238.99998474121094,279.0080051422119)"><path id="SvgjsPath1021" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1022"><text id="SvgjsText1023" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1024" dy="16" x="35"><tspan id="SvgjsTspan1025" style="text-decoration:;">node 0</tspan></tspan></text></g></g><g id="SvgjsG1026" transform="translate(238.99998474121094,492.0080051422119)"><path id="SvgjsPath1027" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1028"><text id="SvgjsText1029" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1030" dy="16" x="35"><tspan id="SvgjsTspan1031" style="text-decoration:;">node 2</tspan></tspan></text></g></g><g id="SvgjsG1032" transform="translate(67.99998474121094,372.0080051422119)"><path id="SvgjsPath1033" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1034"><text id="SvgjsText1035" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1036" dy="16" x="35"><tspan id="SvgjsTspan1037" style="text-decoration:;">node 1</tspan></tspan></text></g></g><g id="SvgjsG1038" transform="translate(67.99998474121094,594.0080051422119)"><path id="SvgjsPath1039" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1040"><text id="SvgjsText1041" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1042" dy="16" x="35"><tspan id="SvgjsTspan1043" style="text-decoration:;">node 3</tspan></tspan></text></g></g><g id="SvgjsG1044"><path id="SvgjsPath1045" d="M102.99998474121094 95.00800514221191C 111.75708566874948 195.1021267645375 282.7570856687495 178.91388351988633 273.99998474121094 279.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1046)"></path></g><g id="SvgjsG1048"><path id="SvgjsPath1049" d="M273.99998474121094 181.0080051422119C 277.41648985691916 220.05883730740834 277.41648985691916 239.9571729770155 273.99998474121094 279.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1050)"></path></g><g id="SvgjsG1052"><path id="SvgjsPath1053" d="M273.99998474121094 349.0080051422119C 278.98529322637694 405.9903418730597 278.98529322637705 435.0256684113641 273.99998474121094 492.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1054)"></path></g><g id="SvgjsG1056"><path id="SvgjsPath1057" d="M102.99998474121094 95.00800514221191C 112.65684103765147 205.38637769077735 112.65684103765147 261.6296325936465 102.99998474121094 372.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1058)"></path></g><g id="SvgjsG1060"><path id="SvgjsPath1061" d="M273.99998474121094 349.0080051422119C 280.01512026970283 417.7613188410721 109.01512026970283 303.2546914433517 102.99998474121094 372.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1062)"></path></g><g id="SvgjsG1064"><path id="SvgjsPath1065" d="M102.99998474121094 442.0080051422119C 108.29905390026858 502.57664278619006 108.29905390026858 533.4393674982338 102.99998474121094 594.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1066)"></path></g><g id="SvgjsG1068"><path id="SvgjsPath1069" d="M273.99998474121094 349.0080051422119C 284.41593834944956 468.0628996675149 113.41593834944956 474.95311061690893 102.99998474121094 594.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1070)"></path></g><g id="SvgjsG1072"><path id="SvgjsPath1073" d="M273.99998474121094 181.0080051422119C 212.99998474121094 264.0080051422119 111.93738864405282 269.8530110818261 102.99998474121094 372.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1074)"></path></g><g id="SvgjsG1076"><path id="SvgjsPath1077" d="M102.99998474121094 442.0080051422119C 109.2110530012738 513.0008402107513 280.21105300127385 421.0151700736725 273.99998474121094 492.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1078)"></path></g><g id="SvgjsG1080"><path id="SvgjsPath1081" d="M273.99998474121094 562.0080051422119C 280.06492236717565 631.3305594199732 109.0649223671756 524.6854508644507 102.99998474121094 594.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1082)"></path></g><g id="SvgjsG1084"><path id="SvgjsPath1085" d="M102.99998474121094 95.00800514221191C 24.999984741210938 329.0080051422119 30.999984741210938 425.0080051422119 102.99998474121094 594.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1086)"></path></g><g id="SvgjsG1088"><path id="SvgjsPath1089" d="M273.99998474121094 181.0080051422119C 222.99998474121094 348.0080051422119 118.5834705403015 415.88794739926834 102.99998474121094 594.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1090)"></path></g><g id="SvgjsG1092"><path id="SvgjsPath1093" d="M273.99998474121094 181.0080051422119C 319.99998474121094 315.0080051422119 358.99998474121094 344.0080051422119 273.99998474121094 492.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1094)"></path></g><g id="SvgjsG1096"><path id="SvgjsPath1097" d="M371.99998474121094 529.0080051422119L417.99998474121094 529.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1098)"></path></g><g id="SvgjsG1100" transform="translate(435.99998474121094,504.0080051422119)"><path id="SvgjsPath1101" d="M 0 0L 238 0L 238 46L 0 46Z" stroke="none" fill="none"></path><g id="SvgjsG1102"><text id="SvgjsText1103" font-family="微软雅黑" text-anchor="start" font-size="13px" width="238px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-1.95" transform="rotate(0)"><tspan id="SvgjsTspan1104" dy="16" x="0"><tspan id="SvgjsTspan1105" style="text-decoration:;">choose one operation from MaxPool, </tspan></tspan><tspan id="SvgjsTspan1106" dy="16" x="0"><tspan id="SvgjsTspan1107" style="text-decoration:;">AvgPool, SepConvBN3x3, SepConvBN</tspan></tspan><tspan id="SvgjsTspan1108" dy="16" x="0"><tspan id="SvgjsTspan1109" style="text-decoration:;">5x5, SkipConnect</tspan></tspan></text></g></g><g id="SvgjsG1110" transform="translate(371.99998474121094,562.0080051422119)"><path id="SvgjsPath1111" d="M 0 25.5C 0 -8.5 51 -8.5 51 25.5C 51 59.5 0 59.5 0 25.5Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#ffff00"></path><g id="SvgjsG1112"><text id="SvgjsText1113" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="31px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.05" transform="rotate(0)"></text></g></g><g id="SvgjsG1114" transform="translate(371.99998474121094,637.0080051422119)"><path id="SvgjsPath1115" d="M 0 25.5C 0 -8.5 51 -8.5 51 25.5C 51 59.5 0 59.5 0 25.5Z" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#00cc66"></path><g id="SvgjsG1116"><text id="SvgjsText1117" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="31px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.05" transform="rotate(0)"></text></g></g><g id="SvgjsG1118" transform="translate(435.99998474121094,566.5080051422119)"><path id="SvgjsPath1119" d="M 0 0L 244 0L 244 42L 0 42Z" stroke="none" fill="none"></path><g id="SvgjsG1120"><text id="SvgjsText1121" font-family="微软雅黑" text-anchor="start" font-size="13px" width="244px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="3.55" transform="rotate(0)"><tspan id="SvgjsTspan1122" dy="16" x="0"><tspan id="SvgjsTspan1123" style="text-decoration:;">accept two operations from all inputs, </tspan></tspan><tspan id="SvgjsTspan1124" dy="16" x="0"><tspan id="SvgjsTspan1125" style="text-decoration:;">then add the results as output</tspan></tspan></text></g></g><g id="SvgjsG1126" transform="translate(435.99998474121094,642.5080051422119)"><path id="SvgjsPath1127" d="M 0 0L 224 0L 224 40L 0 40Z" stroke="none" fill="none"></path><g id="SvgjsG1128"><text id="SvgjsText1129" font-family="微软雅黑" text-anchor="start" font-size="13px" width="224px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="10.55" transform="rotate(0)"><tspan id="SvgjsTspan1130" dy="16" x="0"><tspan id="SvgjsTspan1131" style="text-decoration:;">results of previous layers as input</tspan></tspan></text></g></g><g id="SvgjsG1132"><path id="SvgjsPath1133" d="M102.99998474121094 95.00800514221191C 118.06961340198137 267.2546489180095 289.0696134019814 319.76136136641435 273.99998474121094 492.0080051422119" stroke-dasharray="8,5" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1134)"></path></g><g id="SvgjsG1136" transform="translate(150.99998474121094,729.0880031585693)"><path id="SvgjsPath1137" d="M 0 35C 0 -11.666666666666666 70 -11.666666666666666 70 35C 70 81.66666666666667 0 81.66666666666667 0 35Z" stroke-dasharray="3,4" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#cc99ff"></path><g id="SvgjsG1138"><text id="SvgjsText1139" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="50px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="25.55" transform="rotate(0)"><tspan id="SvgjsTspan1140" dy="16" x="35"><tspan id="SvgjsTspan1141" style="text-decoration:;">output</tspan></tspan></text></g></g><g id="SvgjsG1142"><path id="SvgjsPath1143" d="M102.99998474121094 442.0080051422119C 113.41815075849257 561.0881876185914 196.41815075849257 610.0078206821898 185.99998474121094 729.0880031585693" stroke-dasharray="3,3" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1144)"></path></g><g id="SvgjsG1146"><path id="SvgjsPath1147" d="M102.99998474121094 664.0080051422119C 106.67699141184806 706.0363837051965 189.67699141184806 687.0596245955848 185.99998474121094 729.0880031585693" stroke-dasharray="3,3" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1148)"></path></g><g id="SvgjsG1150"><path id="SvgjsPath1151" d="M273.99998474121094 562.0080051422119C 280.58330524633993 637.2557028416771 192.58330524633993 653.8403054591041 185.99998474121094 729.0880031585693" stroke-dasharray="3,3" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1152)"></path></g><g id="SvgjsG1154"><path id="SvgjsPath1155" d="M273.99998474121094 349.0080051422119C 233.99998474121094 500.92799377441406 199.60096408433088 573.6280978979321 185.99998474121094 729.0880031585693" stroke-dasharray="3,3" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1156)"></path></g><g id="SvgjsG1158"><path id="SvgjsPath1159" d="M369.99998474121094 477.92799377441406L394.49998474121094 477.92799377441406L394.49998474121094 477.92799377441406L418.99998474121094 477.92799377441406" stroke-dasharray="3,3" stroke="#323232" stroke-width="2" fill="none" marker-end="url(#SvgjsMarker1160)"></path></g><g id="SvgjsG1162" transform="translate(435.99998474121094,448.0080051422119)"><path id="SvgjsPath1163" d="M 0 0L 243 0L 243 50.91998863220215L 0 50.91998863220215Z" stroke="none" fill="none"></path><g id="SvgjsG1164"><text id="SvgjsText1165" font-family="微软雅黑" text-anchor="start" font-size="13px" width="243px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.009994316101075" transform="rotate(0)"><tspan id="SvgjsTspan1166" dy="16" x="0"><tspan id="SvgjsTspan1167" style="text-decoration:;">pass the result directly to the output</tspan></tspan></text></g></g><g id="SvgjsG1168" transform="translate(371.99998474121094,709.0080051422119)"><path id="SvgjsPath1169" d="M 0 25.5C 0 -8.5 51 -8.5 51 25.5C 51 59.5 0 59.5 0 25.5Z" stroke-dasharray="3,4" stroke="rgba(50,50,50,1)" stroke-width="2" fill-opacity="1" fill="#cc99ff"></path><g id="SvgjsG1170"><text id="SvgjsText1171" font-family="微软雅黑" text-anchor="middle" font-size="13px" width="31px" fill="#323232" font-weight="400" align="middle" anchor="middle" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="16.05" transform="rotate(0)"></text></g></g><g id="SvgjsG1172" transform="translate(438.99998474121094,729.0880031585693)"><path id="SvgjsPath1173" d="M 0 0L 240 0L 240 19.920001983642578L 0 19.920001983642578Z" stroke="none" fill="none"></path><g id="SvgjsG1174"><text id="SvgjsText1175" font-family="微软雅黑" text-anchor="start" font-size="13px" width="240px" fill="#323232" font-weight="400" align="middle" anchor="start" family="微软雅黑" size="13px" weight="400" font-style="" opacity="1" y="-30.48999900817871" transform="rotate(0)"><tspan id="SvgjsTspan1176" dy="16" x="0"><tspan id="SvgjsTspan1177" style="text-decoration:;">(the node does not exist) </tspan><tspan id="SvgjsTspan1178" style="text-decoration:;font-weight: bold;">Only accept </tspan></tspan><tspan id="SvgjsTspan1179" dy="16" x="0"><tspan id="SvgjsTspan1180" style="text-decoration:;font-weight: bold;">input whose starting point is not </tspan></tspan><tspan id="SvgjsTspan1181" dy="16" x="0"><tspan id="SvgjsTspan1182" style="text-decoration:;font-weight: bold;">served as input for any other node.</tspan><tspan id="SvgjsTspan1183" style="text-decoration:;"> </tspan></tspan><tspan id="SvgjsTspan1184" dy="16" x="0"><tspan id="SvgjsTspan1185" style="text-decoration:;">It calculates inputs' average value as </tspan></tspan><tspan id="SvgjsTspan1186" dy="16" x="0"><tspan id="SvgjsTspan1187" style="text-decoration:;">the output of this cell.</tspan></tspan></text></g></g></svg>
\ No newline at end of file