Skip to content

GitLab

"sgl-router/src/git@developer.sourcefind.cn:change/sglang.git" did not exist on "a69b63701459d2012febe35d58b16ee40e0e98b0"
Unverified Commit 84507248 authored by QuanluZhang's avatar QuanluZhang Committed by GitHub
Browse files

update readme (#3827) 


Co-authored-by: default avatarScarlett Li <39592018+scarlett2018@users.noreply.github.com>
Co-authored-by: default avatarkvartet <48014605+kvartet@users.noreply.github.com>
Co-authored-by: default avatar--help <xyru1999@gmail.com>
parent cf95cfc0
Hide whitespace changes
Inline Side-by-side
Showing with 65 additions and 99 deletions
README.md
View file @ 84507248
......@@ -123,25 +123,19 @@ Within the following table, we summarized the current NNI capabilities, we are g
<li><a href="https://nni.readthedocs.io/en/stable/Tuner/BuiltinTuner.html#SMAC">SMAC</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/Tuner/BuiltinTuner.html#MetisTuner">Metis Tuner</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/Tuner/BuiltinTuner.html#GPTuner">GP Tuner</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/Tuner/BuiltinTuner.html#DNGOTuner">DNGO Tuner</a></li>
</ul>
<b>RL Based</b>
<ul>
<li><a href="https://nni.readthedocs.io/en/stable/Tuner/BuiltinTuner.html#PPOTuner">PPO Tuner</a> </li>
</ul>
</ul>
<a href="https://nni.readthedocs.io/en/stable/NAS/Overview.html">Neural Architecture Search</a>
<a href="https://nni.readthedocs.io/en/stable/NAS/Overview.html">Neural Architecture Search (Retiarii)</a>
<ul>
<ul>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/ENAS.html">ENAS</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/DARTS.html">DARTS</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/PDARTS.html">P-DARTS</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/CDARTS.html">CDARTS</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/SPOS.html">SPOS</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/Proxylessnas.html">ProxylessNAS</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/Tuner/BuiltinTuner.html#NetworkMorphism">Network Morphism</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/TextNAS.html">TextNAS</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/Cream.html">Cream</a></li>
</ul>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/ENAS.html">ENAS</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/DARTS.html">DARTS</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/SPOS.html">SPOS</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/Proxylessnas.html">ProxylessNAS</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/FBNet.html">FBNet</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/ExplorationStrategies.html">Reinforcement Learning</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/ExplorationStrategies.html">Regularized Evolution</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/Overview.html">More...</a></li>
</ul>
<a href="https://nni.readthedocs.io/en/stable/Compression/Overview.html">Model Compression</a>
<ul>
......@@ -154,11 +148,13 @@ Within the following table, we summarized the current NNI capabilities, we are g
<li><a href="https://nni.readthedocs.io/en/stable/Compression/Pruner.html#simulatedannealing-pruner">SimulatedAnnealing Pruner</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/Compression/Pruner.html#admm-pruner">ADMM Pruner</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/Compression/Pruner.html#autocompress-pruner">AutoCompress Pruner</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/Compression/Overview.html">More...</a></li>
</ul>
<b>Quantization</b>
<ul>
<li><a href="https://nni.readthedocs.io/en/stable/Compression/Quantizer.html#qat-quantizer">QAT Quantizer</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/Compression/Quantizer.html#dorefa-quantizer">DoReFa Quantizer</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/Compression/Quantizer.html#bnn-quantizer">BNN Quantizer</a></li>
</ul>
</ul>
<a href="https://nni.readthedocs.io/en/stable/FeatureEngineering/Overview.html">Feature Engineering (Beta)</a>
......@@ -208,6 +204,8 @@ Within the following table, we summarized the current NNI capabilities, we are g
<li><a href="https://nni.readthedocs.io/en/stable/Tuner/CustomizeTuner.html">CustomizeTuner</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/Assessor/CustomizeAssessor.html">CustomizeAssessor</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/Tutorial/InstallCustomizedAlgos.html">Install Customized Algorithms as Builtin Tuners/Assessors/Advisors</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/QuickStart.html#define-your-model-space">Define NAS Model Space</a></li>
<li><a href="https://nni.readthedocs.io/en/stable/NAS/ApiReference.html">NAS/Retiarii APIs</a></li>
</ul>
</td>
<td style="border-top:#FF0000 solid 0px;">
......
docs/en_US/Tuner/BuiltinTuner.rst
View file @ 84507248
......@@ -497,47 +497,6 @@ As a strategy in a Sequential Model-based Global Optimization (SMBO) algorithm,
selection_num_warm_up: 100000
selection_num_starting_points: 250
:raw-html:`<a name="PPOTuner"></a>`
PPO Tuner
^^^^^^^^^
..
Built-in Tuner Name: **PPOTuner**
Note that the only acceptable types within the search space are ``layer_choice`` and ``input_choice``. For ``input_choice``\ , ``n_chosen`` can only be 0, 1, or [0, 1]. Note, the search space file for NAS is usually automatically generated through the command `nnictl ss_gen <../Tutorial/Nnictl.rst>`__.
**Suggested scenario**
PPOTuner is a Reinforcement Learning tuner based on the PPO algorithm. PPOTuner can be used when using the NNI NAS interface to do neural architecture search. In general, the Reinforcement Learning algorithm needs more computing resources, though the PPO algorithm is relatively more efficient than others. It's recommended to use this tuner when you have a large amount of computional resources available. You could try it on a very simple task, such as the :githublink:`mnist-nas <examples/nas/legacy/classic_nas>` example. `See details <./PPOTuner.rst>`__
**classArgs Requirements:**
* **optimize_mode** (*'maximize' or 'minimize'*\ ) - If 'maximize', the tuner will try to maximize metrics. If 'minimize', the tuner will try to minimize metrics.
* **trials_per_update** (*int, optional, default = 20*\ ) - The number of trials to be used for one update. It must be divisible by minibatch_size. ``trials_per_update`` is recommended to be an exact multiple of ``trialConcurrency`` for better concurrency of trials.
* **epochs_per_update** (*int, optional, default = 4*\ ) - The number of epochs for one update.
* **minibatch_size** (*int, optional, default = 4*\ ) - Mini-batch size (i.e., number of trials for a mini-batch) for the update. Note that trials_per_update must be divisible by minibatch_size.
* **ent_coef** (*float, optional, default = 0.0*\ ) - Policy entropy coefficient in the optimization objective.
* **lr** (*float, optional, default = 3e-4*\ ) - Learning rate of the model (lstm network); constant.
* **vf_coef** (*float, optional, default = 0.5*\ ) - Value function loss coefficient in the optimization objective.
* **max_grad_norm** (*float, optional, default = 0.5*\ ) - Gradient norm clipping coefficient.
* **gamma** (*float, optional, default = 0.99*\ ) - Discounting factor.
* **lam** (*float, optional, default = 0.95*\ ) - Advantage estimation discounting factor (lambda in the paper).
* **cliprange** (*float, optional, default = 0.2*\ ) - Cliprange in the PPO algorithm, constant.
**Example Configuration:**
.. code-block:: yaml
# config.yml
tuner:
builtinTunerName: PPOTuner
classArgs:
optimize_mode: maximize
:raw-html:`<a name="PBTTuner"></a>`
PBT Tuner
......@@ -573,6 +532,8 @@ Population Based Training (PBT) bridges and extends parallel search methods and
Note that, to use this tuner, your trial code should be modified accordingly, please refer to `the document of PBTTuner <./PBTTuner.rst>`__ for details.
:raw-html:`<a name="DNGOTuner"></a>`
DNGO Tuner
^^^^^^^^^^
......
docs/en_US/_templates/index.html
View file @ 84507248
......@@ -5,7 +5,7 @@
{% block document %}
<div>
<div class="chinese"><a href="https://nni.readthedocs.io/zh/latest/">简体中文</a></div>
<div class="chinese"><a href="https://nni.readthedocs.io/zh/stable/">简体中文</a></div>
<b>NNI (Neural Network Intelligence)</b> is a lightweight but powerful toolkit to
help users <b>automate</b>
<a href="{{ pathto('FeatureEngineering/Overview') }}">Feature Engineering</a>,
......@@ -23,10 +23,10 @@
<a href="{{ pathto('TrainingService/RemoteMachineMode') }}">Remote Servers</a>,
<a href="{{ pathto('TrainingService/PaiMode') }}">OpenPAI</a>,
<a href="{{ pathto('TrainingService/KubeflowMode') }}">Kubeflow</a>,
<a href="{{ pathto('TrainingService/FrameworkControllerMode') }}">FrameworkController on K8S (AKS etc.)</a>
<a href="{{ pathto('TrainingService/DLTSMode') }}">DLWorkspace (aka. DLTS)</a>
<a href="{{ pathto('TrainingService/AMLMode') }}">AML (Azure Machine Learning)</a>
and other cloud options.
<a href="{{ pathto('TrainingService/FrameworkControllerMode') }}">FrameworkController on K8S (AKS etc.)</a>,
<a href="{{ pathto('TrainingService/DLTSMode') }}">DLWorkspace (aka. DLTS)</a>,
<a href="{{ pathto('TrainingService/AMLMode') }}">AML (Azure Machine Learning)</a>,
<a href="{{ pathto('TrainingService/AdaptDLMode') }}">AdaptDL (aka. ADL)</a>, other cloud options and even <a href="{{ pathto('TrainingService/HybridMode') }}">Hybrid mode</a>.
</p>
<!-- Who should consider using NNI -->
<div>
......@@ -109,13 +109,14 @@
<ul class="circle">
<li><a href="https://github.com/microsoft/nni/tree/master/examples/trials/mnist-pytorch">MNIST-pytorch</li>
</a>
<li><a href="https://github.com/microsoft/nni/tree/master/examples/trials/mnist-tfv1">MNIST-tensorflow</li>
<li><a href="https://github.com/microsoft/nni/tree/master/examples/trials/mnist-tfv2">MNIST-tensorflow</li>
</a>
<li><a href="https://github.com/microsoft/nni/tree/master/examples/trials/mnist-keras">MNIST-keras</li></a>
<li><a href="{{ pathto('TrialExample/GbdtExample') }}">Auto-gbdt</a></li>
<li><a href="{{ pathto('TrialExample/Cifar10Examples') }}">Cifar10-pytorch</li></a>
<li><a href="{{ pathto('TrialExample/SklearnExamples') }}">Scikit-learn</a></li>
<li><a href="{{ pathto('TrialExample/EfficientNet') }}">EfficientNet</a></li>
<li><a href="{{ pathto('TrialExample/OpEvoExamples') }}">Kernel Tunning</li></a>
<a href="{{ pathto('SupportedFramework_Library') }}">More...</a><br />
</ul>
</ul>
......@@ -125,54 +126,58 @@
<ul class="firstUl">
<div><b>Exhaustive search</b></div>
<ul class="circle">
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}">Random Search</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}">Grid Search</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}">Batch</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}#Random">Random Search</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}#GridSearch">Grid Search</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}#Batch">Batch</a></li>
</ul>
<div><b>Heuristic search</b></div>
<ul class="circle">
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}">Naïve Evolution</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}">Anneal</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}">Hyperband</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}#Evolution">Naïve Evolution</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}#Anneal">Anneal</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}#Hyperband">Hyperband</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}#PBTTuner">PBT</a></li>
</ul>
<div><b>Bayesian optimization</b></div>
<ul class="circle">
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}">BOHB</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}">TPE</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}">SMAC</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}">Metis Tuner</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}">GP Tuner</a> </li>
</ul>
<div><b>RL Based</b></div>
<ul class="circle">
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}">PPO Tuner</a> </li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}#BOHB">BOHB</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}#TPE">TPE</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}#SMAC">SMAC</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}#MetisTuner">Metis Tuner</a></li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}#GPTuner">GP Tuner</a> </li>
<li><a href="{{ pathto('Tuner/BuiltinTuner') }}#DNGOTuner">DNGO Tuner</a></li>
</ul>
</ul>
<a href="{{ pathto('NAS/Overview') }}">Neural Architecture Search</a>
<a href="{{ pathto('NAS/Overview') }}">Neural Architecture Search (Retiarii)</a>
<ul class="firstUl">
<ul class="circle">
<li><a href="{{ pathto('NAS/ENAS') }}">ENAS</a></li>
<li><a href="{{ pathto('NAS/DARTS') }}">DARTS</a></li>
<li><a href="{{ pathto('NAS/PDARTS') }}">P-DARTS</a></li>
<li><a href="{{ pathto('NAS/CDARTS') }}">CDARTS</a></li>
<li><a href="{{ pathto('NAS/SPOS') }}">SPOS</a></li>
<li><a href="{{ pathto('NAS/Proxylessnas') }}">ProxylessNAS</a></li>
<li><a href="{{ pathto('Tuner/NetworkmorphismTuner') }}">Network Morphism</a> </li>
<li><a href="{{ pathto('NAS/TextNAS') }}">TextNAS</a> </li>
<li><a href="{{ pathto('NAS/FBNet') }}">FBNet</a></li>
<li><a href="{{ pathto('NAS/ExplorationStrategies') }}">Reinforcement Learning</a></li>
<li><a href="{{ pathto('NAS/ExplorationStrategies') }}">Regularized Evolution</a></li>
<li><a href="{{ pathto('NAS/Overview') }}">More...</a></li>
</ul>
</ul>
<a href="{{ pathto('Compression/Overview') }}">Model Compression</a>
<ul class="firstUl">
<div><b>Pruning</b></div>
<ul class="circle">
<li><a href="{{ pathto('Compression/Pruner') }}">AGP Pruner</a></li>
<li><a href="{{ pathto('Compression/Pruner') }}">Slim Pruner</a></li>
<li><a href="{{ pathto('Compression/Pruner') }}">FPGM Pruner</a></li>
<li><a href="{{ pathto('Compression/Pruner') }}#agp-pruner">AGP Pruner</a></li>
<li><a href="{{ pathto('Compression/Pruner') }}#slim-pruner">Slim Pruner</a></li>
<li><a href="{{ pathto('Compression/Pruner') }}#fpgm-pruner">FPGM Pruner</a></li>
<li><a href="{{ pathto('Compression/Pruner') }}#netadapt-pruner">NetAdapt Pruner</a></li>
<li><a href="{{ pathto('Compression/Pruner') }}#simulatedannealing-pruner">SimulatedAnnealing Pruner</a></li>
<li><a href="{{ pathto('Compression/Pruner') }}#admm-pruner">ADMM Pruner</a></li>
<li><a href="{{ pathto('Compression/Pruner') }}#autocompress-pruner">AutoCompress Pruner</a></li>
<li><a href="{{ pathto('Compression/Overview') }}">More...</a></li>
</ul>
<div><b>Quantization</b></div>
<ul class="circle">
<li><a href="{{ pathto('Compression/Quantizer') }}">QAT Quantizer</a></li>
<li><a href="{{ pathto('Compression/Quantizer') }}">DoReFa Quantizer</a></li>
<li><a href="{{ pathto('Compression/Quantizer') }}#qat-quantize">QAT Quantizer</a></li>
<li><a href="{{ pathto('Compression/Quantizer') }}#dorefa-quantizer">DoReFa Quantizer</a></li>
<li><a href="{{ pathto('Compression/Quantizer') }}#bnn-quantizer">BNN Quantizer</a></li>
</ul>
</ul>
<a href="{{ pathto('FeatureEngineering/Overview') }}">Feature Engineering (Beta)</a>
......@@ -182,23 +187,23 @@
</ul>
<a href="{{ pathto('Assessor/BuiltinAssessor') }}">Early Stop Algorithms</a>
<ul class="circle">
<li><a href="{{ pathto('Assessor/BuiltinAssessor') }}">Median Stop</a></li>
<li><a href="{{ pathto('Assessor/BuiltinAssessor') }}">Curve Fitting</a></li>
<li><a href="{{ pathto('Assessor/BuiltinAssessor') }}#MedianStop">Median Stop</a></li>
<li><a href="{{ pathto('Assessor/BuiltinAssessor') }}#Curvefitting">Curve Fitting</a></li>
</ul>
</td>
<td>
<ul class="firstUl">
<li><a href="{{ pathto('TrainingService/LocalMode') }}">Local Machine</a></li>
<li><a href="{{ pathto('TrainingService/RemoteMachineMode') }}">Remote Servers</a></li>
<li><a href="{{ pathto('TrainingService/HybridMode') }}">Hybrid mode</a></li>
<li><a href="{{ pathto('TrainingService/AMLMode') }}">AML(Azure Machine Learning)</a></li>
<li><b>Kubernetes based services</b></li>
<ul class="circle">
<ul>
<li><a href="{{ pathto('TrainingService/PaiMode') }}">OpenPAI</a></li>
<li><a href="{{ pathto('TrainingService/KubeflowMode') }}">Kubeflow</a></li>
<li><a href="{{ pathto('TrainingService/FrameworkControllerMode') }}">FrameworkController on K8S
(AKSetc.)</a>
</li>
<li><a href="{{ pathto('TrainingService/FrameworkControllerMode') }}">FrameworkController on K8S (AKS etc.)</a></li>
<li><a href="{{ pathto('TrainingService/DLTSMode') }}">DLWorkspace (aka. DLTS)</a></li>
<li><a href="{{ pathto('TrainingService/AMLMode') }}">AML (Azure Machine Learning)</a></li>
<li><a href="{{ pathto('TrainingService/AdaptDLMode') }}">AdaptDL (aka. ADL)</a></li>
</ul>
</ul>
</td>
......@@ -207,9 +212,9 @@
<td class="verticalMiddle"><b>References</b></td>
<td>
<ul class="firstUl">
<li><a href="https://nni.readthedocs.io/en/latest/autotune_ref.html#trial">Python API</a></li>
<li><a href="{{ pathto('Tutorial/HowToLaunchFromPython') }}">Python API</a></li>
<li><a href="{{ pathto('Tutorial/AnnotationSpec') }}">NNI Annotation</a></li>
<li><a href="https://nni.readthedocs.io/en/latest/installation.html">Supported OS</a></li>
<li><a href="{{ pathto('installation') }}">Supported OS</a></li>
</ul>
</td>
<td>
......@@ -217,6 +222,8 @@
<li><a href="{{ pathto('Tuner/CustomizeTuner') }}">CustomizeTuner</a></li>
<li><a href="{{ pathto('Assessor/CustomizeAssessor') }}">CustomizeAssessor</a></li>
<li><a href="{{ pathto('Tutorial/InstallCustomizedAlgos') }}">Install Customized Algorithms as Builtin Tuners/Assessors/Advisors</a></li>
<li><a href="{{ pathto('NAS/QuickStart') }}">Define NAS Model Space</a></li>
<li><a href="{{ pathto('NAS/ApiReference') }}">NAS/Retiarii APIs</a></li>
</ul>
</td>
<td>
......@@ -274,9 +281,9 @@
<li>
<p>Run the MNIST example.</p>
<div class="command-intro">Linux or macOS</div>
<div class="command">nnictl create --config nni/examples/trials/mnist-tfv1/config.yml</div>
<div class="command">nnictl create --config nni/examples/trials/mnist-pytorch/config.yml</div>
<div class="command-intro">Windows</div>
<div class="command">nnictl create --config nni\examples\trials\mnist-tfv1\config_windows.yml</div>
<div class="command">nnictl create --config nni\examples\trials\mnist-pytorch\config_windows.yml</div>
</li>
<li>
<p>
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment