Unverified Commit eda48057 authored by Yuge Zhang's avatar Yuge Zhang Committed by GitHub
Browse files

Change all master links to v1.9 (#3005)

parent a71cbe85
...@@ -57,5 +57,5 @@ Please noted in **2**. The object `trial_history` are exact the object that Tria ...@@ -57,5 +57,5 @@ Please noted in **2**. The object `trial_history` are exact the object that Tria
The working directory of your assessor is `<home>/nni-experiments/<experiment_id>/log`, which can be retrieved with environment variable `NNI_LOG_DIRECTORY`, The working directory of your assessor is `<home>/nni-experiments/<experiment_id>/log`, which can be retrieved with environment variable `NNI_LOG_DIRECTORY`,
More detail example you could see: More detail example you could see:
> * [medianstop-assessor](https://github.com/Microsoft/nni/tree/master/src/sdk/pynni/nni/medianstop_assessor) > * [medianstop-assessor](https://github.com/Microsoft/nni/tree/v1.9/src/sdk/pynni/nni/medianstop_assessor)
> * [curvefitting-assessor](https://github.com/Microsoft/nni/tree/master/src/sdk/pynni/nni/curvefitting_assessor) > * [curvefitting-assessor](https://github.com/Microsoft/nni/tree/v1.9/src/sdk/pynni/nni/curvefitting_assessor)
\ No newline at end of file \ No newline at end of file
...@@ -18,7 +18,7 @@ For now, auto-completion will not be enabled by default if you install NNI throu ...@@ -18,7 +18,7 @@ For now, auto-completion will not be enabled by default if you install NNI throu
cd ~ cd ~
wget https://raw.githubusercontent.com/microsoft/nni/{nni-version}/tools/bash-completion wget https://raw.githubusercontent.com/microsoft/nni/{nni-version}/tools/bash-completion
``` ```
Here, {nni-version} should by replaced by the version of NNI, e.g., `master`, `v1.9`. You can also check the latest `bash-completion` script [here](https://github.com/microsoft/nni/blob/master/tools/bash-completion). Here, {nni-version} should by replaced by the version of NNI, e.g., `master`, `v1.9`. You can also check the latest `bash-completion` script [here](https://github.com/microsoft/nni/blob/v1.9/tools/bash-completion).
### Step 2. Install the script ### Step 2. Install the script
If you are running a root account and want to install this script for all the users If you are running a root account and want to install this script for all the users
......
...@@ -9,7 +9,7 @@ In addition, we provide friendly instructions on the re-implementation of these ...@@ -9,7 +9,7 @@ In addition, we provide friendly instructions on the re-implementation of these
The experiments are performed with the following pruners/datasets/models: The experiments are performed with the following pruners/datasets/models:
* Models: [VGG16, ResNet18, ResNet50](https://github.com/microsoft/nni/tree/master/examples/model_compress/models/cifar10) * Models: [VGG16, ResNet18, ResNet50](https://github.com/microsoft/nni/tree/v1.9/examples/model_compress/models/cifar10)
* Datasets: CIFAR-10 * Datasets: CIFAR-10
...@@ -23,7 +23,7 @@ The experiments are performed with the following pruners/datasets/models: ...@@ -23,7 +23,7 @@ The experiments are performed with the following pruners/datasets/models:
For the pruners with scheduling, `L1Filter Pruner` is used as the base algorithm. That is to say, after the sparsities distribution is decided by the scheduling algorithm, `L1Filter Pruner` is used to performn real pruning. For the pruners with scheduling, `L1Filter Pruner` is used as the base algorithm. That is to say, after the sparsities distribution is decided by the scheduling algorithm, `L1Filter Pruner` is used to performn real pruning.
- All the pruners listed above are implemented in [nni](https://github.com/microsoft/nni/tree/master/docs/en_US/Compression/Overview.md). - All the pruners listed above are implemented in [nni](https://github.com/microsoft/nni/tree/v1.9/docs/en_US/Compression/Overview.md).
## Experiment Result ## Experiment Result
...@@ -60,14 +60,14 @@ From the experiment result, we get the following conclusions: ...@@ -60,14 +60,14 @@ From the experiment result, we get the following conclusions:
* The experiment results are all collected with the default configuration of the pruners in nni, which means that when we call a pruner class in nni, we don't change any default class arguments. * The experiment results are all collected with the default configuration of the pruners in nni, which means that when we call a pruner class in nni, we don't change any default class arguments.
* Both FLOPs and the number of parameters are counted with [Model FLOPs/Parameters Counter](https://github.com/microsoft/nni/tree/master/docs/en_US/Compression/CompressionUtils.md#model-flopsparameters-counter) after [model speed up](https://github.com/microsoft/nni/tree/master/docs/en_US/Compression/ModelSpeedup.md). * Both FLOPs and the number of parameters are counted with [Model FLOPs/Parameters Counter](https://github.com/microsoft/nni/tree/v1.9/docs/en_US/Compression/CompressionUtils.md#model-flopsparameters-counter) after [model speed up](https://github.com/microsoft/nni/tree/v1.9/docs/en_US/Compression/ModelSpeedup.md).
This avoids potential issues of counting them of masked models. This avoids potential issues of counting them of masked models.
* The experiment code can be found [here]( https://github.com/microsoft/nni/tree/master/examples/model_compress/auto_pruners_torch.py). * The experiment code can be found [here]( https://github.com/microsoft/nni/tree/v1.9/examples/model_compress/auto_pruners_torch.py).
### Experiment Result Rendering ### Experiment Result Rendering
* If you follow the practice in the [example]( https://github.com/microsoft/nni/tree/master/examples/model_compress/auto_pruners_torch.py), for every single pruning experiment, the experiment result will be saved in JSON format as follows: * If you follow the practice in the [example]( https://github.com/microsoft/nni/tree/v1.9/examples/model_compress/auto_pruners_torch.py), for every single pruning experiment, the experiment result will be saved in JSON format as follows:
``` json ``` json
{ {
"performance": {"original": 0.9298, "pruned": 0.1, "speedup": 0.1, "finetuned": 0.7746}, "performance": {"original": 0.9298, "pruned": 0.1, "speedup": 0.1, "finetuned": 0.7746},
...@@ -76,8 +76,8 @@ This avoids potential issues of counting them of masked models. ...@@ -76,8 +76,8 @@ This avoids potential issues of counting them of masked models.
} }
``` ```
* The experiment results are saved [here](https://github.com/microsoft/nni/tree/master/examples/model_compress/comparison_of_pruners). * The experiment results are saved [here](https://github.com/microsoft/nni/tree/v1.9/examples/model_compress/comparison_of_pruners).
You can refer to [analyze](https://github.com/microsoft/nni/tree/master/examples/model_compress/comparison_of_pruners/analyze.py) to plot new performance comparison figures. You can refer to [analyze](https://github.com/microsoft/nni/tree/v1.9/examples/model_compress/comparison_of_pruners/analyze.py) to plot new performance comparison figures.
## Contribution ## Contribution
......
...@@ -13,7 +13,7 @@ pruner = LevelPruner(model, config_list) ...@@ -13,7 +13,7 @@ pruner = LevelPruner(model, config_list)
pruner.compress() pruner.compress()
``` ```
The 'default' op_type stands for the module types defined in [default_layers.py](https://github.com/microsoft/nni/blob/master/src/sdk/pynni/nni/compression/torch/default_layers.py) for pytorch. The 'default' op_type stands for the module types defined in [default_layers.py](https://github.com/microsoft/nni/blob/v1.9/src/sdk/pynni/nni/compression/torch/default_layers.py) for pytorch.
Therefore ```{ 'sparsity': 0.8, 'op_types': ['default'] }```means that **all layers with specified op_types will be compressed with the same 0.8 sparsity**. When ```pruner.compress()``` called, the model is compressed with masks and after that you can normally fine tune this model and **pruned weights won't be updated** which have been masked. Therefore ```{ 'sparsity': 0.8, 'op_types': ['default'] }```means that **all layers with specified op_types will be compressed with the same 0.8 sparsity**. When ```pruner.compress()``` called, the model is compressed with masks and after that you can normally fine tune this model and **pruned weights won't be updated** which have been masked.
......
...@@ -29,7 +29,7 @@ class MyMasker(WeightMasker): ...@@ -29,7 +29,7 @@ class MyMasker(WeightMasker):
return {'weight_mask': mask} return {'weight_mask': mask}
``` ```
You can reference nni provided [weight masker](https://github.com/microsoft/nni/blob/master/src/sdk/pynni/nni/compression/torch/pruning/structured_pruning.py) implementations to implement your own weight masker. You can reference nni provided [weight masker](https://github.com/microsoft/nni/blob/v1.9/src/sdk/pynni/nni/compression/torch/pruning/structured_pruning.py) implementations to implement your own weight masker.
A basic `pruner` looks likes this: A basic `pruner` looks likes this:
...@@ -54,7 +54,7 @@ class MyPruner(Pruner): ...@@ -54,7 +54,7 @@ class MyPruner(Pruner):
``` ```
Reference nni provided [pruner](https://github.com/microsoft/nni/blob/master/src/sdk/pynni/nni/compression/torch/pruning/one_shot.py) implementations to implement your own pruner class. Reference nni provided [pruner](https://github.com/microsoft/nni/blob/v1.9/src/sdk/pynni/nni/compression/torch/pruning/one_shot.py) implementations to implement your own pruner class.
*** ***
......
...@@ -48,7 +48,7 @@ quantizer = DoReFaQuantizer(model, configure_list, optimizer) ...@@ -48,7 +48,7 @@ quantizer = DoReFaQuantizer(model, configure_list, optimizer)
quantizer.compress() quantizer.compress()
``` ```
View [example code](https://github.com/microsoft/nni/tree/master/examples/model_compress) for more information. View [example code](https://github.com/microsoft/nni/tree/v1.9/examples/model_compress) for more information.
`Compressor` class provides some utility methods for subclass and users: `Compressor` class provides some utility methods for subclass and users:
......
...@@ -32,7 +32,7 @@ start = time.time() ...@@ -32,7 +32,7 @@ start = time.time()
out = model(dummy_input) out = model(dummy_input)
print('elapsed time: ', time.time() - start) print('elapsed time: ', time.time() - start)
``` ```
For complete examples please refer to [the code](https://github.com/microsoft/nni/tree/master/examples/model_compress/model_speedup.py) For complete examples please refer to [the code](https://github.com/microsoft/nni/tree/v1.9/examples/model_compress/model_speedup.py)
NOTE: The current implementation supports PyTorch 1.3.1 or newer. NOTE: The current implementation supports PyTorch 1.3.1 or newer.
...@@ -44,7 +44,7 @@ For PyTorch we can only replace modules, if functions in `forward` should be rep ...@@ -44,7 +44,7 @@ For PyTorch we can only replace modules, if functions in `forward` should be rep
## Speedup Results of Examples ## Speedup Results of Examples
The code of these experiments can be found [here](https://github.com/microsoft/nni/tree/master/examples/model_compress/model_speedup.py). The code of these experiments can be found [here](https://github.com/microsoft/nni/tree/v1.9/examples/model_compress/model_speedup.py).
### slim pruner example ### slim pruner example
......
...@@ -43,7 +43,7 @@ Pruning algorithms compress the original network by removing redundant weights o ...@@ -43,7 +43,7 @@ Pruning algorithms compress the original network by removing redundant weights o
| [AutoCompress Pruner](https://nni.readthedocs.io/en/latest/Compression/Pruner.html#autocompress-pruner) | Automatic pruning by iteratively call SimulatedAnnealing Pruner and ADMM Pruner [Reference Paper](https://arxiv.org/abs/1907.03141) | | [AutoCompress Pruner](https://nni.readthedocs.io/en/latest/Compression/Pruner.html#autocompress-pruner) | Automatic pruning by iteratively call SimulatedAnnealing Pruner and ADMM Pruner [Reference Paper](https://arxiv.org/abs/1907.03141) |
| [AMC Pruner](https://nni.readthedocs.io/en/latest/Compression/Pruner.html#amc-pruner) | AMC: AutoML for Model Compression and Acceleration on Mobile Devices [Reference Paper](https://arxiv.org/pdf/1802.03494.pdf) | | [AMC Pruner](https://nni.readthedocs.io/en/latest/Compression/Pruner.html#amc-pruner) | AMC: AutoML for Model Compression and Acceleration on Mobile Devices [Reference Paper](https://arxiv.org/pdf/1802.03494.pdf) |
You can refer to this [benchmark](https://github.com/microsoft/nni/tree/master/docs/en_US/CommunitySharings/ModelCompressionComparison.md) for the performance of these pruners on some benchmark problems. You can refer to this [benchmark](https://github.com/microsoft/nni/tree/v1.9/docs/en_US/CommunitySharings/ModelCompressionComparison.md) for the performance of these pruners on some benchmark problems.
### Quantization Algorithms ### Quantization Algorithms
......
...@@ -102,7 +102,7 @@ We implemented one of the experiments in ['Learning Efficient Convolutional Netw ...@@ -102,7 +102,7 @@ We implemented one of the experiments in ['Learning Efficient Convolutional Netw
| VGGNet | 6.34/6.40 | 20.04M | | | VGGNet | 6.34/6.40 | 20.04M | |
| Pruned-VGGNet | 6.20/6.26 | 2.03M | 88.5% | | Pruned-VGGNet | 6.20/6.26 | 2.03M | 88.5% |
The experiments code can be found at [examples/model_compress]( https://github.com/microsoft/nni/tree/master/examples/model_compress/) The experiments code can be found at [examples/model_compress]( https://github.com/microsoft/nni/tree/v1.9/examples/model_compress/)
*** ***
...@@ -185,7 +185,7 @@ We implemented one of the experiments in ['PRUNING FILTERS FOR EFFICIENT CONVNET ...@@ -185,7 +185,7 @@ We implemented one of the experiments in ['PRUNING FILTERS FOR EFFICIENT CONVNET
| VGG-16 | 6.75/6.49 | 1.5x10^7 | | | VGG-16 | 6.75/6.49 | 1.5x10^7 | |
| VGG-16-pruned-A | 6.60/6.47 | 5.4x10^6 | 64.0% | | VGG-16-pruned-A | 6.60/6.47 | 5.4x10^6 | 64.0% |
The experiments code can be found at [examples/model_compress]( https://github.com/microsoft/nni/tree/master/examples/model_compress/) The experiments code can be found at [examples/model_compress]( https://github.com/microsoft/nni/tree/v1.9/examples/model_compress/)
*** ***
...@@ -242,7 +242,7 @@ pruner.compress() ...@@ -242,7 +242,7 @@ pruner.compress()
Note: ActivationAPoZRankFilterPruner is used to prune convolutional layers within deep neural networks, therefore the `op_types` field supports only convolutional layers. Note: ActivationAPoZRankFilterPruner is used to prune convolutional layers within deep neural networks, therefore the `op_types` field supports only convolutional layers.
You can view [example](https://github.com/microsoft/nni/blob/master/examples/model_compress/model_prune_torch.py) for more information. You can view [example](https://github.com/microsoft/nni/blob/v1.9/examples/model_compress/model_prune_torch.py) for more information.
...@@ -277,7 +277,7 @@ pruner.compress() ...@@ -277,7 +277,7 @@ pruner.compress()
Note: ActivationMeanRankFilterPruner is used to prune convolutional layers within deep neural networks, therefore the `op_types` field supports only convolutional layers. Note: ActivationMeanRankFilterPruner is used to prune convolutional layers within deep neural networks, therefore the `op_types` field supports only convolutional layers.
You can view [example](https://github.com/microsoft/nni/blob/master/examples/model_compress/model_prune_torch.py) for more information. You can view [example](https://github.com/microsoft/nni/blob/v1.9/examples/model_compress/model_prune_torch.py) for more information.
### User configuration for ActivationMeanRankFilterPruner ### User configuration for ActivationMeanRankFilterPruner
...@@ -376,7 +376,7 @@ PyTorch code ...@@ -376,7 +376,7 @@ PyTorch code
```python ```python
pruner.update_epoch(epoch) pruner.update_epoch(epoch)
``` ```
You can view [example](https://github.com/microsoft/nni/blob/master/examples/model_compress/model_prune_torch.py) for more information. You can view [example](https://github.com/microsoft/nni/blob/v1.9/examples/model_compress/model_prune_torch.py) for more information.
#### User configuration for AGP Pruner #### User configuration for AGP Pruner
...@@ -410,7 +410,7 @@ pruner = NetAdaptPruner(model, config_list, short_term_fine_tuner=short_term_fin ...@@ -410,7 +410,7 @@ pruner = NetAdaptPruner(model, config_list, short_term_fine_tuner=short_term_fin
pruner.compress() pruner.compress()
``` ```
You can view [example](https://github.com/microsoft/nni/blob/master/examples/model_compress/auto_pruners_torch.py) for more information. You can view [example](https://github.com/microsoft/nni/blob/v1.9/examples/model_compress/auto_pruners_torch.py) for more information.
#### User configuration for NetAdapt Pruner #### User configuration for NetAdapt Pruner
...@@ -449,7 +449,7 @@ pruner = SimulatedAnnealingPruner(model, config_list, evaluator=evaluator, base_ ...@@ -449,7 +449,7 @@ pruner = SimulatedAnnealingPruner(model, config_list, evaluator=evaluator, base_
pruner.compress() pruner.compress()
``` ```
You can view [example](https://github.com/microsoft/nni/blob/master/examples/model_compress/auto_pruners_torch.py) for more information. You can view [example](https://github.com/microsoft/nni/blob/v1.9/examples/model_compress/auto_pruners_torch.py) for more information.
#### User configuration for SimulatedAnnealing Pruner #### User configuration for SimulatedAnnealing Pruner
...@@ -485,7 +485,7 @@ pruner = AutoCompressPruner( ...@@ -485,7 +485,7 @@ pruner = AutoCompressPruner(
pruner.compress() pruner.compress()
``` ```
You can view [example](https://github.com/microsoft/nni/blob/master/examples/model_compress/auto_pruners_torch.py) for more information. You can view [example](https://github.com/microsoft/nni/blob/v1.9/examples/model_compress/auto_pruners_torch.py) for more information.
#### User configuration for AutoCompress Pruner #### User configuration for AutoCompress Pruner
...@@ -519,7 +519,7 @@ pruner = AMCPruner(model, config_list, evaluator, val_loader, flops_ratio=0.5) ...@@ -519,7 +519,7 @@ pruner = AMCPruner(model, config_list, evaluator, val_loader, flops_ratio=0.5)
pruner.compress() pruner.compress()
``` ```
You can view [example](https://github.com/microsoft/nni/blob/master/examples/model_compress/amc/) for more information. You can view [example](https://github.com/microsoft/nni/blob/v1.9/examples/model_compress/amc/) for more information.
#### User configuration for AutoCompress Pruner #### User configuration for AutoCompress Pruner
...@@ -537,7 +537,7 @@ We implemented one of the experiments in [AMC: AutoML for Model Compression and ...@@ -537,7 +537,7 @@ We implemented one of the experiments in [AMC: AutoML for Model Compression and
| ------------- | --------------| -------------- | ----- | | ------------- | --------------| -------------- | ----- |
| MobileNet | 70.5% / 69.9% | 89.3% / 89.1% | 50% | | MobileNet | 70.5% / 69.9% | 89.3% / 89.1% | 50% |
The experiments code can be found at [examples/model_compress]( https://github.com/microsoft/nni/tree/master/examples/model_compress/amc/) The experiments code can be found at [examples/model_compress]( https://github.com/microsoft/nni/tree/v1.9/examples/model_compress/amc/)
## ADMM Pruner ## ADMM Pruner
Alternating Direction Method of Multipliers (ADMM) is a mathematical optimization technique, Alternating Direction Method of Multipliers (ADMM) is a mathematical optimization technique,
...@@ -568,7 +568,7 @@ pruner = ADMMPruner(model, config_list, trainer=trainer, num_iterations=30, epoc ...@@ -568,7 +568,7 @@ pruner = ADMMPruner(model, config_list, trainer=trainer, num_iterations=30, epoc
pruner.compress() pruner.compress()
``` ```
You can view [example](https://github.com/microsoft/nni/blob/master/examples/model_compress/auto_pruners_torch.py) for more information. You can view [example](https://github.com/microsoft/nni/blob/v1.9/examples/model_compress/auto_pruners_torch.py) for more information.
#### User configuration for ADMM Pruner #### User configuration for ADMM Pruner
...@@ -624,7 +624,7 @@ The above configuration means that there are 5 times of iterative pruning. As th ...@@ -624,7 +624,7 @@ The above configuration means that there are 5 times of iterative pruning. As th
### Reproduced Experiment ### Reproduced Experiment
We try to reproduce the experiment result of the fully connected network on MNIST using the same configuration as in the paper. The code can be referred [here](https://github.com/microsoft/nni/tree/master/examples/model_compress/lottery_torch_mnist_fc.py). In this experiment, we prune 10 times, for each pruning we train the pruned model for 50 epochs. We try to reproduce the experiment result of the fully connected network on MNIST using the same configuration as in the paper. The code can be referred [here](https://github.com/microsoft/nni/tree/v1.9/examples/model_compress/lottery_torch_mnist_fc.py). In this experiment, we prune 10 times, for each pruning we train the pruned model for 50 epochs.
![](../../img/lottery_ticket_mnist_fc.png) ![](../../img/lottery_ticket_mnist_fc.png)
......
...@@ -129,7 +129,7 @@ quantizer = BNNQuantizer(model, configure_list) ...@@ -129,7 +129,7 @@ quantizer = BNNQuantizer(model, configure_list)
model = quantizer.compress() model = quantizer.compress()
``` ```
You can view example [examples/model_compress/BNN_quantizer_cifar10.py]( https://github.com/microsoft/nni/tree/master/examples/model_compress/BNN_quantizer_cifar10.py) for more information. You can view example [examples/model_compress/BNN_quantizer_cifar10.py]( https://github.com/microsoft/nni/tree/v1.9/examples/model_compress/BNN_quantizer_cifar10.py) for more information.
#### User configuration for BNN Quantizer #### User configuration for BNN Quantizer
...@@ -146,4 +146,4 @@ We implemented one of the experiments in [Binarized Neural Networks: Training De ...@@ -146,4 +146,4 @@ We implemented one of the experiments in [Binarized Neural Networks: Training De
| VGGNet | 86.93% | | VGGNet | 86.93% |
The experiments code can be found at [examples/model_compress/BNN_quantizer_cifar10.py]( https://github.com/microsoft/nni/tree/master/examples/model_compress/BNN_quantizer_cifar10.py) The experiments code can be found at [examples/model_compress/BNN_quantizer_cifar10.py]( https://github.com/microsoft/nni/tree/v1.9/examples/model_compress/BNN_quantizer_cifar10.py)
\ No newline at end of file \ No newline at end of file
...@@ -42,7 +42,7 @@ After training, you get accuracy of the pruned model. You can export model weigh ...@@ -42,7 +42,7 @@ After training, you get accuracy of the pruned model. You can export model weigh
pruner.export_model(model_path='pruned_vgg19_cifar10.pth', mask_path='mask_vgg19_cifar10.pth') pruner.export_model(model_path='pruned_vgg19_cifar10.pth', mask_path='mask_vgg19_cifar10.pth')
``` ```
The complete code of model compression examples can be found [here](https://github.com/microsoft/nni/blob/master/examples/model_compress/model_prune_torch.py). The complete code of model compression examples can be found [here](https://github.com/microsoft/nni/blob/v1.9/examples/model_compress/model_prune_torch.py).
### Speed up the model ### Speed up the model
......
...@@ -11,7 +11,7 @@ These selectors are suitable for tabular data(which means it doesn't include ima ...@@ -11,7 +11,7 @@ These selectors are suitable for tabular data(which means it doesn't include ima
In addition, those selector only for feature selection. If you want to: In addition, those selector only for feature selection. If you want to:
1) generate high-order combined features on nni while doing feature selection; 1) generate high-order combined features on nni while doing feature selection;
2) leverage your distributed resources; 2) leverage your distributed resources;
you could try this [example](https://github.com/microsoft/nni/tree/master/examples/feature_engineering/auto-feature-engineering). you could try this [example](https://github.com/microsoft/nni/tree/v1.9/examples/feature_engineering/auto-feature-engineering).
## How to use? ## How to use?
...@@ -266,6 +266,6 @@ The code could be refenrence `/examples/feature_engineering/gradient_feature_sel ...@@ -266,6 +266,6 @@ The code could be refenrence `/examples/feature_engineering/gradient_feature_sel
## Reference and Feedback ## Reference and Feedback
* To [report a bug](https://github.com/microsoft/nni/issues/new?template=bug-report.md) for this feature in GitHub; * To [report a bug](https://github.com/microsoft/nni/issues/new?template=bug-report.md) for this feature in GitHub;
* To [file a feature or improvement request](https://github.com/microsoft/nni/issues/new?template=enhancement.md) for this feature in GitHub; * To [file a feature or improvement request](https://github.com/microsoft/nni/issues/new?template=enhancement.md) for this feature in GitHub;
* To know more about [Neural Architecture Search with NNI](https://github.com/microsoft/nni/blob/master/docs/en_US/NAS/Overview.md); * To know more about [Neural Architecture Search with NNI](https://github.com/microsoft/nni/blob/v1.9/docs/en_US/NAS/Overview.md);
* To know more about [Model Compression with NNI](https://github.com/microsoft/nni/blob/master/docs/en_US/Compression/Overview.md); * To know more about [Model Compression with NNI](https://github.com/microsoft/nni/blob/v1.9/docs/en_US/Compression/Overview.md);
* To know more about [Hyperparameter Tuning with NNI](https://github.com/microsoft/nni/blob/master/docs/en_US/Tuner/BuiltinTuner.md); * To know more about [Hyperparameter Tuning with NNI](https://github.com/microsoft/nni/blob/v1.9/docs/en_US/Tuner/BuiltinTuner.md);
...@@ -76,9 +76,9 @@ class RandomMutator(Mutator): ...@@ -76,9 +76,9 @@ class RandomMutator(Mutator):
return self.sample_search() # use the same logic here. you can do something different return self.sample_search() # use the same logic here. you can do something different
``` ```
The complete example of random mutator can be found [here](https://github.com/microsoft/nni/blob/master/src/sdk/pynni/nni/nas/pytorch/random/mutator.py). The complete example of random mutator can be found [here](https://github.com/microsoft/nni/blob/v1.9/src/sdk/pynni/nni/nas/pytorch/random/mutator.py).
For advanced usages, e.g., users want to manipulate the way modules in `LayerChoice` are executed, they can inherit `BaseMutator`, and overwrite `on_forward_layer_choice` and `on_forward_input_choice`, which are the callback implementation of `LayerChoice` and `InputChoice` respectively. Users can still use property `mutables` to get all `LayerChoice` and `InputChoice` in the model code. For details, please refer to [reference](https://github.com/microsoft/nni/tree/master/src/sdk/pynni/nni/nas/pytorch) here to learn more. For advanced usages, e.g., users want to manipulate the way modules in `LayerChoice` are executed, they can inherit `BaseMutator`, and overwrite `on_forward_layer_choice` and `on_forward_input_choice`, which are the callback implementation of `LayerChoice` and `InputChoice` respectively. Users can still use property `mutables` to get all `LayerChoice` and `InputChoice` in the model code. For details, please refer to [reference](https://github.com/microsoft/nni/tree/v1.9/src/sdk/pynni/nni/nas/pytorch) here to learn more.
```eval_rst ```eval_rst
.. tip:: .. tip::
......
...@@ -19,7 +19,7 @@ This is CDARTS based on the NNI platform, which currently supports CIFAR10 searc ...@@ -19,7 +19,7 @@ This is CDARTS based on the NNI platform, which currently supports CIFAR10 searc
## Examples ## Examples
[Example code](https://github.com/microsoft/nni/tree/master/examples/nas/cdarts) [Example code](https://github.com/microsoft/nni/tree/v1.9/examples/nas/cdarts)
```bash ```bash
# In case NNI code is not cloned. If the code is cloned already, ignore this line and enter code folder. # In case NNI code is not cloned. If the code is cloned already, ignore this line and enter code folder.
......
...@@ -24,9 +24,9 @@ At this point, trial code is ready. Then, we can prepare an NNI experiment, i.e. ...@@ -24,9 +24,9 @@ At this point, trial code is ready. Then, we can prepare an NNI experiment, i.e.
A file named `nni_auto_gen_search_space.json` is generated by this command. Then put the path of the generated search space in the field `searchSpacePath` of the experiment config file. The other fields of the config file can be filled by referring [this tutorial](../Tutorial/QuickStart.md). A file named `nni_auto_gen_search_space.json` is generated by this command. Then put the path of the generated search space in the field `searchSpacePath` of the experiment config file. The other fields of the config file can be filled by referring [this tutorial](../Tutorial/QuickStart.md).
Currently, we only support [PPO Tuner](../Tuner/BuiltinTuner.md), [Regularized Evolution Tuner](#regulaized-evolution-tuner) and [Random Tuner](https://github.com/microsoft/nni/tree/master/examples/tuners/random_nas_tuner) for classic NAS. More classic NAS algorithms will be supported soon. Currently, we only support [PPO Tuner](../Tuner/BuiltinTuner.md), [Regularized Evolution Tuner](#regulaized-evolution-tuner) and [Random Tuner](https://github.com/microsoft/nni/tree/v1.9/examples/tuners/random_nas_tuner) for classic NAS. More classic NAS algorithms will be supported soon.
The complete examples can be found [here](https://github.com/microsoft/nni/tree/master/examples/nas/classic_nas) for PyTorch and [here](https://github.com/microsoft/nni/tree/master/examples/nas/classic_nas-tf) for TensorFlow. The complete examples can be found [here](https://github.com/microsoft/nni/tree/v1.9/examples/nas/classic_nas) for PyTorch and [here](https://github.com/microsoft/nni/tree/v1.9/examples/nas/classic_nas-tf) for TensorFlow.
## Standalone mode for easy debugging ## Standalone mode for easy debugging
......
...@@ -21,7 +21,7 @@ The above-mentioned example is meant to reproduce the results in the paper, we d ...@@ -21,7 +21,7 @@ The above-mentioned example is meant to reproduce the results in the paper, we d
### CNN Search Space ### CNN Search Space
[Example code](https://github.com/microsoft/nni/tree/master/examples/nas/darts) [Example code](https://github.com/microsoft/nni/tree/v1.9/examples/nas/darts)
```bash ```bash
# In case NNI code is not cloned. If the code is cloned already, ignore this line and enter code folder. # In case NNI code is not cloned. If the code is cloned already, ignore this line and enter code folder.
......
...@@ -10,7 +10,7 @@ Implementation on NNI is based on the [official implementation in Tensorflow](ht ...@@ -10,7 +10,7 @@ Implementation on NNI is based on the [official implementation in Tensorflow](ht
### CIFAR10 Macro/Micro Search Space ### CIFAR10 Macro/Micro Search Space
[Example code](https://github.com/microsoft/nni/tree/master/examples/nas/enas) [Example code](https://github.com/microsoft/nni/tree/v1.9/examples/nas/enas)
```bash ```bash
# In case NNI code is not cloned. If the code is cloned already, ignore this line and enter code folder. # In case NNI code is not cloned. If the code is cloned already, ignore this line and enter code folder.
......
...@@ -22,7 +22,7 @@ The procedure of classic NAS algorithms is similar to hyper-parameter tuning, us ...@@ -22,7 +22,7 @@ The procedure of classic NAS algorithms is similar to hyper-parameter tuning, us
|Name|Brief Introduction of Algorithm| |Name|Brief Introduction of Algorithm|
|---|---| |---|---|
| [Random Search](https://github.com/microsoft/nni/tree/master/examples/tuners/random_nas_tuner) | Randomly pick a model from search space | | [Random Search](https://github.com/microsoft/nni/tree/v1.9/examples/tuners/random_nas_tuner) | Randomly pick a model from search space |
| [PPO Tuner](https://nni.readthedocs.io/en/latest/Tuner/BuiltinTuner.html#PPOTuner) | PPO Tuner is a Reinforcement Learning tuner based on PPO algorithm. [Reference Paper](https://arxiv.org/abs/1707.06347) | | [PPO Tuner](https://nni.readthedocs.io/en/latest/Tuner/BuiltinTuner.html#PPOTuner) | PPO Tuner is a Reinforcement Learning tuner based on PPO algorithm. [Reference Paper](https://arxiv.org/abs/1707.06347) |
Please refer to [here](ClassicNas.md) for the usage of classic NAS algorithms. Please refer to [here](ClassicNas.md) for the usage of classic NAS algorithms.
......
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
## Examples ## Examples
[Example code](https://github.com/microsoft/nni/tree/master/examples/nas/pdarts) [Example code](https://github.com/microsoft/nni/tree/v1.9/examples/nas/pdarts)
```bash ```bash
# In case NNI code is not cloned. If the code is cloned already, ignore this line and enter code folder. # In case NNI code is not cloned. If the code is cloned already, ignore this line and enter code folder.
......
...@@ -19,7 +19,7 @@ trainer = ProxylessNasTrainer(model, ...@@ -19,7 +19,7 @@ trainer = ProxylessNasTrainer(model,
trainer.train() trainer.train()
trainer.export(args.arch_path) trainer.export(args.arch_path)
``` ```
The complete example code can be found [here](https://github.com/microsoft/nni/tree/master/examples/nas/proxylessnas). The complete example code can be found [here](https://github.com/microsoft/nni/tree/v1.9/examples/nas/proxylessnas).
**Input arguments of ProxylessNasTrainer** **Input arguments of ProxylessNasTrainer**
...@@ -50,7 +50,7 @@ The complete example code can be found [here](https://github.com/microsoft/nni/t ...@@ -50,7 +50,7 @@ The complete example code can be found [here](https://github.com/microsoft/nni/t
The implementation on NNI is based on the [offical implementation](https://github.com/mit-han-lab/ProxylessNAS). The official implementation supports two training approaches: gradient descent and RL based, and support different targeted hardware, including 'mobile', 'cpu', 'gpu8', 'flops'. In our current implementation on NNI, gradient descent training approach is supported, but has not supported different hardwares. The complete support is ongoing. The implementation on NNI is based on the [offical implementation](https://github.com/mit-han-lab/ProxylessNAS). The official implementation supports two training approaches: gradient descent and RL based, and support different targeted hardware, including 'mobile', 'cpu', 'gpu8', 'flops'. In our current implementation on NNI, gradient descent training approach is supported, but has not supported different hardwares. The complete support is ongoing.
Below we will describe implementation details. Like other one-shot NAS algorithms on NNI, ProxylessNAS is composed of two parts: *search space* and *training approach*. For users to flexibly define their own search space and use built-in ProxylessNAS training approach, we put the specified search space in [example code](https://github.com/microsoft/nni/tree/master/examples/nas/proxylessnas) using [NNI NAS interface](NasGuide.md), and put the training approach in [SDK](https://github.com/microsoft/nni/tree/master/src/sdk/pynni/nni/nas/pytorch/proxylessnas). Below we will describe implementation details. Like other one-shot NAS algorithms on NNI, ProxylessNAS is composed of two parts: *search space* and *training approach*. For users to flexibly define their own search space and use built-in ProxylessNAS training approach, we put the specified search space in [example code](https://github.com/microsoft/nni/tree/v1.9/examples/nas/proxylessnas) using [NNI NAS interface](NasGuide.md), and put the training approach in [SDK](https://github.com/microsoft/nni/tree/v1.9/src/sdk/pynni/nni/nas/pytorch/proxylessnas).
![](../../img/proxylessnas.png) ![](../../img/proxylessnas.png)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment