README.md 3.42 KB
Newer Older
qianyj's avatar
qianyj committed
1
2
3
<p align="center">
<img src="docs/img/nni_logo.png" width="300"/>
</p>
4

qianyj's avatar
qianyj committed
5
**NNI (Neural Network Intelligence)** 是一个帮助用户 **自动** 进行特征工程,神经网络架构搜索,超参调优以及模型压缩的轻量且强大的工具包。
6

qianyj's avatar
qianyj committed
7
NNI 管理自动机器学习 (AutoML) 的 Experiment,**调度运行** 由调优算法生成的 Trial 任务来找到最好的神经网络架构和/或超参,支持**各种训练环境**,如本机,远程服务器,OpenPAI,Kubeflow,基于 K8S 的 FrameworkController(如,AKS 等), DLWorkspace (又称 DLTS), AML (Azure Machine Learning), AdaptDL(又称 ADL) ,和其他的云平台甚至 混合模式 。 DLTS),AML (Azure Machine Learning)AdaptDL(又称 ADL) ,和其他的云平台甚至混合模式。""
Lijiaoa's avatar
Lijiaoa committed
8

qianyj's avatar
qianyj committed
9
10
11
12
13
## **使用场景**
* 想要在自己的代码、模型中试验不同的自动机器学习算法。
* 想要在不同的环境中加速运行自动机器学习。
* 想要更容易实现或试验新的自动机器学习算法的研究员或数据科学家,包括:超参调优算法,神经网络搜索算法以及模型压缩算法。
* 在机器学习平台中支持自动机器学习。
Lijiaoa's avatar
Lijiaoa committed
14

qianyj's avatar
qianyj committed
15
## **安装**
16

qianyj's avatar
qianyj committed
17
### **安装方式**
18

qianyj's avatar
qianyj committed
19
#### **使用pip方式安装**
20

qianyj's avatar
qianyj committed
21
nni whl包下载目录:[光合社区/资源工具/AI生态包/nni/dtk23.04](https://cancon.hpccube.com:65024/4/main/nni/dtk23.04),根据python版本选择对应nni的whl包。
Scarlett Li's avatar
Scarlett Li committed
22

qianyj's avatar
qianyj committed
23
24
```bash
pip3 install nni*(下载的nni的whl包)
Yuge Zhang's avatar
Yuge Zhang committed
25
26
```

qianyj's avatar
qianyj committed
27
#### **使用源码编译方式安装**
Yuge Zhang's avatar
Yuge Zhang committed
28

qianyj's avatar
qianyj committed
29
编译之前,需要先安装对应版本python,安装相应的三方包依赖项,并配置DTK环境变量(以Centos7.x为例)。
Chi Song's avatar
Chi Song committed
30

qianyj's avatar
qianyj committed
31
32
33
```bash
# 安装三方包的源
yum install  epel-release  -y
34

qianyj's avatar
qianyj committed
35
36
37
38
# 安装相关依赖项
yum install libffi-devel -y
yum install openssl-devel openssl -y
yum -y install sqlite-devel
Yuge Zhang's avatar
Yuge Zhang committed
39

qianyj's avatar
qianyj committed
40
41
42
43
44
45
# 若python内未包含相关项,需基于上面安装的三方包重新源码编译python,再配置python环境
pip3 install protobuf==3.20
pip3 install wheel
pip3 install jupyter
pip3 install jupyterlab==3.0.9
python3 -m pip install --upgrade pip setuptools
46

qianyj's avatar
qianyj committed
47
48
49
50
51
# 下载DTK并配置环境变量
# DTK tar包下载目录:光合社区/资源工具/DCU Toolkit/DTK23.04(https://cancon.hpccube.com:65024/1/main/DTK-23.04),根据系统选择对应DTK的tar包,并解压至/opt目录。
export ROCM_PATH=/opt/dtk-23.04
source /opt/dtk-23.04/env.sh
```
JSong-Jia's avatar
JSong-Jia committed
52

qianyj's avatar
qianyj committed
53
编译nni
54

qianyj's avatar
qianyj committed
55
56
57
58
59
60
61
62
63
```bash
# 下载源码 
git clone -b dtk23.04-v2.9 http://developer.hpccube.com/codes/aicomponent/nni.git
cd nni
export NNI_RELEASE=2.9
python3 setup.py clean --all
python3 setup.py build_ts
python3 setup.py bdist_wheel -p manylinux1_x86_64
```
Lijiaoa's avatar
Lijiaoa committed
64

qianyj's avatar
qianyj committed
65
安装nni
Chi Song's avatar
Chi Song committed
66

qianyj's avatar
qianyj committed
67
68
69
70
```bash
# nni的whl包会在dist文件夹生成
pip3 install ./dist/nni*
```
71

qianyj's avatar
qianyj committed
72
### **验证安装**
73

qianyj's avatar
qianyj committed
74
tensorflow
75

qianyj's avatar
qianyj committed
76
77
78
79
80
```bash
nnictl create --config nni/examples/trials/mnist-tfv1/config.yml
nnictl create --config nni/examples/trials/mnist-tfv2/config.yml
```
pytorch
81

qianyj's avatar
qianyj committed
82
83
84
85
```bash
nnictl create --config nni/examples/trials/mnist-pytorch/config.yml
``` 
在命令行中等待输出 INFO: Successfully started experiment!。 此消息表明 Experiment 已成功启动。 通过命令行输出的 Web UI url 来访问 Experiment 的界面。 此消息表明 Experiment 已成功启动。 通过命令行输出的 Web UI url 来访问 Experiment 的界面。
86

qianyj's avatar
qianyj committed
87
## **参考**
Daiki Katsuragawa's avatar
Daiki Katsuragawa committed
88

qianyj's avatar
qianyj committed
89
[README_ORIGIN.md](https://developer.hpccube.com/codes/aicomponent/nni/-/blob/master/README_ORIGIN.md)
90
91


qianyj's avatar
qianyj committed
92
## **License**
Microsoft Open Source's avatar
Microsoft Open Source committed
93

qianyj's avatar
qianyj committed
94
The entire codebase is under [MIT license](LICENSE)
Chi Song's avatar
Chi Song committed
95