README.md 3.42 KB
Newer Older
“qianyj”'s avatar
“qianyj” committed
1
2
3
<p align="center">
<img src="docs/img/nni_logo.png" width="300"/>
</p>
qianyj's avatar
qianyj committed
4

“qianyj”'s avatar
“qianyj” committed
5
**NNI (Neural Network Intelligence)** 是一个帮助用户 **自动** 进行特征工程,神经网络架构搜索,超参调优以及模型压缩的轻量且强大的工具包。
qianyj's avatar
qianyj committed
6

“qianyj”'s avatar
“qianyj” committed
7
NNI 管理自动机器学习 (AutoML) 的 Experiment,**调度运行** 由调优算法生成的 Trial 任务来找到最好的神经网络架构和/或超参,支持**各种训练环境**,如本机,远程服务器,OpenPAI,Kubeflow,基于 K8S 的 FrameworkController(如,AKS 等), DLWorkspace (又称 DLTS), AML (Azure Machine Learning), AdaptDL(又称 ADL) ,和其他的云平台甚至 混合模式 。 DLTS),AML (Azure Machine Learning)AdaptDL(又称 ADL) ,和其他的云平台甚至混合模式。""
qianyj's avatar
qianyj committed
8

“qianyj”'s avatar
“qianyj” committed
9
10
11
12
13
## **使用场景**
* 想要在自己的代码、模型中试验不同的自动机器学习算法。
* 想要在不同的环境中加速运行自动机器学习。
* 想要更容易实现或试验新的自动机器学习算法的研究员或数据科学家,包括:超参调优算法,神经网络搜索算法以及模型压缩算法。
* 在机器学习平台中支持自动机器学习。
qianyj's avatar
qianyj committed
14

“qianyj”'s avatar
“qianyj” committed
15
## **安装**
qianyj's avatar
qianyj committed
16

“qianyj”'s avatar
“qianyj” committed
17
### **安装方式**
qianyj's avatar
qianyj committed
18

“qianyj”'s avatar
“qianyj” committed
19
#### **使用pip方式安装**
qianyj's avatar
qianyj committed
20

“qianyj”'s avatar
“qianyj” committed
21
nni whl包下载目录:[光合社区/资源工具/AI生态包/nni/dtk23.04](https://cancon.hpccube.com:65024/4/main/nni/dtk23.04),根据python版本选择对应nni的whl包。
qianyj's avatar
qianyj committed
22

“qianyj”'s avatar
“qianyj” committed
23
24
```bash
pip3 install nni*(下载的nni的whl包)
qianyj's avatar
qianyj committed
25
26
```

“qianyj”'s avatar
“qianyj” committed
27
#### **使用源码编译方式安装**
qianyj's avatar
qianyj committed
28

“qianyj”'s avatar
“qianyj” committed
29
编译之前,需要先安装对应版本python,安装相应的三方包依赖项,并配置DTK环境变量(以Centos7.x为例)。
qianyj's avatar
qianyj committed
30

“qianyj”'s avatar
“qianyj” committed
31
32
33
```bash
# 安装三方包的源
yum install  epel-release  -y
qianyj's avatar
qianyj committed
34

“qianyj”'s avatar
“qianyj” committed
35
36
37
38
# 安装相关依赖项
yum install libffi-devel -y
yum install openssl-devel openssl -y
yum -y install sqlite-devel
qianyj's avatar
qianyj committed
39

“qianyj”'s avatar
“qianyj” committed
40
41
42
43
44
45
# 若python内未包含相关项,需基于上面安装的三方包重新源码编译python,再配置python环境
pip3 install protobuf==3.20
pip3 install wheel
pip3 install jupyter
pip3 install jupyterlab==3.0.9
python3 -m pip install --upgrade pip setuptools
qianyj's avatar
qianyj committed
46

“qianyj”'s avatar
“qianyj” committed
47
48
49
50
51
# 下载DTK并配置环境变量
# DTK tar包下载目录:光合社区/资源工具/DCU Toolkit/DTK23.04(https://cancon.hpccube.com:65024/1/main/DTK-23.04),根据系统选择对应DTK的tar包,并解压至/opt目录。
export ROCM_PATH=/opt/dtk-23.04
source /opt/dtk-23.04/env.sh
```
qianyj's avatar
qianyj committed
52

“qianyj”'s avatar
“qianyj” committed
53
编译nni
qianyj's avatar
qianyj committed
54

“qianyj”'s avatar
“qianyj” committed
55
56
57
58
59
60
61
62
63
```bash
# 下载源码 
git clone -b dtk23.04-v2.9 http://developer.hpccube.com/codes/aicomponent/nni.git
cd nni
export NNI_RELEASE=2.9
python3 setup.py clean --all
python3 setup.py build_ts
python3 setup.py bdist_wheel -p manylinux1_x86_64
```
qianyj's avatar
qianyj committed
64

“qianyj”'s avatar
“qianyj” committed
65
安装nni
qianyj's avatar
qianyj committed
66

“qianyj”'s avatar
“qianyj” committed
67
68
69
70
```bash
# nni的whl包会在dist文件夹生成
pip3 install ./dist/nni*
```
qianyj's avatar
qianyj committed
71

“qianyj”'s avatar
“qianyj” committed
72
### **验证安装**
qianyj's avatar
qianyj committed
73

“qianyj”'s avatar
“qianyj” committed
74
tensorflow
qianyj's avatar
qianyj committed
75

“qianyj”'s avatar
“qianyj” committed
76
77
78
79
80
```bash
nnictl create --config nni/examples/trials/mnist-tfv1/config.yml
nnictl create --config nni/examples/trials/mnist-tfv2/config.yml
```
pytorch
qianyj's avatar
qianyj committed
81

“qianyj”'s avatar
“qianyj” committed
82
83
84
85
```bash
nnictl create --config nni/examples/trials/mnist-pytorch/config.yml
``` 
在命令行中等待输出 INFO: Successfully started experiment!。 此消息表明 Experiment 已成功启动。 通过命令行输出的 Web UI url 来访问 Experiment 的界面。 此消息表明 Experiment 已成功启动。 通过命令行输出的 Web UI url 来访问 Experiment 的界面。
qianyj's avatar
qianyj committed
86

“qianyj”'s avatar
“qianyj” committed
87
## **参考**
qianyj's avatar
qianyj committed
88

“qianyj”'s avatar
“qianyj” committed
89
[README_ORIGIN.md](https://developer.hpccube.com/codes/aicomponent/nni/-/blob/dtk23.04-v2.9/README_ORIGIN.md)
qianyj's avatar
qianyj committed
90

“qianyj”'s avatar
“qianyj” committed
91
## **License**
qianyj's avatar
qianyj committed
92

“qianyj”'s avatar
“qianyj” committed
93
The entire codebase is under [MIT license](LICENSE)
qianyj's avatar
qianyj committed
94