"vscode:/vscode.git/clone" did not exist on "b0f34da1332909f7a3893715ae2ad891d1fd8da4"
ExperimentConfig.md 23.8 KB
Newer Older
Chi Song's avatar
Chi Song committed
1
2
# Experiment(实验)配置参考

Chi Song's avatar
Chi Song committed
3
创建 Experiment 所需要的配置文件。 配置文件的路径会传入 `nnictl` 命令。 配置文件的格式为 YAML。 本文介绍了配置文件的内容,并提供了一些示例和模板。
Chi Song's avatar
Chi Song committed
4

Chi Song's avatar
Chi Song committed
5
6
7
8
- [Experiment(实验)配置参考](#experiment-config-reference) 
  - [模板](#template)
  - [说明](#configuration-spec)
  - [样例](#examples)
Chi Song's avatar
Chi Song committed
9
10
11

## 模板

Chi Song's avatar
Chi Song committed
12
- **简化版(不包含 Annotation(标记)和 Assessor)**
Chi Song's avatar
Chi Song committed
13
14

```yaml
Chi Song's avatar
Chi Song committed
15
16
17
18
19
authorName:
experimentName:
trialConcurrency:
maxExecDuration:
maxTrialNum:
Chi Song's avatar
Chi Song committed
20
# 可选项: local, remote, pai, kubeflow
Chi Song's avatar
Chi Song committed
21
22
trainingServicePlatform:
searchSpacePath:
Chi Song's avatar
Chi Song committed
23
# 可选项: true, false, 默认值: false
Chi Song's avatar
Chi Song committed
24
useAnnotation:
Chi Song's avatar
Chi Song committed
25
# 可选项: true, false, 默认值: false
Chi Song's avatar
Chi Song committed
26
multiPhase:
Chi Song's avatar
Chi Song committed
27
# 可选项: true, false, 默认值: false
Chi Song's avatar
Chi Song committed
28
multiThread:
Chi Song's avatar
Chi Song committed
29
tuner:
Chi Song's avatar
Chi Song committed
30
  # 可选项: TPE, Random, Anneal, Evolution
Chi Song's avatar
Chi Song committed
31
32
  builtinTunerName:
  classArgs:
Chi Song's avatar
Chi Song committed
33
    # 可选项: maximize, minimize
Chi Song's avatar
Chi Song committed
34
    optimize_mode:
Chi Song's avatar
Chi Song committed
35
  gpuIndices:
Chi Song's avatar
Chi Song committed
36
trial:
Chi Song's avatar
Chi Song committed
37
38
39
  command:
  codeDir:
  gpuNum:
Chi Song's avatar
Chi Song committed
40
# 在本机模式下,machineList 可为空
Chi Song's avatar
Chi Song committed
41
machineList:
Chi Song's avatar
Chi Song committed
42
43
44
  - ip:
    port:
    username:
Chi Song's avatar
Chi Song committed
45
46
47
    passwd:
```

Chi Song's avatar
Chi Song committed
48
- **使用 Assessor**
Chi Song's avatar
Chi Song committed
49
50
51
52
53
54
55
56
57
58

```yaml
authorName: 
experimentName: 
trialConcurrency: 
maxExecDuration: 
maxTrialNum: 
#可选项: local, remote, pai, kubeflow
trainingServicePlatform: 
searchSpacePath: 
Chi Song's avatar
Chi Song committed
59
60
61
62
63
64
#可选项: true, false, 默认值: false
useAnnotation:
#可选项: true, false, 默认值: false
multiPhase:
#可选项: true, false, 默认值: false
multiThread:
Chi Song's avatar
Chi Song committed
65
66
67
68
69
70
tuner:
  #可选项: TPE, Random, Anneal, Evolution
  builtinTunerName:
  classArgs:
    #可选项: maximize, minimize
    optimize_mode:
Chi Song's avatar
Chi Song committed
71
  gpuIndices: 
Chi Song's avatar
Chi Song committed
72
73
74
75
76
77
assessor:
  #可选项: Medianstop
  builtinAssessorName:
  classArgs:
    #可选项: maximize, minimize
    optimize_mode:
Chi Song's avatar
Chi Song committed
78
  gpuIndices: 
Chi Song's avatar
Chi Song committed
79
80
81
trial:
  command: 
  codeDir: 
Chi Song's avatar
Chi Song committed
82
  gpuIndices: 
Chi Song's avatar
Chi Song committed
83
84
85
86
87
88
89
90
#在本地使用时,machineList 可为空
machineList:
  - ip: 
    port: 
    username: 
    passwd:
```

Chi Song's avatar
Chi Song committed
91
- **使用 Annotation**
Chi Song's avatar
Chi Song committed
92
93
94
95
96
97
98
99
100

```yaml
authorName: 
experimentName: 
trialConcurrency: 
maxExecDuration: 
maxTrialNum: 
#可选项: local, remote, pai, kubeflow
trainingServicePlatform: 
Chi Song's avatar
Chi Song committed
101
102
103
104
105
106
#可选项: true, false, 默认值: false
useAnnotation:
#可选项: true, false, 默认值: false
multiPhase:
#可选项: true, false, 默认值: false
multiThread:
Chi Song's avatar
Chi Song committed
107
108
109
110
111
112
tuner:
  #可选项: TPE, Random, Anneal, Evolution
  builtinTunerName:
  classArgs:
    #可选项: maximize, minimize
    optimize_mode:
Chi Song's avatar
Chi Song committed
113
  gpuIndices: 
Chi Song's avatar
Chi Song committed
114
115
116
117
118
119
assessor:
  #可选项: Medianstop
  builtinAssessorName:
  classArgs:
    #可选项: maximize, minimize
    optimize_mode:
Chi Song's avatar
Chi Song committed
120
  gpuIndices: 
Chi Song's avatar
Chi Song committed
121
122
123
trial:
  command: 
  codeDir: 
Chi Song's avatar
Chi Song committed
124
  gpuIndices: 
Chi Song's avatar
Chi Song committed
125
126
127
128
129
130
131
132
133
134
#在本地使用时,machineList 可为空
machineList:
  - ip: 
    port: 
    username: 
    passwd:
```

## 说明

Chi Song's avatar
Chi Song committed
135
- **authorName**
Chi Song's avatar
Chi Song committed
136
  
Chi Song's avatar
Chi Song committed
137
  - 说明
Chi Song's avatar
Chi Song committed
138
    
Chi Song's avatar
Chi Song committed
139
140
141
    **authorName** 是创建 Experiment 的作者。
    
    待定: 增加默认值
Chi Song's avatar
Chi Song committed
142

Chi Song's avatar
Chi Song committed
143
- **experimentName**
Chi Song's avatar
Chi Song committed
144
  
Chi Song's avatar
Chi Song committed
145
  - 说明
Chi Song's avatar
Chi Song committed
146
    
Chi Song's avatar
Chi Song committed
147
148
149
    **experimentName** 是创建的 Experiment 的名称。
    
    待定: 增加默认值
Chi Song's avatar
Chi Song committed
150

Chi Song's avatar
Chi Song committed
151
- **trialConcurrency**
Chi Song's avatar
Chi Song committed
152
  
Chi Song's avatar
Chi Song committed
153
  - 说明
Chi Song's avatar
Chi Song committed
154
155
156
157
158
    
    **trialConcurrency** 定义了并发尝试任务的最大数量。
    
    注意:如果 trialGpuNum 大于空闲的 GPU 数量,并且并发的 Trial 任务数量还没达到 trialConcurrency,Trial 任务会被放入队列,等待分配 GPU 资源。

Chi Song's avatar
Chi Song committed
159
- **maxExecDuration**
Chi Song's avatar
Chi Song committed
160
  
Chi Song's avatar
Chi Song committed
161
  - 说明
Chi Song's avatar
Chi Song committed
162
163
164
165
166
    
    **maxExecDuration** 定义 Experiment 执行的最长时间。时间单位:{**s**, **m**, **h**, **d**},分别代表:{*seconds*, *minutes*, *hours*, *days*}。
    
    注意:maxExecDuration 设置的是 Experiment 执行的时间,不是 Trial 的。 如果 Experiment 达到了设置的最大时间,Experiment 不会停止,但不会再启动新的 Trial 作业。

Chi Song's avatar
Chi Song committed
167
168
169
170
171
172
- **versionCheck**
  
  - 说明
    
    NNI 会校验 remote, pai 和 Kubernetes 模式下 NNIManager 与 trialKeeper 进程的版本。 如果需要禁用版本校验,versionCheck 应设置为 false。

Chi Song's avatar
Chi Song committed
173
- **debug**
Chi Song's avatar
Chi Song committed
174
  
Chi Song's avatar
Chi Song committed
175
176
  - 说明
    
Chi Song's avatar
Chi Song committed
177
    调试模式会将 versionCheck 设置为 False,并将 logLevel 设置为 'debug'。
Chi Song's avatar
Chi Song committed
178
179
180
181

- **maxTrialNum**
  
  - 说明
Chi Song's avatar
Chi Song committed
182
183
184
    
    **maxTrialNum** 定义了 Trial 任务的最大数量,成功和失败的都计算在内。

Chi Song's avatar
Chi Song committed
185
- **trainingServicePlatform**
Chi Song's avatar
Chi Song committed
186
  
Chi Song's avatar
Chi Song committed
187
  - 说明
Chi Song's avatar
Chi Song committed
188
189
190
    
    **trainingServicePlatform** 定义运行 Experiment 的平台,包括:{**local**, **remote**, **pai**, **kubeflow**}.
    
Chi Song's avatar
Chi Song committed
191
    - **local** 在本机的 Ubuntu 上运行 Experiment。
Chi Song's avatar
Chi Song committed
192
    
Chi Song's avatar
Chi Song committed
193
    - **remote** 将任务提交到远程的 Ubuntu 上,必须用 **machineList** 来指定远程的 SSH 连接信息。
Chi Song's avatar
Chi Song committed
194
    
Chi Song's avatar
Chi Song committed
195
    - **pai** 提交任务到微软开源的 [OpenPAI](https://github.com/Microsoft/pai) 上。 更多 OpenPAI 配置,参考 [pai 模式](../TrainingService/PaiMode.md)
Chi Song's avatar
Chi Song committed
196
    
Chi Song's avatar
Chi Song committed
197
    - **kubeflow** 提交任务至 [Kubeflow](https://www.kubeflow.org/docs/about/kubeflow/)。 NNI 支持基于 Kubeflow 的 Kubenetes,以及[Azure Kubernetes](https://azure.microsoft.com/en-us/services/kubernetes-service/)。 详情参考 [Kubeflow 文档](../TrainingService/KubeflowMode.md)
Chi Song's avatar
Chi Song committed
198

Chi Song's avatar
Chi Song committed
199
- **searchSpacePath**
Chi Song's avatar
Chi Song committed
200
  
Chi Song's avatar
Chi Song committed
201
  - 说明
Chi Song's avatar
Chi Song committed
202
203
204
205
206
    
    **searchSpacePath** 定义搜索空间文件的路径,此文件必须在运行 nnictl 的本机。
    
    注意: 如果设置了 useAnnotation=True,searchSpacePath 字段必须被删除。

Chi Song's avatar
Chi Song committed
207
- **useAnnotation**
Chi Song's avatar
Chi Song committed
208
  
Chi Song's avatar
Chi Song committed
209
  - 说明
Chi Song's avatar
Chi Song committed
210
211
212
213
214
    
    **useAnnotation** 定义使用标记来分析代码并生成搜索空间。
    
    注意: 如果设置了 useAnnotation=True,searchSpacePath 字段必须被删除。

Chi Song's avatar
Chi Song committed
215
216
217
218
219
220
221
222
223
224
225
226
- **multiPhase**
  
  - 说明
    
    **multiPhase** 启用[多阶段 Experiment](../AdvancedFeature/MultiPhase.md)

- **multiThread**
  
  - 说明
    
    **multiThread** 如果 multiThread 设为 `true`,可启动 Dispatcher 的多线程模式。Dispatcher 会为来自 NNI 管理器的每个命令启动一个线程。

Chi Song's avatar
Chi Song committed
227
- **nniManagerIp**
Chi Song's avatar
Chi Song committed
228
  
Chi Song's avatar
Chi Song committed
229
  - 说明
Chi Song's avatar
Chi Song committed
230
231
232
233
234
    
    **nniManagerIp** 设置 NNI 管理器运行的 IP 地址。 此字段为可选项,如果没有设置,则会使用 eth0 的 IP 地址。
    
    注意: 可在 NNI 管理器机器上运行 ifconfig 来检查 eth0 是否存在。 如果不存在,推荐显式设置 nnimanagerIp。

Chi Song's avatar
Chi Song committed
235
- **logDir**
Chi Song's avatar
Chi Song committed
236
  
Chi Song's avatar
Chi Song committed
237
  - 说明
Chi Song's avatar
Chi Song committed
238
239
240
    
    **logDir** 配置存储日志和数据的目录。 默认值是 `<user home directory>/nni/experiment`

Chi Song's avatar
Chi Song committed
241
- **logLevel**
Chi Song's avatar
Chi Song committed
242
  
Chi Song's avatar
Chi Song committed
243
  - 说明
Chi Song's avatar
Chi Song committed
244
245
246
    
    **logLevel** 为 Experiment 设置日志级别,支持的日志级别有:`trace, debug, info, warning, error, fatal`。 默认值是 `info`

Chi Song's avatar
Chi Song committed
247
248
249
250
251
252
253
- **logCollection**
  
  - 说明
    
    **logCollection** 设置在 remote, pai, kubeflow, frameworkcontroller 平台下收集日志的方法。 日志支持两种设置,一种是通过 `http`,让 Trial 将日志通过 POST 方法发回日志,这种方法会减慢 trialKeeper 的速度。 另一种方法是 `none`,Trial 不将日志回传回来,仅仅回传 Job 的指标。 如果日志较大,可将此参数设置为 `none`

- **Tuner**
Chi Song's avatar
Chi Song committed
254
  
Chi Song's avatar
Chi Song committed
255
  - 说明
Chi Song's avatar
Chi Song committed
256
257
258
    
    **tuner** 指定了 Experiment 的 Tuner 算法。有两种方法可设置 Tuner。 一种方法是使用 SDK 提供的 Tuner,需要设置 **builtinTunerName****classArgs**。 另一种方法,是使用用户自定义的 Tuner,需要设置 **codeDirectory****classFileName****className****classArgs**
  
Chi Song's avatar
Chi Song committed
259
  - **builtinTunerName****classArgs**
Chi Song's avatar
Chi Song committed
260
    
Chi Song's avatar
Chi Song committed
261
    - **builtinTunerName**
Chi Song's avatar
Chi Song committed
262
      
Chi Song's avatar
Chi Song committed
263
      **builtinTunerName** 指定系统 Tuner 的名称,NNI SDK 提供了多个内置 Tuner,详情参考[这里](../Tuner/BuiltinTuner.md)
Chi Song's avatar
Chi Song committed
264
    
Chi Song's avatar
Chi Song committed
265
    - **classArgs**
Chi Song's avatar
Chi Song committed
266
      
Chi Song's avatar
Chi Song committed
267
      **classArgs** 指定了 Tuner 算法的参数。 参考[此文件](../Tuner/BuiltinTuner.md)来了解内置 Tuner 的配置参数。
Chi Song's avatar
Chi Song committed
268
  
Chi Song's avatar
Chi Song committed
269
  - **codeDir**, **classFileName**, **className****classArgs**
Chi Song's avatar
Chi Song committed
270
    
Chi Song's avatar
Chi Song committed
271
    - **codeDir**
Chi Song's avatar
Chi Song committed
272
273
274
      
      **codeDir** 指定 Tuner 代码的目录。
    
Chi Song's avatar
Chi Song committed
275
    - **classFileName**
Chi Song's avatar
Chi Song committed
276
277
278
      
      **classFileName** 指定 Tuner 文件名。
    
Chi Song's avatar
Chi Song committed
279
    - **className**
Chi Song's avatar
Chi Song committed
280
281
282
      
      **className** 指定 Tuner 类名。
    
Chi Song's avatar
Chi Song committed
283
    - **classArgs**
Chi Song's avatar
Chi Song committed
284
285
      
      **classArgs** 指定了 Tuner 算法的参数。
Chi Song's avatar
Chi Song committed
286
  
Chi Song's avatar
Chi Song committed
287
  - **gpuIndices**
Chi Song's avatar
Chi Song committed
288
    
Chi Song's avatar
Chi Song committed
289
        __gpuIndices__ 指定了 Tuner 进程可使用的 GPU。 可以指定单个或多个 GPU 索引,多个索引间使用逗号(,)隔开,例如:`1``0,1,3`。 如果没设置此字段,脚本中的 `CUDA_VISIBLE_DEVICES` 会为空 '',即 Tuner 中找不到 GPU。
Chi Song's avatar
Chi Song committed
290
291
292
293
294
295
        
  
  - **includeIntermediateResults**
    
        如果 __includeIntermediateResults__ 为 true,最后一个 Assessor 的中间结果会被发送给 Tuner 作为最终结果。 __includeIntermediateResults__ 的默认值为 false。
        
Chi Song's avatar
Chi Song committed
296
297
  
  注意:用户只能用一种方法来指定 Tuner,指定 `builtinTunerName``classArgs`,或指定 `codeDir``classFileName``className` 以及 `classArgs`
Chi Song's avatar
Chi Song committed
298

Chi Song's avatar
Chi Song committed
299
- **Assessor**
Chi Song's avatar
Chi Song committed
300
  
Chi Song's avatar
Chi Song committed
301
  - 说明
Chi Song's avatar
Chi Song committed
302
303
304
    
    **assessor** 指定了 Experiment 的 Assessor 算法。有两种方法可设置 Assessor。 一种方法是使用 SDK 提供的 Assessor,需要设置 **builtinAssessorName****classArgs**。 另一种方法,是使用用户自定义的 Assessor,需要设置 **codeDirectory****classFileName****className****classArgs**
  
Chi Song's avatar
Chi Song committed
305
  - **builtinAssessorName****classArgs**
Chi Song's avatar
Chi Song committed
306
    
Chi Song's avatar
Chi Song committed
307
    - **builtinAssessorName**
Chi Song's avatar
Chi Song committed
308
      
Chi Song's avatar
Chi Song committed
309
      **builtinAssessorName** 指定了内置 Assessor 的名称,NNI SDK 提供了多个内置的 Assessor,详情参考[这里](../Assessor/BuiltinAssessor.md)
Chi Song's avatar
Chi Song committed
310
    
Chi Song's avatar
Chi Song committed
311
    - **classArgs**
Chi Song's avatar
Chi Song committed
312
      
Chi Song's avatar
Chi Song committed
313
      **classArgs** 指定了 Assessor 算法的参数。
Chi Song's avatar
Chi Song committed
314
  
Chi Song's avatar
Chi Song committed
315
  - **codeDir**, **classFileName**, **className****classArgs**
Chi Song's avatar
Chi Song committed
316
    
Chi Song's avatar
Chi Song committed
317
    - **codeDir**
Chi Song's avatar
Chi Song committed
318
319
320
      
      **codeDir** 指定 Assessor 代码的目录。
    
Chi Song's avatar
Chi Song committed
321
    - **classFileName**
Chi Song's avatar
Chi Song committed
322
323
324
      
      **classFileName** 指定 Assessor 文件名。
    
Chi Song's avatar
Chi Song committed
325
    - **className**
Chi Song's avatar
Chi Song committed
326
327
328
      
      **className** 指定 Assessor 类名。
    
Chi Song's avatar
Chi Song committed
329
    - **classArgs**
Chi Song's avatar
Chi Song committed
330
331
332
      
      **classArgs** 指定了 Assessor 算法的参数。
  
Chi Song's avatar
Chi Song committed
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
  注意:用户只能用一种方法来指定 Assessor,指定 `builtinAssessorName``classArgs`,或指定 `codeDir``classFileName``className` 以及 `classArgs`。 如果不需要使用 Assessor,此字段可为空。

- **Advisor**
  
  - 说明
    
    **Advisor** 指定了 Experiment 的 Advisor 算法。有两种方法可设置 Advisor。 一种方法是使用 SDK 提供的 Advisor ,需要设置 **builtinAdvisorName****classArgs**。 另一种方法,是使用用户自定义的 Advisor,需要设置 **codeDirectory****classFileName****className****classArgs**
  
  - **builtinAdvisorName****classArgs**
    
    - **builtinAdvisorName**
      
      **builtinAdvisorName** 指定了内置 Advisor 的名称,NNI SDK 提供了多个[内置的 Advisor](../Tuner/BuiltinTuner.md)
    
    - **classArgs**
      
      **classArgs** 指定了 Advisor 算法的参数。 参考[此文件](../Tuner/BuiltinTuner.md)来了解内置 Advisor 的配置参数。
  
  - **codeDir**, **classFileName**, **className****classArgs**
    
    - **codeDir**
      
      **codeDir** 指定 Advisor 代码的目录。
    
    - **classFileName**
      
      **classFileName** 指定 Advisor 文件名。
Chi Song's avatar
Chi Song committed
360
    
Chi Song's avatar
Chi Song committed
361
362
363
    - **className**
      
      **className** 指定 Advisor 类名。
Chi Song's avatar
Chi Song committed
364
    
Chi Song's avatar
Chi Song committed
365
366
367
368
369
370
371
372
373
374
    - **classArgs**
      
      **classArgs** 指定了 Advisor 算法的参数。
  
  - **gpuIndices**
    
        __gpuIndices__ 指定了 Advisor 进程可使用的 GPU。 可以指定单个或多个 GPU 索引,多个索引间使用逗号(,)隔开,例如:`1``0,1,3`。 如果没设置此字段,脚本中的 `CUDA_VISIBLE_DEVICES` 会为空 '',即 Tuner 中找不到 GPU。
        
  
  注意:用户只能用一种方法来指定 Advisor ,指定 `builtinAdvisorName``classArgs`,或指定 `codeDir``classFileName``className` 以及 `classArgs`
Chi Song's avatar
Chi Song committed
375

Chi Song's avatar
Chi Song committed
376
- **trial (local, remote)**
Chi Song's avatar
Chi Song committed
377
  
Chi Song's avatar
Chi Song committed
378
  - **command**
Chi Song's avatar
Chi Song committed
379
380
381
    
    **command** 指定了运行 Trial 进程的命令行。
  
Chi Song's avatar
Chi Song committed
382
  - **codeDir**
Chi Song's avatar
Chi Song committed
383
384
385
    
    **codeDir** 指定了 Trial 代码文件的目录。
  
Chi Song's avatar
Chi Song committed
386
  - **gpuNum**
Chi Song's avatar
Chi Song committed
387
388
389
    
    **gpuNum** 指定了运行 Trial 进程的 GPU 数量。 默认值为 0。

Chi Song's avatar
Chi Song committed
390
- **trial (pai)**
Chi Song's avatar
Chi Song committed
391
  
Chi Song's avatar
Chi Song committed
392
  - **command**
Chi Song's avatar
Chi Song committed
393
394
395
    
    **command** 指定了运行 Trial 进程的命令行。
  
Chi Song's avatar
Chi Song committed
396
  - **codeDir**
Chi Song's avatar
Chi Song committed
397
398
399
    
    **codeDir** 指定了 Trial 代码文件的目录。
  
Chi Song's avatar
Chi Song committed
400
  - **gpuNum**
Chi Song's avatar
Chi Song committed
401
402
403
    
    **gpuNum** 指定了运行 Trial 进程的 GPU 数量。 默认值为 0。
  
Chi Song's avatar
Chi Song committed
404
  - **cpuNum**
Chi Song's avatar
Chi Song committed
405
406
407
    
    **cpuNum** 指定了 OpenPAI 容器中使用的 CPU 数量。
  
Chi Song's avatar
Chi Song committed
408
  - **memoryMB**
Chi Song's avatar
Chi Song committed
409
410
411
    
    **memoryMB** 指定了 OpenPAI 容器中使用的内存数量。
  
Chi Song's avatar
Chi Song committed
412
  - **image**
Chi Song's avatar
Chi Song committed
413
414
415
    
    **image** 指定了 OpenPAI 中使用的 docker 映像。
  
Chi Song's avatar
Chi Song committed
416
  - **dataDir**
Chi Song's avatar
Chi Song committed
417
418
419
    
    **dataDir** 是 HDFS 中用到的数据目录变量。
  
Chi Song's avatar
Chi Song committed
420
  - **outputDir**
Chi Song's avatar
Chi Song committed
421
422
423
    
    **outputDir** 是 HDFS 中用到的输出目录变量。在 OpenPAI 中,stdout 和 stderr 文件会在作业完成后,存放在此目录中。

Chi Song's avatar
Chi Song committed
424
- **trial (kubeflow)**
Chi Song's avatar
Chi Song committed
425
  
Chi Song's avatar
Chi Song committed
426
  - **codeDir**
Chi Song's avatar
Chi Song committed
427
428
429
    
    **codeDir** 指定了代码文件的本机路径。
  
Chi Song's avatar
Chi Song committed
430
  - **ps (可选)**
Chi Song's avatar
Chi Song committed
431
432
433
    
    **ps** 是 Kubeflow 的 Tensorflow-operator 配置。
    
Chi Song's avatar
Chi Song committed
434
    - **replicas**
Chi Song's avatar
Chi Song committed
435
436
437
      
      **replicas****ps** 角色的副本数量。
    
Chi Song's avatar
Chi Song committed
438
    - **command**
Chi Song's avatar
Chi Song committed
439
440
441
      
      **command** 是在 **ps** 的容器中运行的脚本命令。
    
Chi Song's avatar
Chi Song committed
442
    - **gpuNum**
Chi Song's avatar
Chi Song committed
443
444
445
      
      **gpuNum** 是在 **ps** 容器中使用的 GPU 数量。
    
Chi Song's avatar
Chi Song committed
446
    - **cpuNum**
Chi Song's avatar
Chi Song committed
447
448
449
      
      **cpuNum** 是在 **ps** 容器中使用的 CPU 数量。
    
Chi Song's avatar
Chi Song committed
450
    - **memoryMB**
Chi Song's avatar
Chi Song committed
451
452
453
      
      **memoryMB** 指定了容器中使用的内存数量。
    
Chi Song's avatar
Chi Song committed
454
    - **image**
Chi Song's avatar
Chi Song committed
455
      
456
      **image** 设置了 **ps** 使用的 docker 映像。
Chi Song's avatar
Chi Song committed
457
  
Chi Song's avatar
Chi Song committed
458
  - **worker**
Chi Song's avatar
Chi Song committed
459
460
461
    
    **worker** 是 Kubeflow 的 Tensorflow-operator 配置。
    
Chi Song's avatar
Chi Song committed
462
    - **replicas**
Chi Song's avatar
Chi Song committed
463
464
465
      
      **replicas****worker** 角色的副本数量。
    
Chi Song's avatar
Chi Song committed
466
    - **command**
Chi Song's avatar
Chi Song committed
467
468
469
      
      **command** 是在 **worker** 的容器中运行的脚本命令。
    
Chi Song's avatar
Chi Song committed
470
    - **gpuNum**
Chi Song's avatar
Chi Song committed
471
472
473
      
      **gpuNum** 是在 **worker** 容器中使用的 GPU 数量。
    
Chi Song's avatar
Chi Song committed
474
    - **cpuNum**
Chi Song's avatar
Chi Song committed
475
476
477
      
      **cpuNum** 是在 **worker** 容器中使用的 CPU 数量。
    
Chi Song's avatar
Chi Song committed
478
    - **memoryMB**
Chi Song's avatar
Chi Song committed
479
480
481
      
      **memoryMB** 指定了容器中使用的内存数量。
    
Chi Song's avatar
Chi Song committed
482
    - **image**
Chi Song's avatar
Chi Song committed
483
484
485
      
      **image** 设置了 **worker** 使用的 docker 映像。

Chi Song's avatar
Chi Song committed
486
487
488
489
490
491
492
- **localConfig**
  
  **localConfig** 仅在 **trainingServicePlatform** 设为 `local` 时有效,否则,配置文件中不应该有 **localConfig** 部分。
  
  - **gpuIndices**
    
    **gpuIndices** 用于指定 GPU。设置此值后,只有指定的 GPU 会被用来运行 Trial 任务。 可指定单个或多个 GPU 的索引,多个 GPU 之间用逗号(,)隔开,例如 `1``0,1,3`
Chi Song's avatar
Chi Song committed
493
494
495
496
497
498
499
500
  
  - **maxTrialNumPerGpu**
    
    **maxTrialNumPerGpu** 用于指定每个 GPU 设备上最大并发的 Trial 数量。
  
  - **useActiveGpu**
    
    **useActiveGpu** 用于指定 NNI 是否使用还有其它进程的 GPU。 默认情况下,NNI 只会使用没有其它进程的空闲 GPU,如果 **useActiveGpu** 设置为 true,NNI 会使用所有 GPU。 此字段不适用于 Windows 版的 NNI。
Chi Song's avatar
Chi Song committed
501

Chi Song's avatar
Chi Song committed
502
- **machineList**
Chi Song's avatar
Chi Song committed
503
504
505
  
  如果 **trainingServicePlatform** 为 remote,则需要设置 **machineList**。否则应将其置为空。
  
Chi Song's avatar
Chi Song committed
506
  - **ip**
Chi Song's avatar
Chi Song committed
507
508
509
    
    **ip** 是远程计算机的 ip 地址。
  
Chi Song's avatar
Chi Song committed
510
  - **port**
Chi Song's avatar
Chi Song committed
511
512
513
514
515
    
    **端口** 是用于连接远程计算机的 ssh 端口。
    
    注意:如果 port 设为空,则为默认值 22。
  
Chi Song's avatar
Chi Song committed
516
  - **username**
Chi Song's avatar
Chi Song committed
517
518
519
    
    **username** 是远程计算机的用户名。
  
Chi Song's avatar
Chi Song committed
520
  - **passwd**
Chi Song's avatar
Chi Song committed
521
522
523
    
    **passwd** 指定了账户的密码。
  
Chi Song's avatar
Chi Song committed
524
  - **sshKeyPath**
Chi Song's avatar
Chi Song committed
525
526
527
528
529
    
    如果要使用 ssh 密钥登录远程计算机,则需要设置 **sshKeyPath****sshKeyPath** 为有效的 ssh 密钥文件路径。
    
    注意:如果同时设置了 passwd 和 sshKeyPath,NNI 会使用 passwd。
  
Chi Song's avatar
Chi Song committed
530
  - **passphrase**
Chi Song's avatar
Chi Song committed
531
532
    
    **passphrase** 用于保护 ssh 密钥,如果没有使用,可为空。
Chi Song's avatar
Chi Song committed
533
534
535
536
  
  - **gpuIndices**
    
    **gpuIndices** 用于指定 GPU。设置此值后,远程计算机上只有指定的 GPU 会被用来运行 Trial 任务。 可指定单个或多个 GPU 的索引,多个 GPU 之间用逗号(,)隔开,例如 `1``0,1,3`
Chi Song's avatar
Chi Song committed
537
538
539
540
541
542
543
544
  
  - **maxTrialNumPerGpu**
    
    **maxTrialNumPerGpu** 用于指定每个 GPU 设备上最大并发的 Trial 数量。
  
  - **useActiveGpu**
    
    **useActiveGpu** 用于指定 NNI 是否使用还有其它进程的 GPU。 默认情况下,NNI 只会使用没有其它进程的空闲 GPU,如果 **useActiveGpu** 设置为 true,NNI 会使用所有 GPU。 此字段不适用于 Windows 版的 NNI。
Chi Song's avatar
Chi Song committed
545

Chi Song's avatar
Chi Song committed
546
- **kubeflowConfig**:
Chi Song's avatar
Chi Song committed
547
  
Chi Song's avatar
Chi Song committed
548
  - **operator**
Chi Song's avatar
Chi Song committed
549
550
551
    
    **operator** 指定了 kubeflow 使用的 operator,NNI 当前版本支持 **tf-operator**
  
Chi Song's avatar
Chi Song committed
552
  - **storage**
Chi Song's avatar
Chi Song committed
553
554
555
    
    **storage** 指定了 kubeflow 的存储类型,包括 {**nfs****azureStorage**}。 此字段可选,默认值为 **nfs**。 如果使用了 azureStorage,此字段必须填写。
  
Chi Song's avatar
Chi Song committed
556
  - **nfs**
Chi Song's avatar
Chi Song committed
557
558
559
560
561
    
    **server** 是 NFS 服务器的地址
    
    **path** 是 NFS 挂载的路径
  
Chi Song's avatar
Chi Song committed
562
  - **keyVault**
Chi Song's avatar
Chi Song committed
563
    
Chi Song's avatar
Chi Song committed
564
    如果用户使用 Azure Kubernetes Service,需要设置 keyVault 来使用 Azure 存储账户的私钥。 参考: https://docs.microsoft.com/zh-cn/azure/key-vault/key-vault-manage-with-cli2
Chi Song's avatar
Chi Song committed
565
    
Chi Song's avatar
Chi Song committed
566
    - **vaultName**
Chi Song's avatar
Chi Song committed
567
568
569
      
      **vaultName** 是 az 命令中 `--vault-name` 的值。
    
Chi Song's avatar
Chi Song committed
570
    - **name**
Chi Song's avatar
Chi Song committed
571
572
573
      
      **name** 是 az 命令中 `--name` 的值。
  
Chi Song's avatar
Chi Song committed
574
  - **azureStorage**
Chi Song's avatar
Chi Song committed
575
576
577
    
    如果用户使用了 Azure Kubernetes Service,需要设置 Azure 存储账户来存放代码文件。
    
Chi Song's avatar
Chi Song committed
578
    - **accountName**
Chi Song's avatar
Chi Song committed
579
580
581
      
      **accountName** 是 Azure 存储账户的名称。
    
Chi Song's avatar
Chi Song committed
582
    - **azureShare**
Chi Song's avatar
Chi Song committed
583
584
      
      **azureShare** 是 Azure 文件存储的共享参数。
Chi Song's avatar
Chi Song committed
585
586
587
588
  
  - **uploadRetryCount**
    
    如果上传文件至 Azure Storage 失败,NNI 会重试。此字段指定了重试的次数。
Chi Song's avatar
Chi Song committed
589

Chi Song's avatar
Chi Song committed
590
- **paiConfig**
Chi Song's avatar
Chi Song committed
591
  
Chi Song's avatar
Chi Song committed
592
  - **userName**
Chi Song's avatar
Chi Song committed
593
594
595
    
    **userName** 是 OpenPAI 的用户名。
  
Chi Song's avatar
Chi Song committed
596
  - **password**
Chi Song's avatar
Chi Song committed
597
598
599
    
    **password** 是 OpenPAI 用户的密码。
  
Chi Song's avatar
Chi Song committed
600
  - **host**
Chi Song's avatar
Chi Song committed
601
602
603
604
605
    
    **host** 是 OpenPAI 的主机地址。

## 样例

Chi Song's avatar
Chi Song committed
606
- **本机模式**
Chi Song's avatar
Chi Song committed
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
  
  如果要在本机运行 Trial 任务,并使用标记来生成搜索空间,可参考下列配置:
  
  ```yaml
  authorName: test
  experimentName: test_experiment
  trialConcurrency: 3
  maxExecDuration: 1h
  maxTrialNum: 10
  #可选项: local, remote, pai, kubeflow
  trainingServicePlatform: local
  #可选项: true, false
  useAnnotation: true
  tuner:
    #可选项: TPE, Random, Anneal, Evolution
    builtinTunerName: TPE
    classArgs:
      #可选项: maximize, minimize
      optimize_mode: maximize
  trial:
    command: python3 mnist.py
    codeDir: /nni/mnist
    gpuNum: 0
  ```
  
  增加 Assessor 配置
  
  ```yaml
  authorName: test
  experimentName: test_experiment
  trialConcurrency: 3
  maxExecDuration: 1h
  maxTrialNum: 10
  #可选项: local, remote, pai, kubeflow
  trainingServicePlatform: local
  searchSpacePath: /nni/search_space.json
  #可选项: true, false
  useAnnotation: false
  tuner:
    #可选项: TPE, Random, Anneal, Evolution
    builtinTunerName: TPE
    classArgs:
      #可选项: maximize, minimize
      optimize_mode: maximize
  assessor:
    #可选项: Medianstop
    builtinAssessorName: Medianstop
    classArgs:
      #可选项: maximize, minimize
      optimize_mode: maximize
  trial:
    command: python3 mnist.py
    codeDir: /nni/mnist
    gpuNum: 0
  ```
  
  或者可以指定自定义的 Tuner 和 Assessor:
  
  ```yaml
  authorName: test
  experimentName: test_experiment
  trialConcurrency: 3
  maxExecDuration: 1h
  maxTrialNum: 10
  #可选项: local, remote, pai, kubeflow
  trainingServicePlatform: local
  searchSpacePath: /nni/search_space.json
  #可选项: true, false
  useAnnotation: false
  tuner:
    codeDir: /nni/tuner
    classFileName: mytuner.py
    className: MyTuner
    classArgs:
      #可选项: maximize, minimize
      optimize_mode: maximize
  assessor:
    codeDir: /nni/assessor
    classFileName: myassessor.py
    className: MyAssessor
    classArgs:
      #choice: maximize, minimize
      optimize_mode: maximize
  trial:
    command: python3 mnist.py
    codeDir: /nni/mnist
    gpuNum: 0
  ```

Chi Song's avatar
Chi Song committed
696
- **远程模式**
Chi Song's avatar
Chi Song committed
697
  
Chi Song's avatar
Chi Song committed
698
  如果要在远程服务器上运行 Trial 任务,需要增加服务器信息:
Chi Song's avatar
Chi Song committed
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
  
  ```yaml
  authorName: test
  experimentName: test_experiment
  trialConcurrency: 3
  maxExecDuration: 1h
  maxTrialNum: 10
  #可选项: local, remote, pai, kubeflow
  trainingServicePlatform: remote
  searchSpacePath: /nni/search_space.json
  #可选项: true, false
  useAnnotation: false
  tuner:
    #可选项: TPE, Random, Anneal, Evolution
    builtinTunerName: TPE
    classArgs:
      #可选项: maximize, minimize
      optimize_mode: maximize
  trial:
    command: python3 mnist.py
    codeDir: /nni/mnist
    gpuNum: 0
  # 如果是本地 Experiment,machineList 可为空。
  machineList:
  
    - ip: 10.10.10.10
      port: 22
      username: test
      passwd: test
    - ip: 10.10.10.11
      port: 22
      username: test
      passwd: test
    - ip: 10.10.10.12
      port: 22
      username: test
      sshKeyPath: /nni/sshkey
      passphrase: qwert
  ```

Chi Song's avatar
Chi Song committed
739
- **pai 模式**
Chi Song's avatar
Chi Song committed
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
  
  ```yaml
  authorName: test
  experimentName: nni_test1
  trialConcurrency: 1
  maxExecDuration:500h
  maxTrialNum: 1
  #可选项: local, remote, pai, kubeflow
  trainingServicePlatform: pai
  searchSpacePath: search_space.json
  #可选项: true, false
  useAnnotation: false
  tuner:
    #可选项: TPE, Random, Anneal, Evolution, BatchTuner
    #SMAC (SMAC 需要使用 nnictl package 单独安装)
    builtinTunerName: TPE
    classArgs:
      #可选项: maximize, minimize
      optimize_mode: maximize
  trial:
    command: python3 main.py
    codeDir: .
    gpuNum: 4
    cpuNum: 2
    memoryMB: 10000
    # 在 OpenPAI 上用来运行 Nni 作业的 docker 映像
    image: msranni/nni:latest
    # 在 OpenPAI 的 hdfs 上存储数据的目录,如:'hdfs://host:port/directory'
    dataDir: hdfs://10.11.12.13:9000/test
    # 在 OpenPAI 的 hdfs 上存储输出的目录,如:'hdfs://host:port/directory'
    outputDir: hdfs://10.11.12.13:9000/test
  paiConfig:
    # OpenPAI 用户名
    userName: test
    # OpenPAI 密码
    passWord: test
    # OpenPAI 服务器 Ip
    host: 10.10.10.10
  ```

Chi Song's avatar
Chi Song committed
780
- **Kubeflow 模式**
Chi Song's avatar
Chi Song committed
781
782
783
784
785
786
787
788
789
  
  使用 NFS 存储。
  
  ```yaml
  authorName: default
  experimentName: example_mni
  trialConcurrency: 1
  maxExecDuration: 1h
  maxTrialNum: 1
Chi Song's avatar
Chi Song committed
790
  # 可选项: local, remote, pai, kubeflow
Chi Song's avatar
Chi Song committed
791
792
  trainingServicePlatform: kubeflow
  searchSpacePath: search_space.json
Chi Song's avatar
Chi Song committed
793
  # 可选项: true, false
Chi Song's avatar
Chi Song committed
794
795
  useAnnotation: false
  tuner:
Chi Song's avatar
Chi Song committed
796
    # 可选项: TPE, Random, Anneal, Evolution
Chi Song's avatar
Chi Song committed
797
798
    builtinTunerName: TPE
    classArgs:
Chi Song's avatar
Chi Song committed
799
      # 可选项: maximize, minimize
Chi Song's avatar
Chi Song committed
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
      optimize_mode: maximize
  trial:
    codeDir: .
    worker:
      replicas: 1
      command: python3 mnist.py
      gpuNum: 0
      cpuNum: 1
      memoryMB: 8192
      image: msranni/nni:latest
  kubeflowConfig:
    operator: tf-operator
    nfs:
      server: 10.10.10.10
      path: /var/nfs/general
  ```
  
  使用 Azure 存储。
  
  ```yaml
  authorName: default
  experimentName: example_mni
  trialConcurrency: 1
  maxExecDuration: 1h
  maxTrialNum: 1
Chi Song's avatar
Chi Song committed
825
  # 可选项: local, remote, pai, kubeflow
Chi Song's avatar
Chi Song committed
826
827
  trainingServicePlatform: kubeflow
  searchSpacePath: search_space.json
Chi Song's avatar
Chi Song committed
828
  # 可选项: true, false
Chi Song's avatar
Chi Song committed
829
830
831
  useAnnotation: false
  #nniManagerIp: 10.10.10.10
  tuner:
Chi Song's avatar
Chi Song committed
832
    # 可选项: TPE, Random, Anneal, Evolution
Chi Song's avatar
Chi Song committed
833
834
    builtinTunerName: TPE
    classArgs:
Chi Song's avatar
Chi Song committed
835
      # 可选项: maximize, minimize
Chi Song's avatar
Chi Song committed
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
      optimize_mode: maximize
  assessor:
    builtinAssessorName: Medianstop
    classArgs:
      optimize_mode: maximize
  trial:
    codeDir: .
    worker:
      replicas: 1
      command: python3 mnist.py
      gpuNum: 0
      cpuNum: 1
      memoryMB: 4096
      image: msranni/nni:latest
  kubeflowConfig:
    operator: tf-operator
    keyVault:
      vaultName: Contoso-Vault
      name: AzureStorageAccountKey
    azureStorage:
      accountName: storage
      azureShare: share01
  ```