NNICTLDOC.md 12.8 KB
Newer Older
QuanluZhang's avatar
QuanluZhang committed
1
# nnictl
Chi Song's avatar
Chi Song committed
2

QuanluZhang's avatar
QuanluZhang committed
3
## Introduction
Chi Song's avatar
Chi Song committed
4

QuanluZhang's avatar
QuanluZhang committed
5
6
7
__nnictl__ is a command line tool, which can be used to control experiments, such as start/stop/resume an experiment, start/stop NNIBoard, etc.

## Commands
Chi Song's avatar
Chi Song committed
8

QuanluZhang's avatar
QuanluZhang committed
9
nnictl support commands:
Chi Song's avatar
Chi Song committed
10

11
12
13
14
15
16
17
18
19
20
21
22
23
* [nnictl create](#create)
* [nnictl resume](#resume)
* [nnictl stop](#stop)
* [nnictl update](#update)
* [nnictl trial](#trial)
* [nnictl top](#top)
* [nnictl experiment](#experiment)
* [nnictl config](#config)
* [nnictl log](#log)
* [nnictl webui](#webui)
* [nnictl tensorboard](#tensorboard)
* [nnictl package](#package)
* [nnictl --version](#version)
Chi Song's avatar
Chi Song committed
24

Yan Ni's avatar
Yan Ni committed
25
### Manage an experiment
26

Yan Ni's avatar
Yan Ni committed
27
<a name="create"></a>
28
![](https://placehold.it/15/1589F0/000000?text=+) `nnictl create`
29

30
* Description
31

32
  You can use this command to create a new experiment, using the configuration specified in config file. 
33

34
  After this command is successfully done, the context will be set as this experiment, which means the following command you issued is associated with this experiment, unless you explicitly changes the context(not supported yet).
35

36
* Usage
37

38
39
40
  ```bash
  nnictl create [OPTIONS]
  ```
41

42
* Options
43

Chi Song's avatar
Chi Song committed
44
45
46
47
  |Name, shorthand|Required|Default|Description|
  |------|------|------|------|
  |--config, -c|  True| |YAML configure file of the experiment|
  |--port, -p|False| |the port of restful server|
48
49
  |--debug, -d|False||set debug mode|

50
* Examples
51

52
  > create a new experiment with the default port: 8080
53

54
55
56
  ```bash
  nnictl create --config nni/examples/trials/mnist/config.yml
  ```
57

58
  > create a new experiment with specified port 8088
59

60
61
62
  ```bash
  nnictl create --config nni/examples/trials/mnist/config.yml --port 8088
  ```
63

64
  > create a new experiment with specified port 8088 and debug mode
65

66
67
  ```bash
  nnictl create --config nni/examples/trials/mnist/config.yml --port 8088 --debug
68
  ```
69

70
71
72
73
74
75
Note:

```text
Debug mode will disable version check function in Trialkeeper.
```

Yan Ni's avatar
Yan Ni committed
76
<a name="resume"></a>
77
78

![](https://placehold.it/15/1589F0/000000?text=+) `nnictl resume`
QuanluZhang's avatar
QuanluZhang committed
79

80
* Description
Chi Song's avatar
Chi Song committed
81

82
  You can use this command to resume a stopped experiment.
Chi Song's avatar
Chi Song committed
83

84
* Usage
Chi Song's avatar
Chi Song committed
85

86
87
88
  ```bash
  nnictl resume [OPTIONS]
  ```
Chi Song's avatar
Chi Song committed
89

90
* Options
91

Chi Song's avatar
Chi Song committed
92
93
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
94
  |id|  True| |The id of the experiment you want to resume|  
Chi Song's avatar
Chi Song committed
95
  |--port, -p|  False| |Rest port of the experiment you want to resume|
96
  |--debug, -d|False||set debug mode|
Chi Song's avatar
Chi Song committed
97

98
* Example
99

100
  > resume an experiment with specified port 8088
101

102
103
104
  ```bash
  nnictl resume [experiment_id] --port 8088
  ```
105

Yan Ni's avatar
Yan Ni committed
106
<a name="stop"></a>
107
![](https://placehold.it/15/1589F0/000000?text=+) `nnictl stop`
108

109
* Description
Chi Song's avatar
Chi Song committed
110

111
  You can use this command to stop a running experiment or multiple experiments.
Chi Song's avatar
Chi Song committed
112

113
* Usage
Chi Song's avatar
Chi Song committed
114

115
116
117
  ```bash
  nnictl stop [id]
  ```
118

119
* Details & Examples
120

121
  1. If there is no id specified, and there is an experiment running, stop the running experiment, or print error message.
122
123
124
125
126

      ```bash
      nnictl stop
      ```

127
  1. If there is an id specified, and the id matches the running experiment, nnictl will stop the corresponding experiment, or will print error message.
128
129
130
131
132

      ```bash
      nnictl stop [experiment_id]
      ```

133
  1. Users could use 'nnictl stop all' to stop all experiments.
134
135
136
137

      ```bash
      nnictl stop all
      ```
138
139
140
141

  1. If the id ends with *, nnictl will stop all experiments whose ids matchs the regular.
  1. If the id does not exist but match the prefix of an experiment id, nnictl will stop the matched experiment.
  1. If the id does not exist but match multiple prefix of the experiment ids, nnictl will give id information.
142

Yan Ni's avatar
Yan Ni committed
143
<a name="update"></a>
144
145

![](https://placehold.it/15/1589F0/000000?text=+) `nnictl update`
146

147
148
* __nnictl update searchspace__
  * Description
149

150
    You can use this command to update an experiment's search space.
151

152
  * Usage
153

154
155
156
    ```bash
    nnictl update searchspace [OPTIONS]
    ```
157

158
  * Options
159

160
161
162
163
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
  |id|  False| |ID of the experiment you want to set|
  |--filename, -f|  True| |the file storing your new search space|
164

165
  * Example
166

167
    `update experiment's new search space with file dir 'examples/trials/mnist/search_space.json'`
168

169
170
171
    ```bash
    nnictl update searchspace [experiment_id] --file examples/trials/mnist/search_space.json
    ```
172

173
* __nnictl update concurrency__  
174

175
  * Description
176

177
     You can use this command to update an experiment's concurrency.
178

179
  * Usage
180

181
182
183
    ```bash
    nnictl update concurrency [OPTIONS]
    ```
184

185
  * Options
186

187
188
189
190
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
  |id|  False| |ID of the experiment you want to set|
  |--value, -v|  True| |the number of allowed concurrent trials|
191

192
  * Example
193

194
    > update experiment's concurrency
195

196
197
198
    ```bash
    nnictl update concurrency [experiment_id] --value [concurrency_number]
    ```
199

200
* __nnictl update duration__  
201

202
  * Description
203

204
    You can use this command to update an experiment's duration.  
205

206
  * Usage
207

208
209
210
    ```bash
    nnictl update duration [OPTIONS]
    ```
211

212
  * Options
213

214
215
216
217
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
  |id|  False| |ID of the experiment you want to set|
  |--value, -v|  True| |the experiment duration will be NUMBER seconds. SUFFIX may be 's' for seconds (the default), 'm' for minutes, 'h' for hours or 'd' for days.|
218

219
  * Example
220

221
    > update experiment's duration
222

223
224
225
    ```bash
    nnictl update duration [experiment_id] --value [duration]
    ```
226

227
228
* __nnictl update trialnum__  
  * Description
229

230
    You can use this command to update an experiment's maxtrialnum.
231

232
  * Usage
233

234
235
236
    ```bash
    nnictl update trialnum [OPTIONS]
    ```
237

238
  * Options
239

240
241
242
243
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
  |id|  False| |ID of the experiment you want to set|
  |--value, -v|  True| |the new number of maxtrialnum you want to set|
244

245
  * Example
246

247
    > update experiment's trial num
248

249
250
251
    ```bash
    nnictl update trialnum --id [experiment_id] --value [trial_num]
    ```
252

253
<a name="trial"></a>
254
![](https://placehold.it/15/1589F0/000000?text=+) `nnictl trial`
Chi Song's avatar
Chi Song committed
255

256
* __nnictl trial ls__
Chi Song's avatar
Chi Song committed
257

258
  * Description
Chi Song's avatar
Chi Song committed
259

260
    You can use this command to show trial's information.
Chi Song's avatar
Chi Song committed
261

262
  * Usage
263

264
265
266
    ```bash
    nnictl trial ls
    ```
QuanluZhang's avatar
QuanluZhang committed
267

268
  * Options
Chi Song's avatar
Chi Song committed
269

270
271
272
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
  |id|  False| |ID of the experiment you want to set|
SparkSnail's avatar
SparkSnail committed
273

274
* __nnictl trial kill__
275

276
  * Description
277

278
    You can use this command to kill a trial job.
279

280
  * Usage
281

282
283
284
    ```bash
    nnictl trial kill [OPTIONS]
    ```
285

286
  * Options
287

288
289
290
291
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
  |id|  False| |ID of the trial to be killed|
  |--experiment, -E|  True| |Experiment id of the trial|
292

293
  * Example
294

295
    > kill trail job
296

297
    ```bash
Chi Song's avatar
Chi Song committed
298
    nnictl trial [trial_id] --experiment [experiment_id]
299
300
301
302
303
304
305
306
307
308
309
310
    ```

* __nnictl trial export__
  * Description

    You can use this command to export reward & hyper-parameter of trial jobs to a csv file.

  * Usage

    ```bash
    nnictl trial export [OPTIONS]
    ```
311

312
  * Options
Yan Ni's avatar
Yan Ni committed
313
314
315
316
317
318

  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
  |id|  False| |ID of the experiment    |
  |--file|  True| |File path of the output csv file     |

Yan Ni's avatar
Yan Ni committed
319
<a name="top"></a>
320
321

![](https://placehold.it/15/1589F0/000000?text=+) `nnictl top`
SparkSnail's avatar
SparkSnail committed
322

323
* Description
324

325
  Monitor all of running experiments.
326

327
* Usage
328

329
330
331
  ```bash
  nnictl top
  ```
332

333
* Options  
334

Chi Song's avatar
Chi Song committed
335
336
337
338
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
  |id|  False| |ID of the experiment you want to set|
  |--time, -t|  False| |The interval to update the experiment status, the unit of time is second, and the default value is 3 second.|
339

Yan Ni's avatar
Yan Ni committed
340
<a name="experiment"></a>
341
![](https://placehold.it/15/1589F0/000000?text=+) `Manage experiment information`
QuanluZhang's avatar
QuanluZhang committed
342
343

* __nnictl experiment show__
Chi Song's avatar
Chi Song committed
344

QuanluZhang's avatar
QuanluZhang committed
345
  * Description
SparkSnail's avatar
SparkSnail committed
346

Chi Song's avatar
Chi Song committed
347
348
349
350
351
352
353
354
    Show the information of experiment.

  * Usage

    ```bash
    nnictl experiment show
    ```

355
  * Options
Chi Song's avatar
Chi Song committed
356

Chi Song's avatar
Chi Song committed
357
358
359
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
  |id|  False| |ID of the experiment you want to set|
SparkSnail's avatar
SparkSnail committed
360
361

* __nnictl experiment status__
Chi Song's avatar
Chi Song committed
362

SparkSnail's avatar
SparkSnail committed
363
364
  * Description

Chi Song's avatar
Chi Song committed
365
366
367
368
369
370
371
372
    Show the status of experiment.

  * Usage

    ```bash
    nnictl experiment status
    ```

373
  * Options
Chi Song's avatar
Chi Song committed
374

Chi Song's avatar
Chi Song committed
375
376
377
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
  |id|  False| |ID of the experiment you want to set|
SparkSnail's avatar
SparkSnail committed
378
379

* __nnictl experiment list__
380

SparkSnail's avatar
SparkSnail committed
381
  * Description
QuanluZhang's avatar
QuanluZhang committed
382

Chi Song's avatar
Chi Song committed
383
384
385
    Show the information of all the (running) experiments.

  * Usage
386

Chi Song's avatar
Chi Song committed
387
388
389
390
    ```bash
    nnictl experiment list
    ```

Yan Ni's avatar
Yan Ni committed
391
<a name="config"></a>
392
![](https://placehold.it/15/1589F0/000000?text=+) `nnictl config show`
393

394
* Description
395

396
  Display the current context information.
397

398
* Usage
399

400
401
402
  ```bash
  nnictl config show
  ```
403

Yan Ni's avatar
Yan Ni committed
404
<a name="log"></a>
405
406

![](https://placehold.it/15/1589F0/000000?text=+) `Manage log`
Chi Song's avatar
Chi Song committed
407

QuanluZhang's avatar
QuanluZhang committed
408
* __nnictl log stdout__
Chi Song's avatar
Chi Song committed
409
410
411
412
413
414
415
416
417
418
419

  * Description

    Show the stdout log content.

  * Usage

    ```bash
    nnictl log stdout [options]
    ```

420
  * Options
Chi Song's avatar
Chi Song committed
421

Chi Song's avatar
Chi Song committed
422
423
424
425
426
427
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
  |id|  False| |ID of the experiment you want to set|
  |--head, -h| False| |show head lines of stdout|
  |--tail, -t|  False| |show tail lines of stdout|
  |--path, -p|  False| |show the path of stdout file|
Chi Song's avatar
Chi Song committed
428

429
430
431
432
433
434
435
436
  * Example

    > Show the tail of stdout log content

    ```bash
    nnictl log stdout [experiment_id] --tail [lines_number]
    ```

QuanluZhang's avatar
QuanluZhang committed
437
438
* __nnictl log stderr__
  * Description
Chi Song's avatar
Chi Song committed
439
440

    Show the stderr log content.
441

QuanluZhang's avatar
QuanluZhang committed
442
  * Usage
Chi Song's avatar
Chi Song committed
443
444
445
446
447

    ```bash
    nnictl log stderr [options]
    ```

448
  * Options
Chi Song's avatar
Chi Song committed
449

Chi Song's avatar
Chi Song committed
450
451
452
453
454
455
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
  |id|  False| |ID of the experiment you want to set|
  |--head, -h| False| |show head lines of stderr|
  |--tail, -t|  False| |show tail lines of stderr|
  |--path, -p|  False| |show the path of stderr file|
456
457

* __nnictl log trial__
458

459
460
  * Description
  
Chi Song's avatar
Chi Song committed
461
    Show trial log path.
462
463
  
  * Usage
SparkSnail's avatar
SparkSnail committed
464

465
466
467
468
469
470
    ```bash  
    nnictl log trial [options]
    ```

  * Options

Chi Song's avatar
Chi Song committed
471
472
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
473
474
  |id|  False| |ID of the trial to be found the log path|
  |--experiment, -E|  False| |Experiment ID of the trial, required when id is not empty.|
475

Yan Ni's avatar
Yan Ni committed
476
<a name="webui"></a>
477
![](https://placehold.it/15/1589F0/000000?text=+) `Manage webui`
Chi Song's avatar
Chi Song committed
478

SparkSnail's avatar
SparkSnail committed
479
* __nnictl webui url__
SparkSnail's avatar
SparkSnail committed
480

Yan Ni's avatar
Yan Ni committed
481
<a name="tensorboard"></a>
482
![](https://placehold.it/15/1589F0/000000?text=+) `Manage tensorboard`
Chi Song's avatar
Chi Song committed
483

SparkSnail's avatar
SparkSnail committed
484
* __nnictl tensorboard start__
Chi Song's avatar
Chi Song committed
485
486
487
488
489
490
491
492
493
494
495

  * Description

    Start the tensorboard process.
  
  * Usage

    ```bash
    nnictl tensorboard start
    ```

496
  * Options
Chi Song's avatar
Chi Song committed
497

Chi Song's avatar
Chi Song committed
498
499
500
501
502
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
  |id|  False| |ID of the experiment you want to set|
  |--trialid|  False| |ID of the trial|
  |--port|  False| 6006|The port of the tensorboard process|
Chi Song's avatar
Chi Song committed
503
504
505
506
507
508
509
510

  * Detail

    1. NNICTL support tensorboard function in local and remote platform for the moment, other platforms will be supported later.   
    2. If you want to use tensorboard, you need to write your tensorboard log data to environment variable [NNI_OUTPUT_DIR] path.  
    3. In local mode, nnictl will set --logdir=[NNI_OUTPUT_DIR] directly and start a tensorboard process.
    4. In remote mode, nnictl will create a ssh client to copy log data from remote machine to local temp directory firstly, and then start a tensorboard process in your local machine. You need to notice that nnictl only copy the log data one time when you use the command, if you want to see the later result of tensorboard, you should execute nnictl tensorboard command again.
    5. If there is only one trial job, you don't need to set trialid. If there are multiple trial jobs running, you should set the trialid, or you could use [nnictl tensorboard start --trialid all] to map --logdir to all trial log paths.
SparkSnail's avatar
SparkSnail committed
511
512

* __nnictl tensorboard stop__
513
514
  * Description

515
    Stop all of the tensorboard process.
516
517
518
519
520
521
522
523
524

  * Usage

    ```bash
    nnictl tensorboard stop
    ```

  * Options

Chi Song's avatar
Chi Song committed
525
526
527
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
  |id|  False| |ID of the experiment you want to set|
Yan Ni's avatar
Yan Ni committed
528
529

<a name="package"></a>
530
531

![](https://placehold.it/15/1589F0/000000?text=+) `Manage package`
532

Yan Ni's avatar
Yan Ni committed
533
* __nnictl package install__
534
535
536
537
538
539
540
541
542
543
544
545
  * Description

    Install the packages needed in nni experiments.

  * Usage

    ```bash
    nnictl package install [OPTIONS]
    ```

  * Options

Chi Song's avatar
Chi Song committed
546
547
548
  |Name, shorthand|Required|Default|Description|
  |------|------|------ |------|
  |--name|  True| |The name of package to be installed|
Yan Ni's avatar
Yan Ni committed
549

550
551
552
553
554
555
556
557
  * Example

    > Install the packages needed in tuner SMAC

    ```bash
    nnictl package install --name=SMAC
    ```

Yan Ni's avatar
Yan Ni committed
558
* __nnictl package show__
559
560
561
562
563
564
565
566
567
568
569
570

  * Description

    List the packages supported.

  * Usage

    ```bash
    nnictl package show
    ```

<a name="version"></a>
571
572

![](https://placehold.it/15/1589F0/000000?text=+) `Check NNI version`
573
574
575
576
577
578
579
580
581
582
583

* __nnictl --version__

  * Description

    Describe the current version of NNI installed.

  * Usage

    ```bash
    nnictl --version
584
    ```