CustomizeTuner.md 4.69 KB
Newer Older
Yan Ni's avatar
Yan Ni committed
1
2
3
4
# Customize-Tuner

## Customize Tuner

Chi Song's avatar
Chi Song committed
5
NNI provides state-of-the-art tuning algorithm in builtin-tuners. NNI supports to build a tuner by yourself for tuning demand.
Yan Ni's avatar
Yan Ni committed
6

Chi Song's avatar
Chi Song committed
7
If you want to implement your own tuning algorithm, you can implement a customized Tuner, there are three things to do:
Yan Ni's avatar
Yan Ni committed
8

Chi Song's avatar
Chi Song committed
9
10
11
1. Inherit the base Tuner class
1. Implement receive_trial_result and generate_parameter function
1. Configure your customized tuner in experiment YAML config file
Yan Ni's avatar
Yan Ni committed
12
13
14

Here is an example:

Chi Song's avatar
Chi Song committed
15
**1. Inherit the base Tuner class**
Yan Ni's avatar
Yan Ni committed
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

```python
from nni.tuner import Tuner

class CustomizedTuner(Tuner):
    def __init__(self, ...):
        ...
```

**2. Implement receive_trial_result and generate_parameter function**

```python
from nni.tuner import Tuner

class CustomizedTuner(Tuner):
    def __init__(self, ...):
        ...

34
    def receive_trial_result(self, parameter_id, parameters, value, **kwargs):
Yan Ni's avatar
Yan Ni committed
35
    '''
Chi Song's avatar
Chi Song committed
36
    Receive trial's final result.
Yan Ni's avatar
Yan Ni committed
37
38
    parameter_id: int
    parameters: object created by 'generate_parameters()'
Chi Song's avatar
Chi Song committed
39
    value: final metrics of the trial, including default metric
Yan Ni's avatar
Yan Ni committed
40
41
42
43
    '''
    # your code implements here.
    ...

44
    def generate_parameters(self, parameter_id, **kwargs):
Yan Ni's avatar
Yan Ni committed
45
46
47
48
49
50
51
52
53
    '''
    Returns a set of trial (hyper-)parameters, as a serializable object
    parameter_id: int
    '''
    # your code implements here.
    return your_parameters
    ...
```

54
`receive_trial_result` will receive the `parameter_id, parameters, value` as parameters input. Also, Tuner will receive the `value` object are exactly same value that Trial send. If `multiPhase` is set to `true` in the experiment configuration file, an additional `trial_job_id` parameter is passed to `receive_trial_result` and `generate_parameters` through the `**kwargs` parameter.
Yan Ni's avatar
Yan Ni committed
55
56
57
58
59
60
61

The `your_parameters` return from `generate_parameters` function, will be package as json object by NNI SDK. NNI SDK will unpack json object so the Trial will receive the exact same `your_parameters` from Tuner.

For example:
If the you implement the `generate_parameters` like this:

```python
62
def generate_parameters(self, parameter_id, **kwargs):
Yan Ni's avatar
Yan Ni committed
63
64
65
66
67
68
69
70
71
72
73
    '''
    Returns a set of trial (hyper-)parameters, as a serializable object
    parameter_id: int
    '''
    # your code implements here.
    return {"dropout": 0.3, "learning_rate": 0.4}

```

It means your Tuner will always generate parameters `{"dropout": 0.3, "learning_rate": 0.4}`. Then Trial will receive `{"dropout": 0.3, "learning_rate": 0.4}` by calling API `nni.get_next_parameter()`. Once the trial ends with a result (normally some kind of metrics), it can send the result to Tuner by calling API `nni.report_final_result()`, for example `nni.report_final_result(0.93)`. Then your Tuner's `receive_trial_result` function will receied the result like:

Chi Song's avatar
Chi Song committed
74
```python
Yan Ni's avatar
Yan Ni committed
75
76
77
78
79
80
81
parameter_id = 82347
parameters = {"dropout": 0.3, "learning_rate": 0.4}
value = 0.93
```

**Note that** if you want to access a file (e.g., `data.txt`) in the directory of your own tuner, you cannot use `open('data.txt', 'r')`. Instead, you should use the following:

Chi Song's avatar
Chi Song committed
82
```python
Yan Ni's avatar
Yan Ni committed
83
84
85
86
87
88
_pwd = os.path.dirname(__file__)
_fd = open(os.path.join(_pwd, 'data.txt'), 'r')
```

This is because your tuner is not executed in the directory of your tuner (i.e., `pwd` is not the directory of your own tuner).

89
**3. Configure your customized tuner in experiment YAML config file**
Yan Ni's avatar
Yan Ni committed
90
91
92

NNI needs to locate your customized tuner class and instantiate the class, so you need to specify the location of the customized tuner class and pass literal values as parameters to the \_\_init__ constructor.

Yan Ni's avatar
Yan Ni committed
93
```yaml
Yan Ni's avatar
Yan Ni committed
94
95
96
97
98
tuner:
  codeDir: /home/abc/mytuner
  classFileName: my_customized_tuner.py
  className: CustomizedTuner
  # Any parameter need to pass to your tuner class __init__ constructor
99
  # can be specified in this optional classArgs field, for example
Yan Ni's avatar
Yan Ni committed
100
101
102
103
104
105
  classArgs:
    arg1: value1

```

More detail example you could see:
Yan Ni's avatar
Yan Ni committed
106
107
108
> * [evolution-tuner](https://github.com/Microsoft/nni/tree/master/src/sdk/pynni/nni/evolution_tuner)
> * [hyperopt-tuner](https://github.com/Microsoft/nni/tree/master/src/sdk/pynni/nni/hyperopt_tuner)
> * [evolution-based-customized-tuner](https://github.com/Microsoft/nni/tree/master/examples/tuners/ga_customer_tuner)
Yan Ni's avatar
Yan Ni committed
109
110
111

### Write a more advanced automl algorithm

112
The methods above are usually enough to write a general tuner. However, users may also want more methods, for example, intermediate results, trials' state (e.g., the methods in assessor), in order to have a more powerful automl algorithm. Therefore, we have another concept called `advisor` which directly inherits from `MsgDispatcherBase` in [`src/sdk/pynni/nni/msg_dispatcher_base.py`](https://github.com/Microsoft/nni/tree/master/src/sdk/pynni/nni/msg_dispatcher_base.py). Please refer to [here](CustomizeAdvisor.md) for how to write a customized advisor.