Unverified Commit 60ba63bb authored by Yifan Xiong's avatar Yifan Xiong Committed by GitHub
Browse files

CLI - Support host-list for deploy and run commands (#108)

Support `--host-list` for deploy and run commands.

Before this change, an inventory file is needed to use `sb deploy/run`.
Now, `--host-list localhost` or `-l localhost` is sufficient for quick try.
parent 7b0b0e9a
......@@ -61,9 +61,14 @@ sb deploy [--docker-image]
#### Examples
Deploy image `superbench/cuda:11.1` to all nodes in `./host.yaml`:
Deploy default image on local GPU node:
```bash title="SB CLI"
sb deploy --docker-image superbench/cuda:11.1 --host-file ./host.yaml
sb deploy --host-list localhost
```
Deploy image `superbench/cuda:11.1` to all nodes in `./host.ini`:
```bash title="SB CLI"
sb deploy --docker-image superbench/cuda:11.1 --host-file ./host.ini
```
### `sb exec`
......@@ -137,10 +142,15 @@ sb run [--config-file]
#### Examples
Run all benchmarks on all managed nodes in `./host.yaml` using image `superbench/cuda:11.1`
Run all benchmarks on local GPU node:
```bash title="SB CLI"
sb run --host-list localhost
```
Run all benchmarks on all managed nodes in `./host.ini` using image `superbench/cuda:11.1`
and default benchmarking configuration:
```bash title="SB CLI"
sb run --docker-image superbench/cuda:11.1 --host-file ./host.yaml
sb run --docker-image superbench/cuda:11.1 --host-file ./host.ini
```
### `sb version`
......
......@@ -32,8 +32,10 @@
type: command
short-summary: Deploy the SuperBench environments to all given nodes.
examples:
- name: deploy image "superbench/cuda:11.1" to all nodes in ./host.yaml
text: {cli_name} deploy --docker-image superbench/cuda:11.1 --host-file ./host.yaml
- name: deploy default image on local GPU node
text: {cli_name} deploy --host-list localhost
- name: deploy image "superbench/cuda:11.1" to all nodes in ./host.ini
text: {cli_name} deploy --docker-image superbench/cuda:11.1 --host-file ./host.ini
- name: deploy image "superbench/rocm:4.0" to node-0 and node-2, using key file id_rsa for ssh
text: {cli_name} deploy --docker-image superbench/rocm:4.0 --host-list node-0,node-2 --private-key id_rsa
""".format(cli_name=CLI_NAME)
......@@ -52,9 +54,11 @@
type: command
short-summary: Run the SuperBench benchmarks distributedly.
examples:
- name: run all benchmarks on all nodes in ./host.yaml using image "superbench/cuda:11.1"
- name: run all benchmarks on local GPU node
text: {cli_name} run --host-list localhost
- name: run all benchmarks on all nodes in ./host.ini using image "superbench/cuda:11.1"
and default benchmarking configuration
text: {cli_name} run --docker-image superbench/cuda:11.1 --host-file ./host.yaml
text: {cli_name} run --docker-image superbench/cuda:11.1 --host-file ./host.ini
""".format(cli_name=CLI_NAME)
......
......@@ -29,13 +29,18 @@ def __init__(self, config):
}
if config:
inventory_file = getattr(config, 'host_file', None)
if inventory_file:
self._config['inventory'] = inventory_file
inventory_list = getattr(config, 'host_list', None)
if inventory_list:
inventory_list = inventory_list.strip(',')
if inventory_file or inventory_list:
self._config['inventory'] = inventory_file or inventory_list
self._config['host_pattern'] = 'all'
inventory = InventoryManager(loader=DataLoader(), sources=inventory_file)
inventory = InventoryManager(loader=DataLoader(), sources=inventory_file or f'{inventory_list},')
host_list = inventory.get_groups_dict()['all']
if len(host_list) > 0:
self._config['cmdline'] = '--forks {}'.format(len(host_list))
if inventory_list in ['localhost', '127.0.0.1']:
self._config['cmdline'] += ' --connection local'
username = getattr(config, 'host_username', None)
if username:
self._config['cmdline'] += ' --user {}'.format(username)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment