README.md 2.71 KB
Newer Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
# Dynamo CLI Documentation
The Dynamo CLI is a powerful tool for serving, containerizing, and deploying Dynamo applications. It leverages core pieces of the BentoML deployment stack and provides a range of commands to manage your Dynamo services.

Overview
At a high level, the Dynamo CLI allows you to:
- `run` - quickly chat with a model
- `serve` - run a set of services locally (via `depends()` or `.link()`)
- `build` - create an archive of your services (called a `bento`)

# Commands

## `run`

The `run` command allows you to quickly chat with a model. Under the hood - it is running the `dynamo-run` Rust binary. You can find the arguments that it takes here: [dynamo-run docs](../../../../../launch/README.md)

**Example**
```bash
dynamo run deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
```

## `serve`

The `serve` command lets you run a defined inference graph locally. You must point toward your file and intended class using file:Class syntax

**Usage**
```bash
dynamo serve [SERVICE]
```

**Arguments**
- `SERVICE` - The service to start. You use file:Class syntax to specify the service.

**Flags**
- `--file`/`-f` - Path to optional YAML configuration file. An example of the YAML file can be found in the configuration section of the [SDK docs](../sdk/README.md)
- `--dry-run` - Print out the dependency graph and values without starting any services.
36
- `--service-name` - Only serve the specified service name. The rest of the discoverable components in the graph are not started.
37
38
39
40
41
42
- `--working-dir` - Specify the directory to find the Service instance
- Any additional flags that follow Class.key=value will be passed to the service constructor for the target service and parsed. Please see the configuration section of the [SDK docs](../sdk/README.md) for more details.

**Example**
```bash
cd examples
43
# Spin up Frontend, Middle, and Backend components
44
dynamo serve hello_world:Frontend
45
46
47

# Spin up only the Middle component in the graph that is discoverable from the Frontend service
dynamo serve  --service-name Middle hello_world:Frontend
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
```

## `build`

The `build` commmand allows you to package up your inference graph and its dependancies and create an archive of it. This is commonly paired with the `--containerize` flag to create a single docker container that runs your inference graph. As with `serve`, you point toward the first service in your dependency graph.

**Usage**
```bash
dynamo build [SERVICE]
```

**Arguments**
- `SERVICE` - The service to build. You use file:Class syntax to specify the service.

**Flags**
- `--working-dir` - Specify the directory to find the Service instance
- `--containerize` - Whether to containerize the Bento after building

**Example**
```bash
68
cd examples/hello_world
69
70
dynamo build hello_world:Frontend
```