Commit 3983830e authored by Maksim Khadkevich's avatar Maksim Khadkevich Committed by GitHub
Browse files

fix: created documentation to deploy_to_k8s_using_helm (#245)


Co-authored-by: default avatarMeenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>
parent 93a46969
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
replicaCount: 1
# Explicitly remove authentication
auth:
rbac:
create: false
readinessProbe:
enabled: false
livenessProbe:
enabled: false
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
config:
jetstream:
enabled: true
fileStore:
enabled: true
dir: /data
pvc:
enabled: true
size: 10Gi
storageClassName:
......@@ -32,7 +32,7 @@ def create_bentoml_cli() -> click.Command:
from dynamo.sdk.cli.serve import serve_command
# from dynamo.sdk.cli.server import cloud_command
# from dynamo.sdk.cli.start import start_command
from dynamo.sdk.cli.start import start_command
from dynamo.sdk.cli.utils import DynamoCommandGroup
server_context.service_type = "cli"
......@@ -54,7 +54,8 @@ def create_bentoml_cli() -> click.Command:
# Add top-level CLI commands
# bentoml_cli.add_command(cloud_command)
bentoml_cli.add_single_command(bento_command, "build")
# bentoml_cli.add_subcommands(start_command)
bentoml_cli.add_subcommands(start_command)
bentoml_cli.add_single_command(bento_command, "get")
bentoml_cli.add_subcommands(serve_command)
bentoml_cli.add_subcommands(run_command)
# bentoml_cli.add_command(containerize_command)
......
......@@ -157,6 +157,7 @@ def build_start_command() -> click.Group:
help="Print the final service configuration and exit without starting the server",
default=False,
)
@click.pass_context
@add_experimental_docstring
def start(
ctx: click.Context,
......
First install microk8s:
```bash
sudo snap install microk8s --classic
```
add user permissions
```bash
sudo usermod -a -G microk8s $USER
sudo chown -R $USER ~/.kube
```
Log out and log in
start microk8s
```
microk8s start
```
Add GPU support
```
microk8s enable gpu
```
Add storage support (follow https://microk8s.io/docs/addon-hostpath-storage)
```
microk8s enable storage
```
Kube config
```
mkdir -p ~/.kube && microk8s config >> ~/.kube/config
```
Now one can use `kubectl` command.
2. create a namespace
```
export NAMESPACE=dynamo-playground
export RELEASE_NAME=dynamo-platform
#install nats and etcd
cd deploy/Kubernetes/pipeline/dependencies
#install nats
helm repo add nats https://nats-io.github.io/k8s/helm/charts/
helm repo update
helm install --namespace ${NAMESPACE} dynamo-platform-nats nats/nats --create-namespace --values nats-values.yaml
#install etcd
helm install --namespace ${NAMESPACE} dynamo-platform-etcd oci://registry-1.docker.io/bitnamicharts/etcd --values etcd-values.yaml
```
Now let's containerize a hello world pipeline:
1. Build container image for `container/Dockerfile.vllm`
2. Containerize hello world pipeline
```
cd examples/hello_world
export DYNAMO_IMAGE=<dynamo_runtime_image_name>
dynamo build --containerize hello_world:Frontend
```
Once the container is built, it has to be tagged and pushed to container registry:
```
docker tag <BUILT_IMAGE_TAG> <TAG>
docker push <TAG>
```
Now one can deploy the pipeline onto k8s using helm
- get values.yaml for helm chart:
- install chart
```
export HELM_RELEASE=helloworld
dynamo get frontend > pipeline-values.yaml
helm upgrade -i "$HELM_RELEASE" ./chart -f pipeline-values.yaml --set image=<TAG> --set dynamoIdentifier="hello_world:Frontend" -n "$NAMESPACE"
```
Once the deployments are running, one can port-forward to localhost and make API calls to the frontend component:
```
kubectl -n ${NAMESPACE} port-forward svc/helloworld-frontend 3000:80
curl -X 'POST' 'http://localhost:3000/generate' -H 'accept: text/event-stream' -H 'Content-Type: application/json' -d '{"text": "test"}'
```
Full script to build a container, push it to registry and deploya helm chart: `deploy/Kubernetes/pipeline/deploy.sh`
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment