# 🚀 Dynamo Cloud Kubernetes Platform (Dynamo Deploy) The Dynamo Cloud platform is a comprehensive solution for deploying and managing Dynamo inference graphs (also referred to as pipelines) in Kubernetes environments. It provides a streamlined experience for deploying, scaling, and monitoring your inference services. You can interface with Dynamo Cloud using the `deploy` subcommand available in the Dynamo CLI (e.g `dynamo deploy`) ## 📋 Overview The Dynamo cloud platform consists of several key components: - **Dynamo Operator**: A Kubernetes operator that manages the lifecycle of Dynamo inference graphs from build ➡️ deploy. - **API Store**: Stores and manages service configurations and metadata related to Dynamo deployments. Needs to be exposed externally. - **Custom Resources**: Kubernetes custom resources for defining and managing Dynamo services These components work together to provide a seamless deployment experience, handling everything from containerization to scaling and monitoring.  ## 🎯 Prerequisites Before getting started with the Dynamo cloud platform, ensure you have: - A Kubernetes cluster (version 1.24 or later) - [Earthly](https://earthly.dev/) installed for building components - Docker installed and running - Access to a container registry (e.g., Docker Hub, NVIDIA NGC, etc.) - `kubectl` configured to access your cluster - Helm installed (version 3.0 or later) > [!TIP] > Don't have a Kubernetes cluster? Check out our [Minikube setup guide](./minikube.md) to set up a local environment! 🏠 ## 🏗️ Building Docker Images for Dynamo Cloud Components The Dynamo cloud platform components need to be built and pushed to a container registry before deployment. You can build these components individually or all at once. ### ⚙️ Setting Up Environment Variables First, set the required environment variables for building and pushing images: ```bash # Set your container registry export DOCKER_SERVER= # Set the image tag (e.g., latest, 0.0.1, etc.) export IMAGE_TAG= ``` Where: - ``: Your container registry (e.g., `nvcr.io`, `docker.io/`, etc.) - ``: The version tag for your images (e.g., `latest`, `0.0.1`, `v1.0.0`) > [!IMPORTANT] > Make sure you're logged in to your container registry before pushing images: > ```bash > docker login > ``` ### 🛠️ Building Components You can build and push all platform components at once: ```bash earthly --push +all-docker --DOCKER_SERVER=$DOCKER_SERVER --IMAGE_TAG=$IMAGE_TAG ``` ## 🚀 Deploying the Dynamo Cloud Platform Once you've built and pushed the components, you can deploy the platform to your Kubernetes cluster. ### 📋 Prerequisites Before deploying Dynamo Cloud, ensure your Kubernetes cluster meets the following requirements: #### 1. 🛡️ Istio Installation Dynamo Cloud requires Istio for service mesh capabilities. Verify Istio is installed and running: ```bash # Check if Istio is installed kubectl get pods -n istio-system # Expected output should show running Istio pods # istiod-* pods should be in Running state ``` #### 2. 💾 PVC Support with Default Storage Class Dynamo Cloud requires Persistent Volume Claim (PVC) support with a default storage class. Verify your cluster configuration: ```bash # Check if default storage class exists kubectl get storageclass # Expected output should show at least one storage class marked as (default) # Example: # NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE # standard (default) kubernetes.io/gce-pd Delete Immediate true 1d ``` ### 📥 Installation 1. Set the required environment variables: ```bash export DOCKER_USERNAME= export DOCKER_PASSWORD= export DOCKER_SERVER= export IMAGE_TAG= # Use the same tag you used when building the images export NAMESPACE=dynamo-cloud # change this to whatever you want! ``` > [!NOTE] > DOCKER_USERNAME and DOCKER_PASSWORD are optional and only needed if you want to pull docker images from a private registry. > A docker image pull secret will be created automatically if these variables are set. Its name will be `docker-imagepullsecret` unless overridden by the `DOCKER_SECRET_NAME` environment variable. The Dynamo Cloud Platform auto-generates docker images for pipelines and pushes them to a container registry. By default, the platform will use the same container registry as the platform components (specified by `DOCKER_SERVER`). However, you can specify a different container registry for pipelines by additionally setting the following environment variables: ```bash export PIPELINES_DOCKER_SERVER= export PIPELINES_DOCKER_USERNAME= export PIPELINES_DOCKER_PASSWORD= ``` 2. [One-time Action] Create a new kubernetes namespace and set it as your default. ```bash cd deploy/dynamo/helm kubectl create namespace $NAMESPACE kubectl config set-context --current --namespace=$NAMESPACE ``` 3. Deploy the helm chart using the deploy script: ```bash ./deploy.sh ``` if you wish to be guided through the deployment process, you can run the deploy script with the `--interactive` flag: ```bash ./deploy.sh --interactive ``` 4. 🌐 **Expose Dynamo Cloud Externally** > [!NOTE] > The script will automatically display information about the endpoint you can use to access Dynamo Cloud. In our docs, we refer to this externally available endpoint as `DYNAMO_CLOUD`. The simplest way to expose the `dynamo-store` service within the namespace externally is to use a port-forward: ```bash kubectl port-forward svc/dynamo-store :80 -n $NAMESPACE export DYNAMO_CLOUD=http://localhost: ``` ## 🎯 Next Steps After deploying the Dynamo cloud platform, you can: 1. Deploy your first inference graph using the [Dynamo CLI](operator_deployment.md) 2. Deploy Dynamo LLM pipelines to Kubernetes using the [Dynamo CLI](../../../examples/llm/README.md)! 3. Manage your deployments using the Dynamo CLI For more detailed information about deploying inference graphs, see the [Dynamo Deploy Guide](README.md).