Unverified Commit 4581fccb authored by atchernych's avatar atchernych Committed by GitHub
Browse files

bug: In non-epp dynamo integration we do not provide the config file (#2862)


Signed-off-by: default avatarAnna Tchernych <atchernych@nvidia.com>
parent eb1c9f24
...@@ -191,6 +191,7 @@ NAME HOSTNAMES AGE ...@@ -191,6 +191,7 @@ NAME HOSTNAMES AGE
qwen-route 33m qwen-route 33m
``` ```
### 6. Usage ### ### 6. Usage ###
The Inference Gateway provides HTTP endpoints for model inference. The Inference Gateway provides HTTP endpoints for model inference.
...@@ -202,10 +203,11 @@ export GATEWAY_URL=<Gateway-URL> ...@@ -202,10 +203,11 @@ export GATEWAY_URL=<Gateway-URL>
To test the gateway in minikube, use the following command: To test the gateway in minikube, use the following command:
a. User minikube tunnel to expose the gateway to the host a. User minikube tunnel to expose the gateway to the host
This requires `sudo` access to the host machine. alternatively, you can use port-forward to expose the gateway to the host as shown in alternateive (b). This requires `sudo` access to the host machine. alternatively, you can use port-forward to expose the gateway to the host as shown in alternative (b).
```bash ```bash
# in first terminal # in first terminal
minikube tunnel ps aux | grep "minikube tunnel" | grep -v grep # make sure minikube tunnel is not already running.
minikube tunnel & # start the tunnel
# in second terminal where you want to send inference requests # in second terminal where you want to send inference requests
GATEWAY_URL=$(kubectl get svc inference-gateway -n my-model -o yaml -o jsonpath='{.spec.clusterIP}') GATEWAY_URL=$(kubectl get svc inference-gateway -n my-model -o yaml -o jsonpath='{.spec.clusterIP}')
...@@ -299,3 +301,11 @@ Sample inference output: ...@@ -299,3 +301,11 @@ Sample inference output:
} }
} }
``` ```
### 7. Deleting the installation ###
If you need to uninstall run:
```bash
kubectl delete dynamoGraphDeployment vllm-agg
helm uninstall dynamo-gaie -n my-model
```
...@@ -59,9 +59,11 @@ spec: ...@@ -59,9 +59,11 @@ spec:
- "9002" - "9002"
- -grpcHealthPort - -grpcHealthPort
- "9003" - "9003"
{{- if .Values.eppAware.enabled }}
- -configFile - -configFile
- "/etc/epp/epp-config-dynamo.yaml" - "/etc/epp/epp-config-dynamo.yaml"
{{- end }} {{- end }}
{{- end }}
{{- if .Values.eppAware.enabled }} {{- if .Values.eppAware.enabled }}
volumeMounts: volumeMounts:
- name: epp-config - name: epp-config
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment