Skip to content

GitLab

Unverified Commit 4215b59a authored by atchernych's avatar atchernych Committed by GitHub
Browse files

fix: add env var for model (#4475) 


Signed-off-by: default avatarAnna Tchernych <atchernych@nvidia.com>
parent f0e1f226
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
recipes/llama-3-70b/vllm/agg/gaie/k8s-manifests/epp/deployment.yaml
View file @ 4215b59a
...@@ -75,6 +75,8 @@ spec: ...@@ -75,6 +75,8 @@ spec:
value: "nats://dynamo-platform-nats.$(PLATFORM_NAMESPACE):4222" # update dynamo-platform to appropriate namespace value: "nats://dynamo-platform-nats.$(PLATFORM_NAMESPACE):4222" # update dynamo-platform to appropriate namespace
- name: DYNAMO_NAMESPACE - name: DYNAMO_NAMESPACE
value: "$(POD_NAMESPACE)-llama3-70b-agg" value: "$(POD_NAMESPACE)-llama3-70b-agg"
- name: DYNAMO_MODEL
value: "llama3-70b-agg"
- name: DYNAMO_KV_BLOCK_SIZE - name: DYNAMO_KV_BLOCK_SIZE
value: "128" # UPDATE to match the --block-size in your deploy.yaml engine command value: "128" # UPDATE to match the --block-size in your deploy.yaml engine command
- name: USE_STREAMING - name: USE_STREAMING
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment