Skip to content

GitLab

Unverified Commit 65f2de5f authored by Tanmay Verma's avatar Tanmay Verma Committed by GitHub
Browse files

fix: Fix sample disagg config for trtllm standalone (#1566)

parent ac4fd87b
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
launch/dynamo-run/src/subprocess/trtllm_config/sample.yaml
View file @ 65f2de5f
...@@ -20,5 +20,6 @@ ...@@ -20,5 +20,6 @@
# You might have to tweak this config based on your model size and GPU memory. # You might have to tweak this config based on your model size and GPU memory.
backend: pytorch backend: pytorch
disable_overlap_scheduler: true
kv_cache_config: kv_cache_config:
free_gpu_memory_fraction: 0.40 free_gpu_memory_fraction: 0.40
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment