Skip to content

GitLab

"lib/runtime/vscode:/vscode.git/clone" did not exist on "25fc732565007abb48972ca012ff0be02a6adbc1"
Unverified Commit 65f2de5f authored by Tanmay Verma's avatar Tanmay Verma Committed by GitHub
Browse files

fix: Fix sample disagg config for trtllm standalone (#1566)

parent ac4fd87b
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
launch/dynamo-run/src/subprocess/trtllm_config/sample.yaml
View file @ 65f2de5f
......@@ -20,5 +20,6 @@
# You might have to tweak this config based on your model size and GPU memory.
backend: pytorch
disable_overlap_scheduler: true
kv_cache_config:
free_gpu_memory_fraction: 0.40
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment