Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
autodeploy_engine_config.yaml 304 Bytes
Newer Older
Tushar Sharma's avatar
chore: update all copyright headers in repo to 2026 (#5130)  
Tushar Sharma committed
1 
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
William Zhang's avatar
feat: Enable autodeploy as a backend for TRT-LLM (#4347)  
William Zhang committed
2
3
4
5
6
7
8 
# SPDX-License-Identifier: Apache-2.0

backend: _autodeploy
kv_cache_config:
  enable_partial_reuse: false
  free_gpu_memory_fraction: 0.80
  max_tokens: 8192
Yuewei Na's avatar
fix(test): unskip test_smoke by switching to trtllm attention backend (DYN-2607) (#8104)  
Yuewei Na committed
9
10 
attn_backend: trtllm
max_batch_size: 128