Skip to content

GitLab

Unverified Commit d83633b5 authored by Ryan McCormick's avatar Ryan McCormick Committed by GitHub
Browse files

chore: Default to pytorch backend in trtllm worker (#1445)

parent 75d7c3b9
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
launch/dynamo-run/src/subprocess/trtllm_inc.py
View file @ d83633b5
......@@ -352,6 +352,7 @@ async def init(runtime: DistributedRuntime, config: Config):
arg_map = {
"model": model_path,
"tensor_parallel_size": config.tensor_parallel_size,
"backend": "pytorch",
"skip_tokenizer_init": True,
"disable_log_requests": True,
"enable_prefix_caching": True,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment