Skip to content

GitLab

Unverified Commit d83633b5 authored by Ryan McCormick's avatar Ryan McCormick Committed by GitHub
Browse files

chore: Default to pytorch backend in trtllm worker (#1445)

parent 75d7c3b9
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
launch/dynamo-run/src/subprocess/trtllm_inc.py
View file @ d83633b5
...@@ -352,6 +352,7 @@ async def init(runtime: DistributedRuntime, config: Config): ...@@ -352,6 +352,7 @@ async def init(runtime: DistributedRuntime, config: Config):
arg_map = { arg_map = {
"model": model_path, "model": model_path,
"tensor_parallel_size": config.tensor_parallel_size, "tensor_parallel_size": config.tensor_parallel_size,
"backend": "pytorch",
"skip_tokenizer_init": True, "skip_tokenizer_init": True,
"disable_log_requests": True, "disable_log_requests": True,
"enable_prefix_caching": True, "enable_prefix_caching": True,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment