Skip to content

GitLab

Unverified Commit d88c8666 authored by Michael Goin's avatar Michael Goin Committed by GitHub
Browse files

[Bugfix][Example] Fix GCed profiling server for TPU (#12792) 


Signed-off-by: default avatarmgoin <michael@neuralmagic.com>
parent 4fc5c23b
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
examples/offline_inference/profiling_tpu/profiling.py
View file @ d88c8666
......@@ -24,7 +24,7 @@ def main(args: argparse.Namespace):
engine_args = EngineArgs.from_cli_args(args)
llm = LLM(**dataclasses.asdict(engine_args))
_ = xp.start_server(9012)
server = xp.start_server(9012) # noqa: F841
sampling_params = SamplingParams(
temperature=0.0,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment