Unverified Commit e3f15ee1 authored by jthomson04's avatar jthomson04 Committed by GitHub
Browse files

fix: Little kv routing fix (#1677)

parent 8392e7a1
......@@ -100,8 +100,7 @@ def parse_args(service_name, prefix) -> Namespace:
)
parser.add_argument(
"--softmax-sample",
type=bool,
default=False,
action="store_true",
help="Whether to do softmax sampling based on worker logits (default is to pick smallest)",
)
config = ServiceConfig.get_instance()
......
......@@ -29,7 +29,7 @@ Processor:
Router:
min-workers: 1
softmax_sample: true
softmax-sample: true
common-configs: [model, block-size, router]
VllmWorker:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment