Unverified Commit 86a1f67a authored by Teruaki Ishizaki's avatar Teruaki Ishizaki Committed by GitHub
Browse files

[Bugfix][Benchmarks] Allow benchmark of deepspeed-mii backend to select a model (#17285)


Signed-off-by: default avatarTeruaki Ishizaki <teruaki.ishizaki@ntt.com>
parent a257d9bc
...@@ -201,6 +201,7 @@ async def async_request_deepspeed_mii( ...@@ -201,6 +201,7 @@ async def async_request_deepspeed_mii(
timeout=AIOHTTP_TIMEOUT) as session: timeout=AIOHTTP_TIMEOUT) as session:
payload = { payload = {
"model": request_func_input.model,
"prompt": request_func_input.prompt, "prompt": request_func_input.prompt,
"max_tokens": request_func_input.output_len, "max_tokens": request_func_input.output_len,
"temperature": 0.01, # deepspeed-mii does not accept 0.0 temp. "temperature": 0.01, # deepspeed-mii does not accept 0.0 temp.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment