Skip to content

GitLab

"benchmark/git@developer.sourcefind.cn:change/sglang.git" did not exist on "ce832d7034e85a71ce6f7a211ecc7569d9d6d086"
Unverified Commit 1ac4ccf7 authored by Woosuk Kwon's avatar Woosuk Kwon Committed by GitHub
Browse files

Add float16 and float32 (#1115)

parent 2ac4d5e2
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 1 deletion
vllm/engine/arg_utils.py
View file @ 1ac4ccf7
...@@ -93,7 +93,9 @@ class EngineArgs: ...@@ -93,7 +93,9 @@ class EngineArgs:
'--dtype', '--dtype',
type=str, type=str,
default=EngineArgs.dtype, default=EngineArgs.dtype,
choices=['auto', 'half', 'bfloat16', 'float'], choices=[
'auto', 'half', 'float16', 'bfloat16', 'float', 'float32'
],
help='data type for model weights and activations. ' help='data type for model weights and activations. '
'The "auto" option will use FP16 precision ' 'The "auto" option will use FP16 precision '
'for FP32 and FP16 models, and BF16 precision ' 'for FP32 and FP16 models, and BF16 precision '
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment