Unverified Commit 13b89bd8 authored by cjackal's avatar cjackal Committed by GitHub
Browse files

[doc] update `vllm serve` cli args documentation (#24329)


Signed-off-by: default avatarcjackal <44624812+cjackal@users.noreply.github.com>
parent 22a00705
......@@ -134,14 +134,13 @@ schema. Example: `[{"type": "text", "text": "Hello world!"}]`"""
"""If specified, will run the OpenAI frontend server in the same process as
the model serving engine."""
enable_request_id_headers: bool = False
"""If specified, API server will add X-Request-Id header to responses.
Caution: this hurts performance at high QPS."""
"""If specified, API server will add X-Request-Id header to responses."""
enable_auto_tool_choice: bool = False
"""If specified, exclude tool definitions in prompts when
tool_choice='none'."""
exclude_tools_when_tool_choice_none: bool = False
"""Enable auto tool choice for supported models. Use `--tool-call-parser`
to specify which parser to use."""
exclude_tools_when_tool_choice_none: bool = False
"""If specified, exclude tool definitions in prompts when
tool_choice='none'."""
tool_call_parser: Optional[str] = None
"""Select the tool call parser depending on the model that you're using.
This is used to parse the model-generated tool call into OpenAI API format.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment