Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
d1211f87
Unverified
Commit
d1211f87
authored
May 19, 2025
by
Robin
Committed by
GitHub
May 18, 2025
Browse files
[Doc] Add doc to explain the usage of Qwen3 thinking (#18291)
Signed-off-by:
WangErXiao
<
863579016@qq.com
>
parent
b6a6e7a5
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
0 deletions
+5
-0
docs/source/features/reasoning_outputs.md
docs/source/features/reasoning_outputs.md
+5
-0
No files found.
docs/source/features/reasoning_outputs.md
View file @
d1211f87
...
...
@@ -19,6 +19,7 @@ vLLM currently supports the following reasoning models:
:::{note}
IBM Granite 3.2 reasoning is disabled by default; to enable it, you must also pass
`thinking=True`
in your
`chat_template_kwargs`
.
The reasoning feature for the Qwen3 series is enabled by default. To disable it, you must pass
`enable_thinking=False`
in your
`chat_template_kwargs`
.
:::
## Quickstart
...
...
@@ -49,6 +50,8 @@ model = models.data[0].id
# Round 1
messages
=
[{
"role"
:
"user"
,
"content"
:
"9.11 and 9.8, which is greater?"
}]
# For granite, add: `extra_body={"chat_template_kwargs": {"thinking": True}}`
# For Qwen3 series, if you want to disable thinking in reasoning mode, add:
# extra_body={"chat_template_kwargs": {"enable_thinking": False}}
response
=
client
.
chat
.
completions
.
create
(
model
=
model
,
messages
=
messages
)
reasoning_content
=
response
.
choices
[
0
].
message
.
reasoning_content
...
...
@@ -104,6 +107,8 @@ model = models.data[0].id
messages
=
[{
"role"
:
"user"
,
"content"
:
"9.11 and 9.8, which is greater?"
}]
# For granite, add: `extra_body={"chat_template_kwargs": {"thinking": True}}`
# For Qwen3 series, if you want to disable thinking in reasoning mode, add:
# extra_body={"chat_template_kwargs": {"enable_thinking": False}}
stream
=
client
.
chat
.
completions
.
create
(
model
=
model
,
messages
=
messages
,
stream
=
True
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment