"vscode:/vscode.git/clone" did not exist on "29855c2716a99fbb79e0b73dca9c18b650bd8107"
Unverified Commit 25e5d589 authored by Shi Shuai's avatar Shi Shuai Committed by GitHub
Browse files

Doc: Update Grammar Backend (#2545)


Co-authored-by: default avatarChayenne <zhaochen20@outlook.com>
parent 41b1db69
......@@ -221,7 +221,14 @@
"metadata": {},
"source": [
"## Structured decoding (JSON, Regex)\n",
"You can specify a JSON schema or a regular expression to constrain the model output. The model output will be guaranteed to follow the given constraints.\n",
"You can define a JSON schema or regular expression to constrain the model's output. The model output will be guaranteed to follow the given constraints and this depends on the grammar backend.\n",
"\n",
"SGlang has two backends: outlines (default) and Xgrammar. Xgrammar enhances JSON decoding performance but does not support regular expressions. To use Xgrammar, add the `--grammar-backend xgrammar` when launching the server:\n",
"\n",
"```bash\n",
"python -m sglang.launch_server --model-path meta-llama/Meta-Llama-3.1-8B-Instruct \\\n",
"--port 30000 --host 0.0.0.0 --grammar-backend xgrammar\n",
"```\n",
"\n",
"### JSON"
]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment