Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
34c88987
"vscode:/vscode.git/clone" did not exist on "e0fece2b2649d0044958c76e2c7bc701ba182a2e"
Unverified
Commit
34c88987
authored
Mar 09, 2025
by
Ying Sheng
Committed by
GitHub
Mar 09, 2025
Browse files
Check eagle server args (#4217)
parent
0dd6cda2
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
2 deletions
+6
-2
python/sglang/srt/server_args.py
python/sglang/srt/server_args.py
+6
-2
No files found.
python/sglang/srt/server_args.py
View file @
34c88987
...
@@ -284,9 +284,13 @@ class ServerArgs:
...
@@ -284,9 +284,13 @@ class ServerArgs:
"Overlap scheduler are disabled because of using "
"Overlap scheduler are disabled because of using "
"eagle speculative decoding."
"eagle speculative decoding."
)
)
# The token generated from the verify step is counted.
# The token generated from the verify step is counted
in speculative_num_draft_tokens
.
# If sepculative_num_steps >= speculative_num_draft_tokens, the additional tokens will definitely be discarded.
# If sepculative_num_steps >= speculative_num_draft_tokens, the additional tokens will definitely be discarded.
# assert self.speculative_num_steps < self.speculative_num_draft_tokens
assert
self
.
speculative_num_steps
<
self
.
speculative_num_draft_tokens
assert
(
self
.
speculative_num_draft_tokens
-
1
<=
self
.
speculative_num_steps
*
self
.
speculative_eagle_topk
)
# GGUF
# GGUF
if
(
if
(
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment