"vscode:/vscode.git/clone" did not exist on "e0fece2b2649d0044958c76e2c7bc701ba182a2e"
Unverified Commit 34c88987 authored by Ying Sheng's avatar Ying Sheng Committed by GitHub
Browse files

Check eagle server args (#4217)

parent 0dd6cda2
...@@ -284,9 +284,13 @@ class ServerArgs: ...@@ -284,9 +284,13 @@ class ServerArgs:
"Overlap scheduler are disabled because of using " "Overlap scheduler are disabled because of using "
"eagle speculative decoding." "eagle speculative decoding."
) )
# The token generated from the verify step is counted. # The token generated from the verify step is counted in speculative_num_draft_tokens.
# If sepculative_num_steps >= speculative_num_draft_tokens, the additional tokens will definitely be discarded. # If sepculative_num_steps >= speculative_num_draft_tokens, the additional tokens will definitely be discarded.
# assert self.speculative_num_steps < self.speculative_num_draft_tokens assert self.speculative_num_steps < self.speculative_num_draft_tokens
assert (
self.speculative_num_draft_tokens - 1
<= self.speculative_num_steps * self.speculative_eagle_topk
)
# GGUF # GGUF
if ( if (
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment