Skip to content

GitLab

Unverified Commit 8b9241be authored by Cade Daniel's avatar Cade Daniel Committed by GitHub
Browse files

[Speculative decoding] [Bugfix] Fix overallocation in ngram + spec logprobs (#4672)

parent f942efb5
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
vllm/spec_decode/ngram_worker.py
View file @ 8b9241be
...@@ -138,7 +138,7 @@ class NGramWorker(LoraNotSupportedWorkerBase): ...@@ -138,7 +138,7 @@ class NGramWorker(LoraNotSupportedWorkerBase):
SamplerOutput( SamplerOutput(
outputs=None, outputs=None,
sampled_token_probs=token_probs[i], sampled_token_probs=token_probs[i],
logprobs=token_logprobs, logprobs=token_logprobs[i],
sampled_token_ids=token_ids[i], sampled_token_ids=token_ids[i],
)) ))
return outputs, False return outputs, False
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment