Skip to content

GitLab

Unverified Commit 3529c061 authored by Liangsheng Yin's avatar Liangsheng Yin Committed by GitHub
Browse files

[spec v2] Fix output repetition by speculative sampling error (#12561)

parent ffb32a85
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 3 deletions
python/sglang/srt/speculative/eagle_info_v2.py
View file @ 3529c061
...@@ -290,9 +290,7 @@ class EagleVerifyInputV2Mixin: ...@@ -290,9 +290,7 @@ class EagleVerifyInputV2Mixin:
), ),
) )
target_probs = target_probs.reshape(bs, self.draft_token_num, -1) target_probs = target_probs.reshape(bs, self.draft_token_num, -1)
draft_probs = torch.zeros_like(target_probs)
# This is currently not used
draft_probs = torch.empty_like(target_probs)
# coins for rejection sampling # coins for rejection sampling
coins = torch.rand_like(candidates, dtype=torch.float32, device=device) coins = torch.rand_like(candidates, dtype=torch.float32, device=device)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment