Skip to content

GitLab

"tools/vscode:/vscode.git/clone" did not exist on "286ca71e1a521ef5f24b3956227f1398140188a8"
Unverified Commit 28b0a62b authored by zifeitong's avatar zifeitong Committed by GitHub
Browse files

Bug: Fix min_p sampling crash when using flashinfer backend (#3207) 


Co-authored-by: default avatarzhaochenyang20 <zhaochen20@outlook.com>
parent 566d61d9
Show whitespace changes
Inline Side-by-side
Showing with 4 additions and 4 deletions
python/sglang/srt/layers/sampler.py
View file @ 28b0a62b
...@@ -85,7 +85,7 @@ class Sampler(nn.Module): ...@@ -85,7 +85,7 @@ class Sampler(nn.Module):
if sampling_info.need_min_p_sampling: if sampling_info.need_min_p_sampling:
probs = top_k_renorm_prob(probs, sampling_info.top_ks) probs = top_k_renorm_prob(probs, sampling_info.top_ks)
probs = top_p_renorm_prob(probs, sampling_info.top_ps) probs = top_p_renorm_prob(probs, sampling_info.top_ps)
batch_next_token_ids, success = min_p_sampling_from_probs( batch_next_token_ids = min_p_sampling_from_probs(
probs, uniform_samples, sampling_info.min_ps probs, uniform_samples, sampling_info.min_ps
) )
else: else:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment