Unverified Commit 532a6cfc authored by Russell Bryant's avatar Russell Bryant Committed by GitHub
Browse files

[ux] Switch a warning to debug about a pytorch fallback (#23750)


Signed-off-by: default avatarRussell Bryant <rbryant@redhat.com>
parent eb32335e
......@@ -109,9 +109,9 @@ class TopKTopPSampler(nn.Module):
# CPU-GPU synchronization while `flashinfer_sample` does.
if (k is None and p is None) or generators:
if generators:
logger.warning_once("FlashInfer 0.2.3+ does not support "
"per-request generators. Falling back to "
"PyTorch-native implementation.")
logger.debug_once("FlashInfer 0.2.3+ does not support "
"per-request generators. Falling back to "
"PyTorch-native implementation.")
return self.forward_native(logits, generators, k, p)
assert self.logprobs_mode not in (
"processed_logits", "processed_logprobs"
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment