Skip to content

GitLab

Unverified Commit e1eefa4c authored by Wentao Ye's avatar Wentao Ye Committed by GitHub
Browse files

[Bug] Fix torch warning of tf32 usage (#29112) 


Signed-off-by: default avataryewentao256 <zhyanwentao@126.com>
parent ed6ae1e3
Show whitespace changes
Inline Side-by-side
Showing with 3 additions and 2 deletions
vllm/model_executor/layers/batch_invariant.py
View file @ e1eefa4c
......@@ -852,5 +852,6 @@ def init_batch_invariance():
enable_batch_invariant_mode()
# Disable TF32 for batch invariance - it causes non-deterministic rounding
torch.backends.cuda.matmul.allow_tf32 = False
torch.backends.cudnn.allow_tf32 = False
torch.backends.cuda.matmul.fp32_precision = "ieee"
torch.backends.cudnn.conv.fp32_precision = "ieee"
torch.backends.cudnn.rnn.fp32_precision = "ieee"
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment