Unverified Commit 4c7b4242 authored by fzyzcjy's avatar fzyzcjy Committed by GitHub
Browse files

Hint users DeepEP normal mode is incompatible with CUDA Graph (#5014)

parent 38053c33
......@@ -323,6 +323,9 @@ class ServerArgs:
assert (
not self.enable_dp_attention
), "DeepEP MoE `auto` mode is not supported with DP Attention."
if self.deepep_mode == "normal":
logger.warning("Cuda graph is disabled because deepep_mode=`normal`")
self.disable_cuda_graph = True
self.ep_size = self.tp_size
self.enable_sp_layernorm = (
self.dp_size < self.tp_size if self.enable_dp_attention else True
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment