Unverified Commit 487745ff authored by Gregory Shtrasberg's avatar Gregory Shtrasberg Committed by GitHub
Browse files

[ROCm][Bugfix] Only enable +rms_norm based on aiter if not explicitly disabled (#25275)


Signed-off-by: default avatarGregory Shtrasberg <Gregory.Shtrasberg@amd.com>
parent 9313be50
......@@ -340,7 +340,8 @@ class RocmPlatform(Platform):
else:
parallel_config.worker_cls = "vllm.worker.worker.Worker"
# Aiter rms norm perform best when CUDA Graph capture is enabled.
if use_v1 and use_aiter_rms_norm and not is_eager_execution:
if (use_v1 and use_aiter_rms_norm and not is_eager_execution
and "-rms_norm" not in compilation_config.custom_ops):
compilation_config.custom_ops.append("+rms_norm")
@classmethod
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment