Skip to content

GitLab

"lib/bindings/python/rust/llm/aic_callback.rs" did not exist on "c3908a36ca6ca0d5984f97ae7fe666230a5a696b"
Unverified Commit 74898a70 authored by Danielle Robinson's avatar Danielle Robinson Committed by GitHub
Browse files

[BugFix][LoRA] TritonExperts is ModularMoEPath for FP8 models (#33393) 


Signed-off-by: default avatarDanielle Robinson <dmmaddix@amazon.com>
Co-authored-by: default avatarDanielle Robinson <dmmaddix@amazon.com>
parent 8f5d5120
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 3 deletions
vllm/lora/layers/fused_moe.py
View file @ 74898a70
...@@ -143,9 +143,7 @@ class FusedMoEWithLoRA(BaseLayerWithLoRA): ...@@ -143,9 +143,7 @@ class FusedMoEWithLoRA(BaseLayerWithLoRA):
m_fused_moe_fn.fused_experts, (MarlinExperts, UnfusedOAITritonExperts) m_fused_moe_fn.fused_experts, (MarlinExperts, UnfusedOAITritonExperts)
) )
else: else:
assert isinstance( assert isinstance(m_fused_moe_fn.fused_experts, TritonExperts)
m_fused_moe_fn.fused_experts, (MarlinExperts, TritonExperts)
)
def fwd_decorator(layer, func): def fwd_decorator(layer, func):
def wrapper(*args, **kwargs): def wrapper(*args, **kwargs):
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment