Skip to content

GitLab

Commit 284b2450 authored by SAC_fanth's avatar SAC_fanth
Browse files

适配block-wise fp8接口

parent 1a64d266
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
vllm/model_executor/layers/quantization/fp8.py
View file @ 284b2450
...@@ -917,7 +917,7 @@ class Fp8MoEMethod(FusedMoEMethodBase): ...@@ -917,7 +917,7 @@ class Fp8MoEMethod(FusedMoEMethodBase):
logical_to_physical_map: Optional[torch.Tensor] = None, logical_to_physical_map: Optional[torch.Tensor] = None,
logical_replica_count: Optional[torch.Tensor] = None, logical_replica_count: Optional[torch.Tensor] = None,
topk_weights: Optional[torch.Tensor] = None, topk_weights: Optional[torch.Tensor] = None,
topk_ids: Optional[torch.Tensor] = None, topk_ids: Optional[torch.Tensor] = None,**_,
) -> Union[torch.Tensor, tuple[torch.Tensor, torch.Tensor]]: ) -> Union[torch.Tensor, tuple[torch.Tensor, torch.Tensor]]:
if enable_eplb: if enable_eplb:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment