Tune MI300X fused MoE Triton kernel JSON config. (#3492)

cadd5dbe · Wen-Heng (Jack) Chung · GitHub · bb418ced · cadd5dbe
Unverified Commit cadd5dbe authored Feb 11, 2025 by Wen-Heng (Jack) Chung Committed by GitHub Feb 11, 2025
Showing with 2 additions and 2 deletions

python/sglang/srt/layers/moe/fused_moe_triton/configs/E=256,N=256,device_name=AMD_Instinct_MI300X,dtype=fp8_w8a8,block_shape=[128, 128].json ...nstinct_MI300X,dtype=fp8_w8a8,block_shape=[128, 128].json +2 -2

No files found.
--- a/python/sglang/srt/layers/moe/fused_moe_triton/configs/E=256,N=256,device_name=AMD_Instinct_MI300X,dtype=fp8_w8a8,block_shape=[128, 128].json
+++ b/python/sglang/srt/layers/moe/fused_moe_triton/configs/E=256,N=256,device_name=AMD_Instinct_MI300X,dtype=fp8_w8a8,block_shape=[128, 128].json
@@ -72,10 +72,10 @@
        "waves_per_eu": 0
    },
    "64": {
-        "BLOCK_SIZE_M": 256,
+        "BLOCK_SIZE_M": 32,
        "BLOCK_SIZE_N": 128,
        "BLOCK_SIZE_K": 128,
-        "GROUP_SIZE_M": 1,
+        "GROUP_SIZE_M": 4,
        "num_warps": 4,
        "num_stages": 2,
        "waves_per_eu": 0