Skip to content

GitLab

Unverified Commit 0d243f2a authored by Divakar Verma's avatar Divakar Verma Committed by GitHub
Browse files

[ROCm][MoE] mi300 mixtral8x7B perf for specific BS (#13577) 


Signed-off-by: default avatarDivakar Verma <divakar.verma@amd.com>
parent 88f6ba32
Hide whitespace changes
Inline Side-by-side
Showing with 5 additions and 5 deletions
vllm/model_executor/layers/fused_moe/configs/E=8,N=14336,device_name=AMD_Instinct_MI300X.json
View file @ 0d243f2a
...@@ -45,8 +45,8 @@ ...@@ -45,8 +45,8 @@
}, },
"16": { "16": {
"BLOCK_SIZE_M": 16, "BLOCK_SIZE_M": 16,
"BLOCK_SIZE_N": 16, "BLOCK_SIZE_N": 64,
"BLOCK_SIZE_K": 256, "BLOCK_SIZE_K": 64,
"GROUP_SIZE_M": 1, "GROUP_SIZE_M": 1,
"num_warps": 2, "num_warps": 2,
"num_stages": 2, "num_stages": 2,
......
vllm/model_executor/layers/fused_moe/configs/E=8,N=3584,device_name=AMD_Instinct_MI300X.json
View file @ 0d243f2a
...@@ -45,8 +45,8 @@ ...@@ -45,8 +45,8 @@
}, },
"16": { "16": {
"BLOCK_SIZE_M": 16, "BLOCK_SIZE_M": 16,
"BLOCK_SIZE_N": 32, "BLOCK_SIZE_N": 64,
"BLOCK_SIZE_K": 256, "BLOCK_SIZE_K": 128,
"GROUP_SIZE_M": 1, "GROUP_SIZE_M": 1,
"num_warps": 2, "num_warps": 2,
"num_stages": 2, "num_stages": 2,
......
vllm/model_executor/layers/fused_moe/configs/E=8,N=7168,device_name=AMD_Instinct_MI300X.json
View file @ 0d243f2a
...@@ -128,7 +128,7 @@ ...@@ -128,7 +128,7 @@
"num_warps": 8, "num_warps": 8,
"num_stages": 2, "num_stages": 2,
"waves_per_eu": 0, "waves_per_eu": 0,
"matrix_instr_nonkdim": 32, "matrix_instr_nonkdim": 16,
"kpack": 2 "kpack": 2
}, },
"512": { "512": {
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment