Unverified Commit 51d41265 authored by Harry Mellor's avatar Harry Mellor Committed by GitHub
Browse files

[Docs] Fix typos in EP deployment doc (#24669)


Signed-off-by: default avatarHarry Mellor <19981378+hmellor@users.noreply.github.com>
parent 4984a291
...@@ -158,10 +158,10 @@ vllm serve Qwen/Qwen3-30B-A3B \ ...@@ -158,10 +158,10 @@ vllm serve Qwen/Qwen3-30B-A3B \
### Memory Footprint Overhead ### Memory Footprint Overhead
EPLB uses redundant experts to that need to fit in GPU memory. This means that EPLB may not be a good fit for memory constrained environments or when KV cache space is at a premium. EPLB uses redundant experts that need to fit in GPU memory. This means that EPLB may not be a good fit for memory constrained environments or when KV cache space is at a premium.
This overhead equals `NUM_MOE_LAYERS * BYTES_PER_EXPERT * (NUM_TOTAL_EXPERTS + NUM_REDUNDANT_EXPERTS) ÷ NUM_EP_RANKS`. This overhead equals `NUM_MOE_LAYERS * BYTES_PER_EXPERT * (NUM_TOTAL_EXPERTS + NUM_REDUNDANT_EXPERTS) ÷ NUM_EP_RANKS`.
For DeepSeekV3, this is approximately `2.4 GB` for one redundant expert per rank. For DeepSeekV3, this is approximately `2.4 GB` for one redundant expert per EP rank.
### Example Command ### Example Command
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment