"git@developer.sourcefind.cn:OpenDAS/dynamo.git" did not exist on "c522253b021bc09e10e44403fa5c0727d92000bc"
Commit 8eedb807 authored by Suman Tatiraju's avatar Suman Tatiraju Committed by GitHub
Browse files

docs: update images to high res (#230)

parent cea1902d
...@@ -57,7 +57,7 @@ Dynamo prioritizes seamless integration. Its modular design allows it to work ha ...@@ -57,7 +57,7 @@ Dynamo prioritizes seamless integration. Its modular design allows it to work ha
Disaggregating prefill and decode significantly boosts performance, gaining efficiency the more GPUs that are involved in inference. For example, for Llama 70B, single-node tests show a 30% throughput/GPU improvement, while two-node setups achieve over 2X gains due to better parallelization. Disaggregating prefill and decode significantly boosts performance, gaining efficiency the more GPUs that are involved in inference. For example, for Llama 70B, single-node tests show a 30% throughput/GPU improvement, while two-node setups achieve over 2X gains due to better parallelization.
<figure> <figure>
<img src='images/disagg_perf_benefit.png' alt='missing' /> <img src='images/disagg_perf_benefit.png' alt='missing' width="1200" height="400" />
<p>Tested on H100s with R1 Distilled Llama 70B model FP8 using vLLM. 3K ISL/ 150 OSL</p> <p>Tested on H100s with R1 Distilled Llama 70B model FP8 using vLLM. 3K ISL/ 150 OSL</p>
</figure> </figure>
...@@ -94,7 +94,7 @@ Dynamo's design enables KV cache offloading to system CPU memory, and will be ex ...@@ -94,7 +94,7 @@ Dynamo's design enables KV cache offloading to system CPU memory, and will be ex
<figure> <figure>
<img src='images/nixl.png' alt='missing' /> <img src='images/kv_manager.png' alt='missing' />
<p>Tested with 100K requests to R1 using R1 Distilled Llama 70B FP8 on 2 nodes of H100s. Avg 4K ISL / 800 OSL</p> <p>Tested with 100K requests to R1 using R1 Distilled Llama 70B FP8 on 2 nodes of H100s. Avg 4K ISL / 800 OSL</p>
</figure> </figure>
......
docs/images/disagg_perf_benefit.png

40.8 KB | W: | H:

docs/images/disagg_perf_benefit.png

235 KB | W: | H:

docs/images/disagg_perf_benefit.png
docs/images/disagg_perf_benefit.png
docs/images/disagg_perf_benefit.png
docs/images/disagg_perf_benefit.png
  • 2-up
  • Swipe
  • Onion skin
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment