Skip to content

GitLab

"ssh:/git@developer.sourcefind.cn:2222/OpenDAS/vllm_cscc.git" did not exist on "202f147cf213b2b1f5407d90ab09209542304d9d"
Unverified Commit 1e668608 authored by ishandhanani's avatar ishandhanani Committed by GitHub
Browse files

docs: agentic inference blog post (#7073) 


Signed-off-by: default avatarDan Gil <dagil@nvidia.com>
Co-authored-by: default avatarDan Gil <dagil@nvidia.com>
parent ba52c8ff
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 1177 additions and 0 deletions
docs/blogs/agentic-inference/agentic-inference.md 0 → 100644
View file @ 1e668608
This diff is collapsed.
docs/blogs/agentic-inference/cumulative-reads-writes.png 0 → 100644
View file @ 1e668608
File suppressed by a .gitattributes entry or the file's encoding is unsupported.
docs/blogs/agentic-inference/kv-memory-hierarchy.d2 0 → 100644
View file @ 1e668608
direction: right
vars: {
d2-config: {
theme-overrides: {
N1: "#e0e0e0"
N2: "#a0a0a0"
N3: "#707070"
N4: "#404040"
N5: "#5d5d5d"
N6: "#151515"
N7: "#0a0a0a"
B1: "#0a0a0a"
B2: "#111111"
B3: "#181818"
B4: "#222222"
B5: "#5d5d5d"
B6: "#404040"
AA2: "#76b900"
AA4: "#3d7ab5"
AA5: "#c4a035"
AB4: "#7a3050"
AB5: "#2a6b55"
}
}
}
gpu: "GPU (HBM)" {
style: {
fill: "#2a4a10"
stroke: "#76b900"
stroke-width: 2
font-color: "#e0e0e0"
font-size: 24
bold: true
border-radius: 0
}
latency: "~ns" {
shape: text
style: {
font-size: 18
font-color: "#76b900"
}
}
cap: "Fastest, smallest" {
shape: text
style: {
font-size: 14
font-color: "#8a8a8a"
italic: true
}
}
}
cpu: "CPU (pinned DRAM)" {
style: {
fill: "#0f1e30"
stroke: "#3d7ab5"
stroke-width: 2
font-color: "#e0e0e0"
font-size: 24
bold: true
border-radius: 0
}
latency: "~us" {
shape: text
style: {
font-size: 18
font-color: "#3d7ab5"
}
}
cap: "10-100x GPU capacity" {
shape: text
style: {
font-size: 14
font-color: "#8a8a8a"
italic: true
}
}
}
nvme: "Local NVMe" {
style: {
fill: "#142025"
stroke: "#50a090"
stroke-width: 2
font-color: "#e0e0e0"
font-size: 24
bold: true
border-radius: 0
}
latency: "~ms" {
shape: text
style: {
font-size: 18
font-color: "#50a090"
}
}
cap: "TBs per node" {
shape: text
style: {
font-size: 14
font-color: "#8a8a8a"
italic: true
}
}
}
remote: "Remote Storage (NIXL)" {
style: {
fill: "#201810"
stroke: "#c08050"
stroke-width: 2
font-color: "#e0e0e0"
font-size: 24
bold: true
border-radius: 0
}
latency: "~ms (RDMA)" {
shape: text
style: {
font-size: 18
font-color: "#c08050"
}
}
cap: "Cluster-wide, shared" {
shape: text
style: {
font-size: 14
font-color: "#8a8a8a"
italic: true
}
}
}
gpu -> cpu: "offload" {
style: {
stroke: "#c4a035"
stroke-width: 2
font-color: "#c4a035"
font-size: 18
}
}
cpu -> nvme: "write-through" {
style: {
stroke: "#c4a035"
stroke-width: 2
font-color: "#c4a035"
font-size: 18
}
}
nvme -> remote: "replicate" {
style: {
stroke: "#c4a035"
stroke-width: 2
font-color: "#c4a035"
font-size: 18
}
}
docs/blogs/agentic-inference/kv-memory-hierarchy.svg 0 → 100644
View file @ 1e668608
This diff is collapsed.
docs/blogs/agentic-inference/protocol-stack.svg 0 → 100644
View file @ 1e668608
<?xml version='1.0' encoding='utf-8'?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 960 420" width="960" height="420">
<rect x="0" y="0" width="960" height="420" fill="#0a0a0a" stroke="none" rx="0" ry="0" />
<defs>
<filter id="glow" x="-20%" y="-20%" width="140%" height="140%">
<feGaussianBlur stdDeviation="3" result="blur" in="SourceGraphic" />
<feMerge>
<feMergeNode in="blur" />
<feMergeNode in="SourceGraphic" />
</feMerge>
</filter>
</defs>
<text x="70" y="88.0" fill="#767676" font-size="9" font-family="Arial, Helvetica, sans-serif" text-anchor="end" font-weight="600" font-style="normal" dy="0.35em" letter-spacing="0.1em">APPLICATION</text>
<text x="70" y="154.0" fill="#767676" font-size="9" font-family="Arial, Helvetica, sans-serif" text-anchor="end" font-weight="600" font-style="normal" dy="0.35em" letter-spacing="0.1em">HARNESS</text>
<text x="70" y="225.0" fill="#767676" font-size="9" font-family="Arial, Helvetica, sans-serif" text-anchor="end" font-weight="600" font-style="normal" dy="0.35em" letter-spacing="0.1em">PROTOCOL</text>
<text x="70" y="296.0" fill="#767676" font-size="9" font-family="Arial, Helvetica, sans-serif" text-anchor="end" font-weight="600" font-style="normal" dy="0.35em" letter-spacing="0.1em">TARGETS</text>
<text x="285.0" y="38" fill="#767676" font-size="15" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" dy="0" letter-spacing="0.05em">Today</text>
<rect x="80" y="58" width="410" height="60" fill="#1a1a1a" stroke="#636363" rx="0" ry="0" />
<text x="285.0" y="88.0" fill="#e0e0e0" font-size="15" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" dy="0.35em">Agent Logic</text>
<rect x="80" y="124" width="410" height="60" fill="#1a1428" stroke="#7650a0" rx="0" ry="0" />
<text x="285.0" y="146.0" fill="#e0e0e0" font-size="15" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" dy="0.35em">Harness</text>
<text x="285.0" y="164.0" fill="#767676" font-size="11" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="normal" font-style="normal" dy="0.35em">Claude Code · Codex · Custom</text>
<rect x="80.0" y="190" width="132.66666666666666" height="70" fill="#1a1a1a" stroke="#636363" rx="0" ry="0" />
<text x="146.33333333333331" y="217.0" fill="#e0e0e0" font-size="15" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" dy="0.35em">MCP</text>
<text x="146.33333333333331" y="235.0" fill="#767676" font-size="11" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="normal" font-style="normal" dy="0.35em">Tools &amp; Data</text>
<rect x="218.66666666666666" y="190" width="132.66666666666666" height="70" fill="#1a1a1a" stroke="#636363" rx="0" ry="0" />
<text x="285.0" y="217.0" fill="#e0e0e0" font-size="15" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" dy="0.35em">A2A</text>
<text x="285.0" y="235.0" fill="#767676" font-size="11" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="normal" font-style="normal" dy="0.35em">Other Agents</text>
<rect x="357.3333333333333" y="190" width="132.66666666666666" height="70" fill="none" stroke="#636363" rx="0" ry="0" stroke-dasharray="6,4" />
<text x="423.66666666666663" y="225.0" fill="#636363" font-size="15" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" dy="0.35em">???</text>
<rect x="80" y="266" width="410" height="60" fill="#1a1a1a" stroke="#636363" rx="0" ry="0" />
<text x="285.0" y="296.0" fill="#e0e0e0" font-size="15" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" dy="0.35em">External Systems</text>
<text x="423.66666666666663" y="350" fill="#767676" font-size="11" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="normal" font-style="italic" dy="0">No standard exists</text>
<text x="735.0" y="38" fill="#76b900" font-size="15" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" dy="0" letter-spacing="0.05em">With Dynamo</text>
<rect x="530" y="58" width="410" height="60" fill="#1a1a1a" stroke="#636363" rx="0" ry="0" />
<text x="735.0" y="88.0" fill="#e0e0e0" font-size="15" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" dy="0.35em">Agent Logic</text>
<rect x="530" y="124" width="410" height="60" fill="#1a1428" stroke="#7650a0" rx="0" ry="0" />
<text x="735.0" y="146.0" fill="#e0e0e0" font-size="15" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" dy="0.35em">Harness</text>
<text x="735.0" y="164.0" fill="#767676" font-size="11" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="normal" font-style="normal" dy="0.35em">Claude Code · Codex · Custom</text>
<rect x="530.0" y="190" width="132.66666666666666" height="70" fill="#1a1a1a" stroke="#636363" rx="0" ry="0" />
<text x="596.3333333333334" y="217.0" fill="#e0e0e0" font-size="15" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" dy="0.35em">MCP</text>
<text x="596.3333333333334" y="235.0" fill="#767676" font-size="11" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="normal" font-style="normal" dy="0.35em">Tools &amp; Data</text>
<rect x="668.6666666666666" y="190" width="132.66666666666666" height="70" fill="#1a1a1a" stroke="#636363" rx="0" ry="0" />
<text x="735.0" y="217.0" fill="#e0e0e0" font-size="15" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" dy="0.35em">A2A</text>
<text x="735.0" y="235.0" fill="#767676" font-size="11" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="normal" font-style="normal" dy="0.35em">Other Agents</text>
<rect x="807.3333333333333" y="190" width="132.66666666666666" height="70" fill="#2a4a10" stroke="#76b900" rx="0" ry="0" stroke-width="2" filter="url(#glow)" />
<text x="873.6666666666666" y="217.0" fill="#76b900" font-size="15" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" dy="0.35em">nvext</text>
<text x="873.6666666666666" y="235.0" fill="#76b900" font-size="11" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="normal" font-style="normal" dy="0.35em">Inference Infrastructure</text>
<rect x="530" y="266" width="410" height="60" fill="#151515" stroke="#4a8c00" rx="0" ry="0" />
<text x="735.0" y="296.0" fill="#e0e0e0" font-size="15" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" dy="0.35em">GPU Cluster · SGLang · vLLM · TRT-LLM</text>
<text x="873.6666666666666" y="350" fill="#76b900" font-size="11" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="normal" font-style="italic" dy="0">Dynamo operates here</text>
</svg>
\ No newline at end of file
docs/blogs/agentic-inference/subagent-lifecycle-vertical.d2 0 → 100644
View file @ 1e668608
vars: {
d2-config: {
theme-overrides: {
N1: "#e0e0e0"
N2: "#a0a0a0"
N3: "#707070"
N4: "#404040"
N5: "#5d5d5d"
N6: "#151515"
N7: "#0a0a0a"
B1: "#0a0a0a"
B2: "#111111"
B3: "#181818"
B4: "#222222"
B5: "#5d5d5d"
B6: "#404040"
AA2: "#76b900"
AA4: "#3d7ab5"
AA5: "#c4a035"
AB4: "#7a3050"
AB5: "#2a6b55"
}
}
}
grid-rows: 3
grid-gap: 20
# Row 1: Lead agent prefix + thinking
row1: "" {
direction: right
style: { fill: transparent; stroke: transparent }
prefix: "System Prompt + Tools + History" {
style: {
fill: "#0f1e30"
stroke: "#3d7ab5"
stroke-width: 2
font-color: "#e0e0e0"
font-size: 13
bold: true
border-radius: 0
}
}
think1: "<think> reasoning" {
style: {
fill: "#2a1a1a"
stroke: "#7a3050"
stroke-width: 2
font-color: "#e0e0e0"
font-size: 13
bold: true
border-radius: 0
}
}
handoff: "Tool Call (handoff)" {
style: {
fill: "#1a1a2e"
stroke: "#5d5d5d"
stroke-width: 2
font-color: "#a0a0a0"
font-size: 13
bold: true
border-radius: 0
}
}
prefix -> think1: {
style: { stroke: "#5d5d5d"; stroke-width: 1 }
}
think1 -> handoff: {
style: { stroke: "#5d5d5d"; stroke-width: 1 }
}
}
# Row 2: Subagent branch
row2: "" {
direction: right
style: { fill: transparent; stroke: transparent }
sub_prefix: "Shared Prefix + Task" {
style: {
fill: "#1a2a1a"
stroke: "#76b900"
stroke-width: 1
font-color: "#e0e0e0"
font-size: 13
bold: true
border-radius: 0
stroke-dash: 5
}
}
sub_think: "<think>" {
style: {
fill: "#2a1a1a"
stroke: "#7a3050"
stroke-width: 1
font-color: "#e0e0e0"
font-size: 13
bold: true
border-radius: 0
stroke-dash: 5
}
}
sub_result: "Result" {
style: {
fill: "#1a2a1a"
stroke: "#76b900"
stroke-width: 1
font-color: "#e0e0e0"
font-size: 13
bold: true
border-radius: 0
stroke-dash: 5
}
}
sub_prefix -> sub_think: {
style: { stroke: "#5d5d5d"; stroke-width: 1; stroke-dash: 5 }
}
sub_think -> sub_result: {
style: { stroke: "#5d5d5d"; stroke-width: 1; stroke-dash: 5 }
}
}
# Row 3: Resume
row3: "" {
direction: right
style: { fill: transparent; stroke: transparent }
resume: "Lead Agent Resumes" {
style: {
fill: "#2a4a10"
stroke: "#76b900"
stroke-width: 2
font-color: "#e0e0e0"
font-size: 13
bold: true
border-radius: 0
}
}
}
# Cross-row connections
row1.handoff -> row2.sub_prefix: "spawn" {
style: {
stroke: "#c4a035"
stroke-width: 2
font-color: "#c4a035"
font-size: 12
}
}
row2.sub_result -> row3.resume: "end (evict ephemeral KV)" {
style: {
stroke: "#76b900"
stroke-width: 2
font-color: "#76b900"
font-size: 12
}
}
row1.handoff -> row3.resume: {
style: {
stroke: "#5d5d5d"
stroke-width: 1
}
}
docs/blogs/agentic-inference/subagent-lifecycle-vertical.svg 0 → 100644
View file @ 1e668608
This diff is collapsed.
docs/blogs/agentic-inference/tool-call-offload-prefetch.d2 0 → 100644
View file @ 1e668608
vars: {
d2-config: {
theme-overrides: {
N1: "#e0e0e0"
N2: "#a0a0a0"
N3: "#707070"
N4: "#404040"
N5: "#5d5d5d"
N6: "#151515"
N7: "#0a0a0a"
B1: "#0a0a0a"
B2: "#111111"
B3: "#181818"
B4: "#222222"
B5: "#5d5d5d"
B6: "#404040"
AA2: "#76b900"
AA4: "#3d7ab5"
AA5: "#c4a035"
AB4: "#7a3050"
AB5: "#2a6b55"
}
}
}
grid-rows: 2
grid-gap: 40
# Row 1: GPU timeline
timeline: "" {
direction: right
style: {
fill: transparent
stroke: transparent
}
prefill: "Prefill" {
style: {
fill: "#2a4a10"
stroke: "#76b900"
stroke-width: 2
font-color: "#e0e0e0"
font-size: 18
bold: true
border-radius: 0
}
}
call1: "LLM Call 1" {
style: {
fill: "#2a4a10"
stroke: "#76b900"
stroke-width: 2
font-color: "#e0e0e0"
font-size: 18
bold: true
border-radius: 0
}
}
tool: "Tool Call" {
style: {
fill: "#1a1a2e"
stroke: "#5d5d5d"
stroke-width: 2
font-color: "#a0a0a0"
font-size: 18
bold: true
border-radius: 0
}
}
call2: "LLM Call 2" {
style: {
fill: "#2a4a10"
stroke: "#76b900"
stroke-width: 2
font-color: "#e0e0e0"
font-size: 18
bold: true
border-radius: 0
}
}
prefill -> call1: {
style: {
stroke: "#5d5d5d"
stroke-width: 1
}
}
call1 -> tool: {
style: {
stroke: "#5d5d5d"
stroke-width: 1
}
}
tool -> call2: {
style: {
stroke: "#5d5d5d"
stroke-width: 1
}
}
}
# Row 2: Storage
storage: "" {
direction: right
style: {
fill: transparent
stroke: transparent
}
pad_left: "" {
style: {
fill: transparent
stroke: transparent
font-size: 8
font-color: transparent
}
width: 120
height: 1
}
host: "Host Memory" {
style: {
fill: "#0f1e30"
stroke: "#3d7ab5"
stroke-width: 2
font-color: "#e0e0e0"
font-size: 18
bold: true
border-radius: 0
}
}
disk: "Storage" {
shape: cylinder
style: {
fill: "#142025"
stroke: "#50a090"
stroke-width: 2
font-color: "#e0e0e0"
font-size: 18
bold: true
}
}
pad_left -> host: {
style: {
stroke: transparent
stroke-width: 0
}
}
host -> disk: {
style: {
stroke: "#5d5d5d"
stroke-width: 1
}
}
disk -> pad_right: {
style: {
stroke: transparent
stroke-width: 0
}
}
pad_right: "" {
style: {
fill: transparent
stroke: transparent
font-size: 8
font-color: transparent
}
width: 120
height: 1
}
}
# Cross-row connections
timeline.call1 -> storage.host: "KV offload" {
style: {
stroke: "#c4a035"
stroke-width: 2
font-color: "#c4a035"
font-size: 16
}
}
storage.host -> timeline.call2: "KV prefetch" {
style: {
stroke: "#76b900"
stroke-width: 2
font-color: "#76b900"
font-size: 16
}
}
docs/blogs/agentic-inference/tool-call-offload-prefetch.svg 0 → 100644
View file @ 1e668608
This diff is collapsed.
docs/blogs/agentic-inference/two-gates.svg 0 → 100644
View file @ 1e668608
<?xml version='1.0' encoding='utf-8'?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 960 380" width="960" height="380">
<rect x="0" y="0" width="960" height="380" fill="#0a0a0a" stroke="none" rx="0" ry="0" />
<text x="140.0" y="36" fill="#767676" font-size="9" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" letter-spacing="0.1em">ARRIVAL ORDER</text>
<text x="485.0" y="36" fill="#767676" font-size="9" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" letter-spacing="0.1em">DISPATCH ORDER</text>
<text x="800.0" y="36" fill="#767676" font-size="9" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal" letter-spacing="0.1em">ENGINE TREATMENT</text>
<text x="330" y="55" fill="#e0e0e0" font-size="13" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal">GATE 1: ROUTER</text>
<text x="330" y="71" fill="#76b900" font-size="11" font-family="'Courier New', Courier, monospace" text-anchor="middle" font-weight="normal" font-style="italic">latency_sensitivity</text>
<text x="330" y="85" fill="#767676" font-size="10" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="normal" font-style="normal">How soon does this reach a worker?</text>
<text x="640" y="55" fill="#e0e0e0" font-size="13" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal">GATE 2: ENGINE</text>
<text x="640" y="71" fill="#76b900" font-size="11" font-family="'Courier New', Courier, monospace" text-anchor="middle" font-weight="normal" font-style="italic">priority</text>
<text x="640" y="85" fill="#767676" font-size="10" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="normal" font-style="normal">How is it treated once there?</text>
<line x1="330" y1="92" x2="330" y2="340" stroke="#636363" stroke-dasharray="6,4" />
<line x1="640" y1="92" x2="640" y2="340" stroke="#636363" stroke-dasharray="6,4" />
<rect x="30" y="115" width="220" height="52" fill="#1a1a1a" stroke="#636363" rx="0" ry="0" />
<text x="140.0" y="137" fill="#a0a0a0" font-size="13" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal">Background · Lint Check</text>
<text x="140.0" y="155" fill="#a0a0a0" font-size="10" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="normal" font-style="italic">ls=0.2 p=1</text>
<rect x="30" y="182" width="220" height="52" fill="#201810" stroke="#c4a035" rx="0" ry="0" />
<text x="140.0" y="204" fill="#c4a035" font-size="13" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal">Subagent · Code Search</text>
<text x="140.0" y="222" fill="#c4a035" font-size="10" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="normal" font-style="italic">ls=0.7 p=5</text>
<rect x="30" y="249" width="220" height="52" fill="#2a4a10" stroke="#76b900" rx="0" ry="0" />
<text x="140.0" y="271" fill="#76b900" font-size="13" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="bold" font-style="normal">Lead Agent · Developer Response</text>
<text x="140.0" y="289" fill="#76b900" font-size="10" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="normal" font-style="italic">ls=0.9 p=10</text>
<path d="M 250,275.0 C 286.0,275.0 294.0,141.0 330,141.0 L 665,141.0" stroke="#76b900" fill="none" stroke-width="2" opacity="0.8" />
<path d="M 250,141.0 C 286.0,141.0 294.0,275.0 330,275.0 L 665,275.0" stroke="#636363" fill="none" stroke-width="2" opacity="0.8" />
<path d="M 250,208.0 L 665,208.0" stroke="#c4a035" fill="none" stroke-width="2" opacity="0.8" />
<rect x="665" y="122.0" width="270" height="38" fill="#2a4a10" stroke="#76b900" rx="0" ry="0" />
<text x="800.0" y="145.0" fill="#76b900" font-size="11" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="600" font-style="normal">batch: first · cache: pinned</text>
<rect x="665" y="189.0" width="270" height="38" fill="#201810" stroke="#c4a035" rx="0" ry="0" />
<text x="800.0" y="212.0" fill="#c4a035" font-size="11" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="600" font-style="normal">batch: middle · cache: moderate</text>
<rect x="665" y="256.0" width="270" height="38" fill="#1a1a1a" stroke="#636363" rx="0" ry="0" />
<text x="800.0" y="279.0" fill="#b04040" font-size="11" font-family="Arial, Helvetica, sans-serif" text-anchor="middle" font-weight="600" font-style="normal">batch: last · cache: evict first</text>
</svg>
\ No newline at end of file
docs/index.yml
View file @ 1e668608
......@@ -273,6 +273,9 @@ navigation:
path: blogs/index.mdx
slug: blog
contents:
- page: "Full-Stack Optimizations for Agentic Inference"
path: blogs/agentic-inference/agentic-inference.md
slug: agentic-inference
- page: "Flash Indexer: Inter-Galactic KV Routing"
path: blogs/flash-indexer/flash-indexer.md
slug: flash-indexer
......
fern/docs.yml
View file @ 1e668608
......@@ -64,6 +64,8 @@ navbar-links:
links:
- text: All Posts
href: /dynamo/dev/blog
- text: "Full-Stack Optimizations for Agentic Inference"
href: /dynamo/dev/blog/agentic-inference
- text: "Flash Indexer: Inter-Galactic KV Routing"
href: /dynamo/dev/blog/flash-indexer
- type: github
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment