Unverified Commit ece08dc9 authored by Neal Vaidya's avatar Neal Vaidya Committed by GitHub
Browse files

docs: restructure docs directory and move fern config to fern/ (#6700)


Signed-off-by: default avatarNeal Vaidya <nealv@nvidia.com>
Co-authored-by: default avatarClaude Opus 4.6 <noreply@anthropic.com>
parent 1412e44b
...@@ -33,11 +33,11 @@ Ask for: ...@@ -33,11 +33,11 @@ Ask for:
- **Page title** — appears in the sidebar and as the H1 - **Page title** — appears in the sidebar and as the H1
- **Target section** — which sidebar section (e.g., `Getting Started`, `User Guides`, `Components`) - **Target section** — which sidebar section (e.g., `Getting Started`, `User Guides`, `Components`)
- **Filename** — kebab-case `.md` file (e.g., `my-new-feature.md`) - **Filename** — kebab-case `.md` file (e.g., `my-new-feature.md`)
- **Subdirectory** — which `docs/pages/` subdirectory (e.g., `getting-started`, `features`, `components`) - **Subdirectory** — which `docs/` subdirectory (e.g., `getting-started`, `features`, `components`)
### 2. Create the Page ### 2. Create the Page
Create `docs/pages/<subdirectory>/<filename>.md` with SPDX header and Fern frontmatter: Create `docs/<subdirectory>/<filename>.md` with SPDX header and Fern frontmatter:
```markdown ```markdown
--- ---
...@@ -53,14 +53,14 @@ title: <Page Title> ...@@ -53,14 +53,14 @@ title: <Page Title>
### 3. Add Navigation Entry ### 3. Add Navigation Entry
Edit `docs/versions/dev.yml` and add the page under the correct section: Edit `docs/index.yml` and add the page under the correct section:
```yaml ```yaml
- page: <Page Title> - page: <Page Title>
path: ../pages/<subdirectory>/<filename>.md path: <subdirectory>/<filename>.md
``` ```
**Section locations in `dev.yml`** (search for the comment banner): **Section locations in `index.yml`** (search for the comment banner):
- `# ==================== Getting Started ====================` - `# ==================== Getting Started ====================`
- `# ==================== Kubernetes Deployment ====================` - `# ==================== Kubernetes Deployment ====================`
- `# ==================== User Guides ====================` - `# ==================== User Guides ====================`
...@@ -105,8 +105,6 @@ Reference images from `docs/assets/`: ...@@ -105,8 +105,6 @@ Reference images from `docs/assets/`:
### 5. Validate ### 5. Validate
```bash ```bash
cd docs
ln -sf . fern # symlink required by Fern CLI
fern check fern check
fern docs broken-links fern docs broken-links
``` ```
...@@ -114,7 +112,6 @@ fern docs broken-links ...@@ -114,7 +112,6 @@ fern docs broken-links
### 6. Preview Locally (Optional) ### 6. Preview Locally (Optional)
```bash ```bash
cd docs
fern docs dev fern docs dev
``` ```
...@@ -123,7 +120,8 @@ Opens a local preview at `http://localhost:3000` with hot reload. No token requi ...@@ -123,7 +120,8 @@ Opens a local preview at `http://localhost:3000` with hot reload. No token requi
### 7. Commit ### 7. Commit
```bash ```bash
git add docs/pages/<subdirectory>/<filename>.md docs/versions/dev.yml git add docs/<subdirectory>/<filename>.md docs/index.yml
git commit -s -m "docs: add <page-title> page" git commit -s -m "docs: add <page-title> page"
``` ```
...@@ -131,8 +129,8 @@ git commit -s -m "docs: add <page-title> page" ...@@ -131,8 +129,8 @@ git commit -s -m "docs: add <page-title> page"
### `fern check` fails ### `fern check` fails
- **Invalid YAML in `dev.yml`:** Check indentation — nav entries use 2-space indent. A `- page:` must be inside a `contents:` block. - **Invalid YAML in `index.yml`:** Check indentation — nav entries use 2-space indent. A `- page:` must be inside a `contents:` block.
- **Missing file:** The `path:` in `dev.yml` must match the actual file location. Paths are relative to `docs/versions/` (e.g., `../pages/getting-started/quickstart.md`). - **Missing file:** The `path:` in `index.yml` must match the actual file location. Paths are relative to `docs/` (e.g., `getting-started/quickstart.md`).
### `fern docs broken-links` reports errors ### `fern docs broken-links` reports errors
...@@ -143,11 +141,11 @@ git commit -s -m "docs: add <page-title> page" ...@@ -143,11 +141,11 @@ git commit -s -m "docs: add <page-title> page"
- **MDX parse error:** Angle-bracket URLs like `<https://example.com>` break MDX parsing. Use `[text](https://example.com)` instead. - **MDX parse error:** Angle-bracket URLs like `<https://example.com>` break MDX parsing. Use `[text](https://example.com)` instead.
- **Broken links check:** The `detect_broken_links.py` job checks relative links across all docs. If your new page links to a file that doesn't exist yet, CI will fail. - **Broken links check:** The `detect_broken_links.py` job checks relative links across all docs. If your new page links to a file that doesn't exist yet, CI will fail.
- **Fern publish error:** Check the Actions tab for the `Fern Docs` workflow. Common causes: expired `FERN_TOKEN`, invalid `docs.yml` syntax, or a file referenced in `dev.yml` that wasn't synced to `docs-website`. - **Fern publish error:** Check the Actions tab for the `Fern Docs` workflow. Common causes: expired `FERN_TOKEN`, invalid `fern/docs.yml` syntax, or a file referenced in `index.yml` that wasn't synced to `docs-website`.
### Page doesn't appear on the live site ### Page doesn't appear on the live site
- **Missing nav entry:** The page exists but isn't in `docs/versions/dev.yml`. Add it. - **Missing nav entry:** The page exists but isn't in `docs/index.yml`. Add it.
- **Hidden section:** The page is inside a `hidden: true` section. It's accessible by direct URL but won't appear in the sidebar. - **Hidden section:** The page is inside a `hidden: true` section. It's accessible by direct URL but won't appear in the sidebar.
- **Sync delay:** After merge to `main`, the sync-dev workflow takes a few minutes to publish. - **Sync delay:** After merge to `main`, the sync-dev workflow takes a few minutes to publish.
...@@ -155,8 +153,9 @@ git commit -s -m "docs: add <page-title> page" ...@@ -155,8 +153,9 @@ git commit -s -m "docs: add <page-title> page"
| File | Purpose | | File | Purpose |
|------|---------| |------|---------|
| `docs/versions/dev.yml` | Navigation tree — add entries here | | `docs/index.yml` | Navigation tree — add entries here |
| `docs/pages/` | Content directory — create pages here | | `docs/` | Content directory — create pages here |
| `docs/assets/` | Images, SVGs, fonts | | `docs/assets/` | Images, SVGs, fonts |
| `docs/convert_callouts.py` | Callout conversion rules (GitHub → Fern) | | `fern/docs.yml` | Fern site configuration |
| `fern/convert_callouts.py` | Callout conversion rules (GitHub → Fern) |
| `docs/README.md` | Full architecture guide | | `docs/README.md` | Full architecture guide |
...@@ -30,25 +30,25 @@ Must be in the `dynamo` repo (not `dynamo-tpm`). Architecture details: `docs/REA ...@@ -30,25 +30,25 @@ Must be in the `dynamo` repo (not `dynamo-tpm`). Architecture details: `docs/REA
### 1. Identify the Page ### 1. Identify the Page
Ask for the page to remove (accepts any of): Ask for the page to remove (accepts any of):
- File path (e.g., `docs/pages/guides/old-page.md`) - File path (e.g., `docs/guides/old-page.md`)
- Page title (e.g., "Old Page") - Page title (e.g., "Old Page")
- Topic keyword to search for - Topic keyword to search for
If given a title or keyword, search for the page: If given a title or keyword, search for the page:
```bash ```bash
# Search by title in navigation # Search by title in navigation
grep -n "<title>" docs/versions/dev.yml grep -n "<title>" docs/index.yml
# Search by keyword in content # Search by keyword in content
grep -rl "<keyword>" docs/pages/ grep -rl "<keyword>" docs/
``` ```
### 2. Find the Navigation Entry ### 2. Find the Navigation Entry
Locate the page's entry in `docs/versions/dev.yml`: Locate the page's entry in `docs/index.yml`:
```bash ```bash
grep -n "<filename>" docs/versions/dev.yml grep -n "<filename>" docs/index.yml
``` ```
Note the exact `- page:` block and its indentation level. If the page is the Note the exact `- page:` block and its indentation level. If the page is the
...@@ -60,10 +60,10 @@ Search for references to this page from other docs: ...@@ -60,10 +60,10 @@ Search for references to this page from other docs:
```bash ```bash
# Search for the filename across all docs pages # Search for the filename across all docs pages
grep -r "<filename>" docs/pages/ --include="*.md" grep -r "<filename>" docs/ --include="*.md"
# Also check the navigation file for any cross-references # Also check the navigation file for any cross-references
grep -r "<filename>" docs/versions/ grep -r "<filename>" docs/
``` ```
Report any files that link to the page being removed — these links will break Report any files that link to the page being removed — these links will break
...@@ -72,12 +72,12 @@ and need updating. ...@@ -72,12 +72,12 @@ and need updating.
### 4. Remove the Markdown File ### 4. Remove the Markdown File
```bash ```bash
git rm docs/pages/<subdirectory>/<filename>.md git rm docs/<subdirectory>/<filename>.md
``` ```
### 5. Remove the Navigation Entry ### 5. Remove the Navigation Entry
Edit `docs/versions/dev.yml` and delete the `- page:` block (and its `path:` Edit `docs/index.yml` and delete the `- page:` block (and its `path:`
line). If this was the last page in a section, remove the entire `- section:` line). If this was the last page in a section, remove the entire `- section:`
block. block.
...@@ -91,8 +91,6 @@ For each file that linked to the removed page: ...@@ -91,8 +91,6 @@ For each file that linked to the removed page:
### 7. Validate ### 7. Validate
```bash ```bash
cd docs
ln -sf . fern # symlink required by Fern CLI
fern check fern check
fern docs broken-links fern docs broken-links
``` ```
...@@ -100,7 +98,6 @@ fern docs broken-links ...@@ -100,7 +98,6 @@ fern docs broken-links
### 8. Preview Locally (Optional) ### 8. Preview Locally (Optional)
```bash ```bash
cd docs
fern docs dev fern docs dev
``` ```
...@@ -117,18 +114,18 @@ git commit -s -m "docs: remove <page-title> page" ...@@ -117,18 +114,18 @@ git commit -s -m "docs: remove <page-title> page"
### `fern check` fails ### `fern check` fails
- **Orphaned nav entry:** You deleted the file but left the `- page:` entry in `dev.yml`. Remove it. - **Orphaned nav entry:** You deleted the file but left the `- page:` entry in `index.yml`. Remove it.
- **Empty section:** If the removed page was the only entry in a section, delete the entire `- section:` block from `dev.yml`. - **Empty section:** If the removed page was the only entry in a section, delete the entire `- section:` block from `index.yml`.
### `fern docs broken-links` reports errors ### `fern docs broken-links` reports errors
- **Incoming links to removed page:** Other pages still link to the deleted file. Search with `grep -r "<filename>" docs/pages/` and update or remove those links. - **Incoming links to removed page:** Other pages still link to the deleted file. Search with `grep -r "<filename>" docs/` and update or remove those links.
### CI fails after merge ### CI fails after merge
- **MDX parse error:** Angle-bracket URLs like `<https://example.com>` break MDX parsing. Use `[text](https://example.com)` instead. - **MDX parse error:** Angle-bracket URLs like `<https://example.com>` break MDX parsing. Use `[text](https://example.com)` instead.
- **Broken links check:** The `detect_broken_links.py` job found pages that still reference the removed file. Fix all incoming links before merging. - **Broken links check:** The `detect_broken_links.py` job found pages that still reference the removed file. Fix all incoming links before merging.
- **Fern publish error:** Check the Actions tab for the `Fern Docs` workflow. Common causes: expired `FERN_TOKEN`, invalid `docs.yml` syntax. - **Fern publish error:** Check the Actions tab for the `Fern Docs` workflow. Common causes: expired `FERN_TOKEN`, invalid `fern/docs.yml` syntax.
### Page still appears on the live site ### Page still appears on the live site
...@@ -139,6 +136,7 @@ git commit -s -m "docs: remove <page-title> page" ...@@ -139,6 +136,7 @@ git commit -s -m "docs: remove <page-title> page"
| File | Purpose | | File | Purpose |
|------|---------| |------|---------|
| `docs/versions/dev.yml` | Navigation tree — remove entries here | | `docs/index.yml` | Navigation tree — remove entries here |
| `docs/pages/` | Content directory — delete pages here | | `docs/` | Content directory — delete pages here |
| `fern/docs.yml` | Fern site configuration |
| `docs/README.md` | Full architecture guide | | `docs/README.md` | Full architecture guide |
...@@ -30,29 +30,29 @@ Must be in the `dynamo` repo (not `dynamo-tpm`). Architecture details: `docs/REA ...@@ -30,29 +30,29 @@ Must be in the `dynamo` repo (not `dynamo-tpm`). Architecture details: `docs/REA
### 1. Locate the Page ### 1. Locate the Page
Ask for the page to update (accepts any of): Ask for the page to update (accepts any of):
- File path (e.g., `docs/pages/guides/quickstart.md`) - File path (e.g., `docs/guides/quickstart.md`)
- Page title (e.g., "Quickstart") - Page title (e.g., "Quickstart")
- Topic keyword to search for - Topic keyword to search for
If given a title or keyword, find the page: If given a title or keyword, find the page:
```bash ```bash
# Search by title in navigation # Search by title in navigation
grep -n "<title>" docs/versions/dev.yml grep -n "<title>" docs/index.yml
# Search by keyword in content # Search by keyword in content
grep -rl "<keyword>" docs/pages/ grep -rl "<keyword>" docs/
``` ```
### 2. Read Current Content ### 2. Read Current Content
Read the page and its navigation entry: Read the page and its navigation entry:
- The markdown file in `docs/pages/` - The markdown file in `docs/`
- The corresponding entry in `docs/versions/dev.yml` - The corresponding entry in `docs/index.yml`
Note the current: Note the current:
- **Title** (from frontmatter `title:` field) - **Title** (from frontmatter `title:` field)
- **Section** (which sidebar section it belongs to) - **Section** (which sidebar section it belongs to)
- **Path** (relative path in `dev.yml`) - **Path** (relative path in `index.yml`)
### 3. Apply Edits ### 3. Apply Edits
...@@ -65,16 +65,16 @@ Edit the markdown file directly. No navigation changes needed. ...@@ -65,16 +65,16 @@ Edit the markdown file directly. No navigation changes needed.
#### Title Change #### Title Change
1. Update the `title:` field in the markdown frontmatter 1. Update the `title:` field in the markdown frontmatter
2. Update the `- page:` display name in `docs/versions/dev.yml` 2. Update the `- page:` display name in `docs/index.yml`
#### Section Move #### Section Move
1. Move the markdown file to the new subdirectory: 1. Move the markdown file to the new subdirectory:
```bash ```bash
git mv docs/pages/<old-subdir>/<file>.md docs/pages/<new-subdir>/<file>.md git mv docs/<old-subdir>/<file>.md docs/<new-subdir>/<file>.md
``` ```
2. Remove the old `- page:` entry from `dev.yml` 2. Remove the old `- page:` entry from `index.yml`
3. Add a new `- page:` entry under the target section in `dev.yml` 3. Add a new `- page:` entry under the target section in `index.yml`
4. Update the `path:` to reflect the new location 4. Update the `path:` to reflect the new location
### 4. Check Incoming Links (If Path Changed) ### 4. Check Incoming Links (If Path Changed)
...@@ -83,8 +83,8 @@ If the file was moved, search for references that need updating: ...@@ -83,8 +83,8 @@ If the file was moved, search for references that need updating:
```bash ```bash
# Search for the old path across all docs # Search for the old path across all docs
grep -r "<old-filename>" docs/pages/ --include="*.md" grep -r "<old-filename>" docs/ --include="*.md"
grep -r "<old-filename>" docs/versions/ grep -r "<old-filename>" docs/
``` ```
Update all references to point to the new path. Update all references to point to the new path.
...@@ -114,8 +114,6 @@ Use standard **GitHub-flavored markdown**. For callouts, use GitHub's native syn ...@@ -114,8 +114,6 @@ Use standard **GitHub-flavored markdown**. For callouts, use GitHub's native syn
### 6. Validate ### 6. Validate
```bash ```bash
cd docs
ln -sf . fern # symlink required by Fern CLI
fern check fern check
fern docs broken-links fern docs broken-links
``` ```
...@@ -123,7 +121,6 @@ fern docs broken-links ...@@ -123,7 +121,6 @@ fern docs broken-links
### 7. Preview Locally (Optional) ### 7. Preview Locally (Optional)
```bash ```bash
cd docs
fern docs dev fern docs dev
``` ```
...@@ -132,7 +129,7 @@ Opens a local preview at `http://localhost:3000` with hot reload. No token requi ...@@ -132,7 +129,7 @@ Opens a local preview at `http://localhost:3000` with hot reload. No token requi
### 8. Commit ### 8. Commit
```bash ```bash
git add docs/pages/ docs/versions/dev.yml git add docs/
git commit -s -m "docs: update <page-title>" git commit -s -m "docs: update <page-title>"
``` ```
...@@ -140,32 +137,33 @@ git commit -s -m "docs: update <page-title>" ...@@ -140,32 +137,33 @@ git commit -s -m "docs: update <page-title>"
### `fern check` fails ### `fern check` fails
- **Invalid YAML in `dev.yml`:** Check indentation — nav entries use 2-space indent. A `- page:` must be inside a `contents:` block. - **Invalid YAML in `index.yml`:** Check indentation — nav entries use 2-space indent. A `- page:` must be inside a `contents:` block.
- **Missing file:** If you moved a page, the `path:` in `dev.yml` must match the new location. - **Missing file:** If you moved a page, the `path:` in `index.yml` must match the new location.
- **Duplicate entry:** The same page appears twice in `dev.yml`. Remove the old entry after a section move. - **Duplicate entry:** The same page appears twice in `index.yml`. Remove the old entry after a section move.
### `fern docs broken-links` reports errors ### `fern docs broken-links` reports errors
- **Stale internal links:** After moving or renaming a page, other pages may still link to the old path. Search with `grep -r "<old-filename>" docs/pages/` and update references. - **Stale internal links:** After moving or renaming a page, other pages may still link to the old path. Search with `grep -r "<old-filename>" docs/` and update references.
- **Anchor not found:** A `#section-heading` link doesn't match any heading in the target page. Check if the heading text changed. - **Anchor not found:** A `#section-heading` link doesn't match any heading in the target page. Check if the heading text changed.
### CI fails after merge ### CI fails after merge
- **MDX parse error:** Angle-bracket URLs like `<https://example.com>` break MDX parsing. Use `[text](https://example.com)` instead. - **MDX parse error:** Angle-bracket URLs like `<https://example.com>` break MDX parsing. Use `[text](https://example.com)` instead.
- **Broken links check:** The `detect_broken_links.py` job found stale references to the old path. Fix all incoming links before merging. - **Broken links check:** The `detect_broken_links.py` job found stale references to the old path. Fix all incoming links before merging.
- **Fern publish error:** Check the Actions tab for the `Fern Docs` workflow. Common causes: expired `FERN_TOKEN`, invalid `docs.yml` syntax, or a moved file that wasn't synced to `docs-website`. - **Fern publish error:** Check the Actions tab for the `Fern Docs` workflow. Common causes: expired `FERN_TOKEN`, invalid `fern/docs.yml` syntax, or a moved file that wasn't synced to `docs-website`.
### Changes don't appear on the live site ### Changes don't appear on the live site
- **Title mismatch:** You updated the frontmatter `title:` but not the `- page:` name in `dev.yml` (or vice versa). Keep them in sync. - **Title mismatch:** You updated the frontmatter `title:` but not the `- page:` name in `index.yml` (or vice versa). Keep them in sync.
- **Sync delay:** After merge to `main`, the sync-dev workflow takes a few minutes to publish. - **Sync delay:** After merge to `main`, the sync-dev workflow takes a few minutes to publish.
## Key References ## Key References
| File | Purpose | | File | Purpose |
|------|---------| |------|---------|
| `docs/versions/dev.yml` | Navigation tree — update entries here if title/path changes | | `docs/index.yml` | Navigation tree — update entries here if title/path changes |
| `docs/pages/` | Content directory — edit pages here | | `docs/` | Content directory — edit pages here |
| `docs/assets/` | Images, SVGs, fonts | | `docs/assets/` | Images, SVGs, fonts |
| `docs/convert_callouts.py` | Callout conversion rules (GitHub → Fern) | | `fern/docs.yml` | Fern site configuration |
| `fern/convert_callouts.py` | Callout conversion rules (GitHub → Fern) |
| `docs/README.md` | Full architecture guide | | `docs/README.md` | Full architecture guide |
...@@ -100,7 +100,7 @@ operator: ...@@ -100,7 +100,7 @@ operator:
- *ci - *ci
- 'deploy/operator/**' - 'deploy/operator/**'
- 'deploy/operator/.*' - 'deploy/operator/.*'
- 'docs/pages/kubernetes/api-reference.md' - 'docs/kubernetes/api-reference.md'
deploy: deploy:
- '!**/*.md' - '!**/*.md'
......
...@@ -101,13 +101,7 @@ jobs: ...@@ -101,13 +101,7 @@ jobs:
- name: Install Fern CLI - name: Install Fern CLI
run: npm install -g fern-api run: npm install -g fern-api
# Fern CLI expects a fern/ directory; create symlink after docs/ migration
- name: Create fern directory symlink
working-directory: docs
run: ln -s . fern
- name: Validate Fern configuration - name: Validate Fern configuration
working-directory: docs
run: fern check run: fern check
fern-broken-links: fern-broken-links:
...@@ -130,13 +124,7 @@ jobs: ...@@ -130,13 +124,7 @@ jobs:
- name: Install Fern CLI - name: Install Fern CLI
run: npm install -g fern-api run: npm install -g fern-api
# Fern CLI expects a fern/ directory; create symlink after docs/ migration
- name: Create fern directory symlink
working-directory: docs
run: ln -s . fern
- name: Check for broken links - name: Check for broken links
working-directory: docs
run: fern docs broken-links run: fern docs broken-links
############################################################################# #############################################################################
...@@ -178,23 +166,34 @@ jobs: ...@@ -178,23 +166,34 @@ jobs:
token: ${{ secrets.GITHUB_TOKEN }} token: ${{ secrets.GITHUB_TOKEN }}
- name: Sync dev content from main - name: Sync dev content from main
run: | run: |
# Sync pages/ directory (dev content) # Sync content directories from docs/ to fern/pages-dev/ on docs-website
echo "Syncing pages/ to docs-website branch..." # Excludes non-content items that are synced separately or not needed
rm -rf docs-checkout/fern/pages echo "Syncing content pages to docs-website branch..."
cp -r source-checkout/docs/pages docs-checkout/fern/pages rm -rf docs-checkout/fern/pages-dev
mkdir -p docs-checkout/fern/pages-dev
# Sync versions/dev.yml (dev navigation) rsync -a \
echo "Syncing versions/dev.yml to docs-website branch..." --exclude='assets' \
cp source-checkout/docs/versions/dev.yml docs-checkout/fern/versions/dev.yml --exclude='blogs' \
--exclude='diagrams' \
--exclude='index.yml' \
source-checkout/docs/ docs-checkout/fern/pages-dev/
# Sync index.yml as versions/dev.yml and transform paths for docs-website layout
echo "Syncing index.yml to docs-website branch as versions/dev.yml..."
cp source-checkout/docs/index.yml docs-checkout/fern/versions/dev.yml
# Sync assets/ directory # Sync assets/ directory
echo "Syncing assets/ to docs-website branch..." echo "Syncing assets/ to docs-website branch..."
rm -rf docs-checkout/fern/assets rm -rf docs-checkout/fern/assets
cp -r source-checkout/docs/assets docs-checkout/fern/assets cp -r source-checkout/docs/assets docs-checkout/fern/assets
# Symlink assets into pages-dev/ so relative image paths in markdown resolve correctly
# (e.g. ../../assets/img/foo.png from pages-dev/observability/metrics.md)
ln -sfn ../assets docs-checkout/fern/pages-dev/assets
# Sync fern.config.json # Sync fern.config.json
echo "Syncing fern.config.json to docs-website branch..." echo "Syncing fern.config.json to docs-website branch..."
cp source-checkout/docs/fern.config.json docs-checkout/fern/fern.config.json cp source-checkout/fern/fern.config.json docs-checkout/fern/fern.config.json
# Sync docs/README.md (developer guide, referenced from dev.yml) # Sync docs/README.md (developer guide, referenced from dev.yml)
if [ -f source-checkout/docs/README.md ]; then if [ -f source-checkout/docs/README.md ]; then
...@@ -202,20 +201,20 @@ jobs: ...@@ -202,20 +201,20 @@ jobs:
fi fi
# Sync .gitignore if it exists # Sync .gitignore if it exists
if [ -f source-checkout/docs/.gitignore ]; then if [ -f source-checkout/fern/.gitignore ]; then
cp source-checkout/docs/.gitignore docs-checkout/fern/.gitignore cp source-checkout/fern/.gitignore docs-checkout/fern/.gitignore
fi fi
# Sync convert_callouts.py script # Sync convert_callouts.py script
if [ -f source-checkout/docs/convert_callouts.py ]; then if [ -f source-checkout/fern/convert_callouts.py ]; then
cp source-checkout/docs/convert_callouts.py docs-checkout/fern/convert_callouts.py cp source-checkout/fern/convert_callouts.py docs-checkout/fern/convert_callouts.py
fi fi
# Sync components/ directory (e.g., CustomFooter.tsx) # Sync components/ directory (e.g., CustomFooter.tsx)
if [ -d source-checkout/docs/components ]; then if [ -d source-checkout/fern/components ]; then
echo "Syncing components/ to docs-website branch..." echo "Syncing components/ to docs-website branch..."
rm -rf docs-checkout/fern/components rm -rf docs-checkout/fern/components
cp -r source-checkout/docs/components docs-checkout/fern/components cp -r source-checkout/fern/components docs-checkout/fern/components
fi fi
# Sync blogs/ directory # Sync blogs/ directory
...@@ -226,15 +225,26 @@ jobs: ...@@ -226,15 +225,26 @@ jobs:
fi fi
# Sync main.css # Sync main.css
if [ -f source-checkout/docs/main.css ]; then if [ -f source-checkout/fern/main.css ]; then
echo "Syncing main.css to docs-website branch..." echo "Syncing main.css to docs-website branch..."
cp source-checkout/docs/main.css docs-checkout/fern/main.css cp source-checkout/fern/main.css docs-checkout/fern/main.css
fi fi
- name: Transform paths in dev.yml for docs-website layout
run: |
# In the source repo, index.yml uses paths relative to docs/ (e.g. getting-started/quickstart.md).
# On docs-website, fern/versions/dev.yml needs ../pages-dev/ prefix for content
# and ../ prefix for blogs (which are a sibling of pages/, not inside it).
#
# Order matters: transform blogs/ first so the second sub() won't re-match them
# (they'll start with ../ which doesn't match ^[a-zA-Z]).
yq -i '(.. | select(has("path")).path) |= sub("^blogs/", "../blogs/")' docs-checkout/fern/versions/dev.yml
yq -i '(.. | select(has("path")).path) |= sub("^([a-zA-Z])", "../pages-dev/${1}")' docs-checkout/fern/versions/dev.yml
- name: Convert GitHub callouts to Fern format - name: Convert GitHub callouts to Fern format
run: | run: |
echo "Converting GitHub-style callouts to Fern format in pages/..." echo "Converting GitHub-style callouts to Fern format in pages/..."
python3 docs-checkout/fern/convert_callouts.py --dir docs-checkout/fern/pages python3 docs-checkout/fern/convert_callouts.py --dir docs-checkout/fern/pages-dev
echo "Callout conversion complete." echo "Callout conversion complete."
- name: Update docs.yml preserving products - name: Update docs.yml preserving products
...@@ -248,7 +258,11 @@ jobs: ...@@ -248,7 +258,11 @@ jobs:
cat /tmp/preserved_product.yml cat /tmp/preserved_product.yml
# Copy docs.yml from source to get config updates (redirects, layout, etc.) # Copy docs.yml from source to get config updates (redirects, layout, etc.)
cp ../../source-checkout/docs/docs.yml docs.yml cp ../../source-checkout/fern/docs.yml docs.yml
# Fix asset paths: source uses ../docs/assets/ (relative to fern/ in source repo),
# but on docs-website assets live at fern/assets/ so we need ./assets/
sed -i 's|\.\./docs/assets/|./assets/|g' docs.yml
# Restore the preserved products[0] block # Restore the preserved products[0] block
yq -i '.products[0] = load("/tmp/preserved_product.yml")' docs.yml yq -i '.products[0] = load("/tmp/preserved_product.yml")' docs.yml
...@@ -290,11 +304,15 @@ jobs: ...@@ -290,11 +304,15 @@ jobs:
env: env:
FERN_TOKEN: ${{ secrets.FERN_TOKEN }} FERN_TOKEN: ${{ secrets.FERN_TOKEN }}
run: | run: |
OUTPUT=$(fern generate --docs --preview 2>&1) if OUTPUT=$(fern generate --docs --preview 2>&1); then
FERN_EXIT=$? FERN_EXIT=0
else
FERN_EXIT=$?
fi
echo "$OUTPUT" echo "$OUTPUT"
if [ $FERN_EXIT -ne 0 ]; then if [ $FERN_EXIT -ne 0 ]; then
echo "::warning::Fern docs preview generation failed (exit $FERN_EXIT)" echo "::error::Fern docs preview generation failed (exit $FERN_EXIT)"
exit 1
fi fi
URL=$(echo "$OUTPUT" | grep -oP 'Published docs to \K\S+') || true URL=$(echo "$OUTPUT" | grep -oP 'Published docs to \K\S+') || true
if [ -n "$URL" ]; then if [ -n "$URL" ]; then
...@@ -411,10 +429,10 @@ jobs: ...@@ -411,10 +429,10 @@ jobs:
run: | run: |
TAG="${{ steps.version.outputs.tag }}" TAG="${{ steps.version.outputs.tag }}"
echo "Creating fern/pages-$TAG/ from fern/pages/..." echo "Creating fern/pages-$TAG/ from fern/pages-dev/..."
# Copy current pages/ to pages-vX.Y.Z/ # Copy current pages-dev/ to pages-vX.Y.Z/
cp -r fern/pages "fern/pages-$TAG" cp -r fern/pages-dev "fern/pages-$TAG"
echo "Created fern/pages-$TAG/" echo "Created fern/pages-$TAG/"
ls -la "fern/pages-$TAG/" | head -20 ls -la "fern/pages-$TAG/" | head -20
...@@ -466,8 +484,8 @@ jobs: ...@@ -466,8 +484,8 @@ jobs:
sed -i "s/# Navigation structure for Latest version/# Navigation structure for $TAG version/" "$VERSION_FILE" sed -i "s/# Navigation structure for Latest version/# Navigation structure for $TAG version/" "$VERSION_FILE"
sed -i "s|# Matching https://docs.nvidia.com/dynamo/latest/|# Snapshot from tag $TAG|" "$VERSION_FILE" sed -i "s|# Matching https://docs.nvidia.com/dynamo/latest/|# Snapshot from tag $TAG|" "$VERSION_FILE"
# Update all page paths from ../pages/ to ../pages-vX.Y.Z/ # Update all page paths from ../pages-dev/ to ../pages-vX.Y.Z/
sed -i "s|path: \.\./pages/|path: ../pages-$TAG/|g" "$VERSION_FILE" sed -i "s|path: \.\./pages-dev/|path: ../pages-$TAG/|g" "$VERSION_FILE"
echo "Created $VERSION_FILE" echo "Created $VERSION_FILE"
echo "First 30 lines:" echo "First 30 lines:"
......
...@@ -129,7 +129,7 @@ Issues labeled `good-first-issue` are sized for new contributors. We provide ext ...@@ -129,7 +129,7 @@ Issues labeled `good-first-issue` are sized for new contributors. We provide ext
<!-- We were given the feedback that having information on architecture, languages used, etc. would be helpful for external contributors --> <!-- We were given the feedback that having information on architecture, languages used, etc. would be helpful for external contributors -->
Understanding Dynamo's architecture helps you find where to make changes. For the complete picture, see the [Architecture Documentation](docs/pages/design-docs/architecture.md) and [Support Matrix](docs/pages/reference/support-matrix.md). Understanding Dynamo's architecture helps you find where to make changes. For the complete picture, see the [Architecture Documentation](docs/design-docs/architecture.md) and [Support Matrix](docs/reference/support-matrix.md).
![Dynamo Architecture](docs/assets/img/architecture.png) ![Dynamo Architecture](docs/assets/img/architecture.png)
...@@ -148,9 +148,9 @@ Understanding Dynamo's architecture helps you find where to make changes. For th ...@@ -148,9 +148,9 @@ Understanding Dynamo's architecture helps you find where to make changes. For th
| Plane | Purpose | Documentation | | Plane | Purpose | Documentation |
|-------|---------|---------------| |-------|---------|---------------|
| **Discovery Plane** | Service registration and discovery across components | [docs/pages/design-docs/distributed-runtime.md](docs/pages/design-docs/distributed-runtime.md) | | **Discovery Plane** | Service registration and discovery across components | [docs/design-docs/distributed-runtime.md](docs/design-docs/distributed-runtime.md) |
| **Request Plane** | High-performance request routing between components | [docs/pages/design-docs/request-plane.md](docs/pages/design-docs/request-plane.md) | | **Request Plane** | High-performance request routing between components | [docs/design-docs/request-plane.md](docs/design-docs/request-plane.md) |
| **KV Event Plane** | KV cache event propagation for cache-aware routing | [docs/pages/design-docs/event-plane.md](docs/pages/design-docs/event-plane.md) | | **KV Event Plane** | KV cache event propagation for cache-aware routing | [docs/design-docs/event-plane.md](docs/design-docs/event-plane.md) |
### Kubernetes Deployment ### Kubernetes Deployment
......
...@@ -22,7 +22,7 @@ limitations under the License. ...@@ -22,7 +22,7 @@ limitations under the License.
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/ai-dynamo/dynamo) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/ai-dynamo/dynamo)
[![Discord](https://dcbadge.limes.pink/api/server/D92uqZRjCZ?style=flat)](https://discord.gg/D92uqZRjCZ) ![Community Contributors](https://img.shields.io/badge/community_contributors-70%2B-brightgreen) [![Discord](https://dcbadge.limes.pink/api/server/D92uqZRjCZ?style=flat)](https://discord.gg/D92uqZRjCZ) ![Community Contributors](https://img.shields.io/badge/community_contributors-70%2B-brightgreen)
| **[Roadmap](https://github.com/ai-dynamo/dynamo/issues/5506)** | **[Support Matrix](https://github.com/ai-dynamo/dynamo/blob/main/docs/pages/reference/support-matrix.md)** | **[Docs](https://docs.nvidia.com/dynamo/)** | **[Recipes](https://github.com/ai-dynamo/dynamo/tree/main/recipes)** | **[Examples](https://github.com/ai-dynamo/dynamo/tree/main/examples)** | **[Prebuilt Containers](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo)** | **[Design Proposals](https://github.com/ai-dynamo/enhancements)** | **[Blogs](https://developer.nvidia.com/blog/tag/nvidia-dynamo)** | **[Roadmap](https://github.com/ai-dynamo/dynamo/issues/5506)** | **[Support Matrix](https://github.com/ai-dynamo/dynamo/blob/main/docs/reference/support-matrix.md)** | **[Docs](https://docs.nvidia.com/dynamo/)** | **[Recipes](https://github.com/ai-dynamo/dynamo/tree/main/recipes)** | **[Examples](https://github.com/ai-dynamo/dynamo/tree/main/examples)** | **[Prebuilt Containers](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo)** | **[Design Proposals](https://github.com/ai-dynamo/enhancements)** | **[Blogs](https://developer.nvidia.com/blog/tag/nvidia-dynamo)**
# NVIDIA Dynamo # NVIDIA Dynamo
...@@ -48,17 +48,17 @@ Built in Rust for performance and Python for extensibility, Dynamo is fully open ...@@ -48,17 +48,17 @@ Built in Rust for performance and Python for extensibility, Dynamo is fully open
## Backend Feature Support ## Backend Feature Support
| | [SGLang](docs/pages/backends/sglang/README.md) | [TensorRT-LLM](docs/pages/backends/trtllm/README.md) | [vLLM](docs/pages/backends/vllm/README.md) | | | [SGLang](docs/backends/sglang/README.md) | [TensorRT-LLM](docs/backends/trtllm/README.md) | [vLLM](docs/backends/vllm/README.md) |
|---|:----:|:----------:|:--:| |---|:----:|:----------:|:--:|
| **Best For** | High-throughput serving | Maximum performance | Broadest feature coverage | | **Best For** | High-throughput serving | Maximum performance | Broadest feature coverage |
| [**Disaggregated Serving**](docs/pages/design-docs/disagg-serving.md) | ✅ | ✅ | ✅ | | [**Disaggregated Serving**](docs/design-docs/disagg-serving.md) | ✅ | ✅ | ✅ |
| [**KV-Aware Routing**](docs/pages/components/router/README.md) | ✅ | ✅ | ✅ | | [**KV-Aware Routing**](docs/components/router/README.md) | ✅ | ✅ | ✅ |
| [**SLA-Based Planner**](docs/pages/components/planner/planner-guide.md) | ✅ | ✅ | ✅ | | [**SLA-Based Planner**](docs/components/planner/planner-guide.md) | ✅ | ✅ | ✅ |
| [**KVBM**](docs/pages/components/kvbm/README.md) | 🚧 | ✅ | ✅ | | [**KVBM**](docs/components/kvbm/README.md) | 🚧 | ✅ | ✅ |
| [**Multimodal**](docs/pages/features/multimodal/README.md) | ✅ | ✅ | ✅ | | [**Multimodal**](docs/features/multimodal/README.md) | ✅ | ✅ | ✅ |
| [**Tool Calling**](docs/pages/agents/tool-calling.md) | ✅ | ✅ | ✅ | | [**Tool Calling**](docs/agents/tool-calling.md) | ✅ | ✅ | ✅ |
> **[Full Feature Matrix →](docs/pages/reference/feature-matrix.md)** — Detailed compatibility including LoRA, Request Migration, Speculative Decoding, and feature interactions. > **[Full Feature Matrix →](docs/reference/feature-matrix.md)** — Detailed compatibility including LoRA, Request Migration, Speculative Decoding, and feature interactions.
## Dynamo Architecture ## Dynamo Architecture
...@@ -66,7 +66,7 @@ Built in Rust for performance and Python for extensibility, Dynamo is fully open ...@@ -66,7 +66,7 @@ Built in Rust for performance and Python for extensibility, Dynamo is fully open
<img src="./docs/assets/img/frontpage-architecture.png" alt="Dynamo architecture" width="600" /> <img src="./docs/assets/img/frontpage-architecture.png" alt="Dynamo architecture" width="600" />
</p> </p>
> **[Architecture Deep Dive →](docs/pages/design-docs/architecture.md)** > **[Architecture Deep Dive →](docs/design-docs/architecture.md)**
## Latest News ## Latest News
...@@ -87,7 +87,7 @@ Want to help shape the future of distributed LLM inference? See the **[Contribut ...@@ -87,7 +87,7 @@ Want to help shape the future of distributed LLM inference? See the **[Contribut
# Local Quick Start # Local Quick Start
The following examples require a few system level packages. The following examples require a few system level packages.
Recommended to use Ubuntu 24.04 with a x86_64 CPU. See [docs/pages/reference/support-matrix.md](docs/pages/reference/support-matrix.md) Recommended to use Ubuntu 24.04 with a x86_64 CPU. See [docs/reference/support-matrix.md](docs/reference/support-matrix.md)
## Install Dynamo ## Install Dynamo
...@@ -108,7 +108,7 @@ docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/vllm-runt ...@@ -108,7 +108,7 @@ docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/vllm-runt
> **Tip:** To run frontend and worker in the same container, either run processes in background with `&` (see below), or open a second terminal and use `docker exec -it <container_id> bash`. > **Tip:** To run frontend and worker in the same container, either run processes in background with `&` (see below), or open a second terminal and use `docker exec -it <container_id> bash`.
See [Release Artifacts](docs/pages/reference/release-artifacts.md#container-images) for available versions. See [Release Artifacts](docs/reference/release-artifacts.md#container-images) for available versions.
### Option B: Install from PyPI ### Option B: Install from PyPI
...@@ -143,7 +143,7 @@ pip install torch==2.9.0 torchvision --index-url https://download.pytorch.org/wh ...@@ -143,7 +143,7 @@ pip install torch==2.9.0 torchvision --index-url https://download.pytorch.org/wh
pip install --pre --extra-index-url https://pypi.nvidia.com "ai-dynamo[trtllm]" pip install --pre --extra-index-url https://pypi.nvidia.com "ai-dynamo[trtllm]"
``` ```
> **Note:** TensorRT-LLM requires `pip` due to a transitive Git URL dependency that `uv` doesn't resolve. We recommend using the [TensorRT-LLM container](docs/pages/reference/release-artifacts.md#container-images) for broader compatibility. > **Note:** TensorRT-LLM requires `pip` due to a transitive Git URL dependency that `uv` doesn't resolve. We recommend using the [TensorRT-LLM container](docs/reference/release-artifacts.md#container-images) for broader compatibility.
**vLLM** **vLLM**
...@@ -222,7 +222,7 @@ For production deployments on Kubernetes clusters with multiple GPUs. ...@@ -222,7 +222,7 @@ For production deployments on Kubernetes clusters with multiple GPUs.
## Prerequisites ## Prerequisites
- Kubernetes cluster with GPU nodes - Kubernetes cluster with GPU nodes
- [Dynamo Platform installed](docs/pages/kubernetes/README.md) - [Dynamo Platform installed](docs/kubernetes/README.md)
- HuggingFace token for model downloads - HuggingFace token for model downloads
## Production Recipes ## Production Recipes
...@@ -348,8 +348,8 @@ python3 -m dynamo.frontend ...@@ -348,8 +348,8 @@ python3 -m dynamo.frontend
Dynamo provides comprehensive benchmarking tools: Dynamo provides comprehensive benchmarking tools:
- **[Benchmarking Guide](docs/pages/benchmarks/benchmarking.md)** – Compare deployment topologies using AIPerf - **[Benchmarking Guide](docs/benchmarks/benchmarking.md)** – Compare deployment topologies using AIPerf
- **[SLA-Driven Deployments](docs/pages/components/planner/planner-guide.md)** – Optimize deployments to meet SLA requirements - **[SLA-Driven Deployments](docs/components/planner/planner-guide.md)** – Optimize deployments to meet SLA requirements
## Frontend OpenAPI Specification ## Frontend OpenAPI Specification
...@@ -359,11 +359,11 @@ The OpenAI-compatible frontend exposes an OpenAPI 3 spec at `/openapi.json`. To ...@@ -359,11 +359,11 @@ The OpenAI-compatible frontend exposes an OpenAPI 3 spec at `/openapi.json`. To
cargo run -p dynamo-llm --bin generate-frontend-openapi cargo run -p dynamo-llm --bin generate-frontend-openapi
``` ```
This writes to `docs/pages/reference/api/openapi.json`. This writes to `docs/reference/api/openapi.json`.
## Service Discovery and Messaging ## Service Discovery and Messaging
Dynamo uses TCP for inter-component communication. On Kubernetes, native resources ([CRDs + EndpointSlices](docs/pages/kubernetes/service-discovery.md)) handle service discovery. External services are optional for most deployments: Dynamo uses TCP for inter-component communication. On Kubernetes, native resources ([CRDs + EndpointSlices](docs/kubernetes/service-discovery.md)) handle service discovery. External services are optional for most deployments:
| Deployment | etcd | NATS | Notes | | Deployment | etcd | NATS | Notes |
|------------|------|------|-------| |------------|------|------|-------|
...@@ -389,11 +389,11 @@ See [TRT-LLM on Slurm](examples/basics/multinode/trtllm/README.md) for deploymen ...@@ -389,11 +389,11 @@ See [TRT-LLM on Slurm](examples/basics/multinode/trtllm/README.md) for deploymen
- [10/16] [How Baseten achieved 2x faster inference with NVIDIA Dynamo](https://www.baseten.co/blog/how-baseten-achieved-2x-faster-inference-with-nvidia-dynamo/) - [10/16] [How Baseten achieved 2x faster inference with NVIDIA Dynamo](https://www.baseten.co/blog/how-baseten-achieved-2x-faster-inference-with-nvidia-dynamo/)
<!-- Reference links for Feature Compatibility Matrix --> <!-- Reference links for Feature Compatibility Matrix -->
[disagg]: docs/pages/design-docs/disagg-serving.md [disagg]: docs/design-docs/disagg-serving.md
[kv-routing]: docs/pages/components/router/README.md [kv-routing]: docs/components/router/README.md
[planner]: docs/pages/components/planner/planner-guide.md [planner]: docs/components/planner/planner-guide.md
[kvbm]: docs/pages/components/kvbm/README.md [kvbm]: docs/components/kvbm/README.md
[mm]: examples/multimodal/ [mm]: examples/multimodal/
[migration]: docs/pages/fault-tolerance/request-migration.md [migration]: docs/fault-tolerance/request-migration.md
[lora]: examples/backends/vllm/deploy/lora/README.md [lora]: examples/backends/vllm/deploy/lora/README.md
[tools]: docs/pages/agents/tool-calling.md [tools]: docs/agents/tool-calling.md
...@@ -20,7 +20,7 @@ This directory contains benchmarking scripts and tools for performance evaluatio ...@@ -20,7 +20,7 @@ This directory contains benchmarking scripts and tools for performance evaluatio
## Quick Start ## Quick Start
### Benchmark a Dynamo Deployment ### Benchmark a Dynamo Deployment
First, deploy your DynamoGraphDeployment using the [deployment documentation](../docs/pages/kubernetes/), then: First, deploy your DynamoGraphDeployment using the [deployment documentation](../docs/kubernetes/), then:
```bash ```bash
# Port-forward your deployment to http://localhost:8000 # Port-forward your deployment to http://localhost:8000
...@@ -71,4 +71,4 @@ Detailed information is provided in the `prefix_data_generator` directory. ...@@ -71,4 +71,4 @@ Detailed information is provided in the `prefix_data_generator` directory.
## Comprehensive Guide ## Comprehensive Guide
For detailed documentation, configuration options, and advanced usage, see the [complete benchmarking guide](../docs/pages/benchmarks/benchmarking.md). For detailed documentation, configuration options, and advanced usage, see the [complete benchmarking guide](../docs/benchmarks/benchmarking.md).
...@@ -165,7 +165,7 @@ To see all available router arguments, run: ...@@ -165,7 +165,7 @@ To see all available router arguments, run:
python -m dynamo.frontend --help python -m dynamo.frontend --help
``` ```
For detailed explanations of router arguments (especially KV cache routing parameters), see the [Router Guide](../../docs/pages/components/router/router-guide.md). For detailed explanations of router arguments (especially KV cache routing parameters), see the [Router Guide](../../docs/components/router/router-guide.md).
> [!Note] > [!Note]
> If you're unsure whether your backend engines correctly emit KV events for certain models (e.g., hybrid models like gpt-oss or nemotron nano 2), use the `--no-kv-events` flag to disable KV event tracking and use approximate KV indexing instead: > If you're unsure whether your backend engines correctly emit KV events for certain models (e.g., hybrid models like gpt-oss or nemotron nano 2), use the `--no-kv-events` flag to disable KV event tracking and use approximate KV indexing instead:
...@@ -184,7 +184,7 @@ When you launch prefill workers using `run_engines.sh --prefill`, the frontend a ...@@ -184,7 +184,7 @@ When you launch prefill workers using `run_engines.sh --prefill`, the frontend a
- Uses the same routing mode as the frontend's `--router-mode` setting - Uses the same routing mode as the frontend's `--router-mode` setting
- Seamlessly integrates with your decode workers for token generation - Seamlessly integrates with your decode workers for token generation
No additional configuration is needed - simply launch both decode and prefill workers, and the system handles the rest. See the [Router Guide](../../docs/pages/components/router/router-guide.md#disaggregated-serving) for more details. No additional configuration is needed - simply launch both decode and prefill workers, and the system handles the rest. See the [Router Guide](../../docs/components/router/router-guide.md#disaggregated-serving) for more details.
> [!Note] > [!Note]
> The unified frontend with automatic prefill routing is currently enabled for vLLM and TensorRT-LLM backends. For SGLang (work in progress), you need to launch a separate standalone router as the prefill router targeting the prefill endpoints. See example script: [`examples/backends/sglang/launch/disagg_router.sh`](../../examples/backends/sglang/launch/disagg_router.sh) > The unified frontend with automatic prefill routing is currently enabled for vLLM and TensorRT-LLM backends. For SGLang (work in progress), you need to launch a separate standalone router as the prefill router targeting the prefill endpoints. See example script: [`examples/backends/sglang/launch/disagg_router.sh`](../../examples/backends/sglang/launch/disagg_router.sh)
......
...@@ -25,9 +25,9 @@ This directory contains the core components that make up the Dynamo inference fr ...@@ -25,9 +25,9 @@ This directory contains the core components that make up the Dynamo inference fr
Dynamo supports multiple inference engines, each with their own deployment configurations and capabilities: Dynamo supports multiple inference engines, each with their own deployment configurations and capabilities:
- **[vLLM](/docs/pages/backends/vllm/README.md)** - Full-featured vLLM integration with disaggregated serving, KV-aware routing, SLA-based planning, native KV cache events, and NIXL-based transfer mechanisms - **[vLLM](/docs/backends/vllm/README.md)** - Full-featured vLLM integration with disaggregated serving, KV-aware routing, SLA-based planning, native KV cache events, and NIXL-based transfer mechanisms
- **[SGLang](/docs/pages/backends/sglang/README.md)** - SGLang engine integration with ZMQ-based communication, supporting disaggregated serving and KV-aware routing - **[SGLang](/docs/backends/sglang/README.md)** - SGLang engine integration with ZMQ-based communication, supporting disaggregated serving and KV-aware routing
- **[TensorRT-LLM](/docs/pages/backends/trtllm/README.md)** - TensorRT-LLM integration with disaggregated serving capabilities and TensorRT acceleration - **[TensorRT-LLM](/docs/backends/trtllm/README.md)** - TensorRT-LLM integration with disaggregated serving capabilities and TensorRT acceleration
Each engine provides launch and deploy scripts for different deployment patterns in the [examples](../examples/backends/) folder. Each engine provides launch and deploy scripts for different deployment patterns in the [examples](../examples/backends/) folder.
......
...@@ -5,4 +5,4 @@ ...@@ -5,4 +5,4 @@
The API gateway for serving LLM inference requests with OpenAI-compatible HTTP and KServe gRPC endpoints. The API gateway for serving LLM inference requests with OpenAI-compatible HTTP and KServe gRPC endpoints.
See [docs/pages/components/frontend/](../../../../docs/pages/components/frontend/) for documentation. See [docs/components/frontend/](../../../../docs/components/frontend/) for documentation.
...@@ -67,7 +67,7 @@ python -m dynamo.mocker \ ...@@ -67,7 +67,7 @@ python -m dynamo.mocker \
The profile results directory should contain `selected_prefill_interpolation/` and `selected_decode_interpolation/` subdirectories with `raw_data.npz` files. This works seamlessly in Kubernetes where profile data is mounted via ConfigMap or PersistentVolume. The profile results directory should contain `selected_prefill_interpolation/` and `selected_decode_interpolation/` subdirectories with `raw_data.npz` files. This works seamlessly in Kubernetes where profile data is mounted via ConfigMap or PersistentVolume.
To generate profiling data for your own model/hardware configuration, run the profiler (see [SLA-driven profiling documentation](../../../../docs/pages/components/profiler/profiler-guide.md) for details): To generate profiling data for your own model/hardware configuration, run the profiler (see [SLA-driven profiling documentation](../../../../docs/components/profiler/profiler-guide.md) for details):
```bash ```bash
python components/src/dynamo/profiler/profile_sla.py \ python components/src/dynamo/profiler/profile_sla.py \
......
...@@ -42,5 +42,5 @@ When both modes are enabled, throughput-based scaling provides a lower bound on ...@@ -42,5 +42,5 @@ When both modes are enabled, throughput-based scaling provides a lower bound on
## Documentation ## Documentation
- **User docs**: [Planner Guide](../../../../docs/pages/components/planner/planner-guide.md) (deployment, configuration, examples) - **User docs**: [Planner Guide](../../../../docs/components/planner/planner-guide.md) (deployment, configuration, examples)
- **Design docs**: [Planner Design](../../../../docs/pages/design-docs/planner-design.md) (architecture, algorithms) - **Design docs**: [Planner Design](../../../../docs/design-docs/planner-design.md) (architecture, algorithms)
...@@ -29,7 +29,7 @@ logger = logging.getLogger(__name__) ...@@ -29,7 +29,7 @@ logger = logging.getLogger(__name__)
MISSING_PROFILING_DATA_ERROR_MESSAGE = ( MISSING_PROFILING_DATA_ERROR_MESSAGE = (
"SLA-Planner requires pre-deployment profiling results to run.\n" "SLA-Planner requires pre-deployment profiling results to run.\n"
"Please follow /docs/pages/components/profiler/profiler-guide.md to run the profiling first,\n" "Please follow /docs/components/profiler/profiler-guide.md to run the profiling first,\n"
"and make sure the profiling results are present in --profile-results-dir." "and make sure the profiling results are present in --profile-results-dir."
) )
......
...@@ -6,8 +6,8 @@ SPDX-License-Identifier: Apache-2.0 ...@@ -6,8 +6,8 @@ SPDX-License-Identifier: Apache-2.0
# Profiler # Profiler
Documentation for the Dynamo Profiler has moved to [docs/pages/components/profiler/](../../../../docs/pages/components/profiler/README.md). Documentation for the Dynamo Profiler has moved to [docs/components/profiler/](../../../../docs/components/profiler/README.md).
- [Profiler Overview](../../../../docs/pages/components/profiler/README.md) - [Profiler Overview](../../../../docs/components/profiler/README.md)
- [Profiler Guide](../../../../docs/pages/components/profiler/profiler-guide.md) - [Profiler Guide](../../../../docs/components/profiler/profiler-guide.md)
- [Profiler Examples](../../../../docs/pages/components/profiler/profiler-examples.md) - [Profiler Examples](../../../../docs/components/profiler/profiler-examples.md)
...@@ -620,7 +620,7 @@ def create_gradio_interface( ...@@ -620,7 +620,7 @@ def create_gradio_interface(
> 📝 **Note:** The dotted red line in the prefill and decode charts are default TTFT and ITL SLAs if not specified. > 📝 **Note:** The dotted red line in the prefill and decode charts are default TTFT and ITL SLAs if not specified.
> ⚠️ **Warning:** The TTFT values here represent the ideal case when requests arrive uniformly, minimizing queueing. Real-world TTFT may be higher than profiling results. To mitigate the issue, planner uses [correction factors](https://github.com/ai-dynamo/dynamo/blob/main/docs/pages/design-docs/planner-design.md#step-2-correction-factor-calculation) to adjust dynamically at runtime. > ⚠️ **Warning:** The TTFT values here represent the ideal case when requests arrive uniformly, minimizing queueing. Real-world TTFT may be higher than profiling results. To mitigate the issue, planner uses [correction factors](https://github.com/ai-dynamo/dynamo/blob/main/docs/design-docs/planner-design.md#step-2-correction-factor-calculation) to adjust dynamically at runtime.
> 💡 **Tip:** Use the GPU cost checkbox and input in the charts section to convert GPU hours to cost. > 💡 **Tip:** Use the GPU cost checkbox and input in the charts section to convert GPU hours to cost.
""" """
......
...@@ -3,7 +3,7 @@ ...@@ -3,7 +3,7 @@
# Standalone Router # Standalone Router
A backend-agnostic standalone KV-aware router service for Dynamo deployments. For details on how KV-aware routing works, see the [Router Guide](/docs/pages/components/router/router-guide.md). A backend-agnostic standalone KV-aware router service for Dynamo deployments. For details on how KV-aware routing works, see the [Router Guide](/docs/components/router/router-guide.md).
## Overview ## Overview
...@@ -29,7 +29,7 @@ python -m dynamo.router \ ...@@ -29,7 +29,7 @@ python -m dynamo.router \
- `--endpoint`: Full endpoint path for workers in the format `namespace.component.endpoint` (e.g., `dynamo.prefill.generate`) - `--endpoint`: Full endpoint path for workers in the format `namespace.component.endpoint` (e.g., `dynamo.prefill.generate`)
**Router Configuration:** **Router Configuration:**
All router options use the `--router-*` prefix (e.g., `--router-block-size`, `--router-kv-overlap-score-weight`, `--router-temperature`, `--router-kv-events` / `--no-router-kv-events`, `--router-replica-sync`, `--router-snapshot-threshold`, `--router-reset-states`, `--router-track-active-blocks` / `--no-router-track-active-blocks`). Legacy names without the prefix (e.g., `--block-size`, `--kv-events`) are still accepted but deprecated. For detailed descriptions, see the [Router Guide](/docs/pages/components/router/router-guide.md). All router options use the `--router-*` prefix (e.g., `--router-block-size`, `--router-kv-overlap-score-weight`, `--router-temperature`, `--router-kv-events` / `--no-router-kv-events`, `--router-replica-sync`, `--router-snapshot-threshold`, `--router-reset-states`, `--router-track-active-blocks` / `--no-router-track-active-blocks`). Legacy names without the prefix (e.g., `--block-size`, `--kv-events`) are still accepted but deprecated. For detailed descriptions, see the [Router Guide](/docs/components/router/router-guide.md).
## Architecture ## Architecture
...@@ -43,7 +43,7 @@ Clients call the `generate` endpoint to stream completions, or call `best_worker ...@@ -43,7 +43,7 @@ Clients call the `generate` endpoint to stream completions, or call `best_worker
## Example: Manual Disaggregated Serving (Alternative Setup) ## Example: Manual Disaggregated Serving (Alternative Setup)
> [!Note] > [!Note]
> **This is an alternative advanced setup.** The recommended approach for disaggregated serving is to use the frontend's automatic prefill routing, which activates when you register workers with `ModelType.Prefill`. See the [Router Guide](/docs/pages/components/router/router-guide.md#disaggregated-serving) for the default setup. > **This is an alternative advanced setup.** The recommended approach for disaggregated serving is to use the frontend's automatic prefill routing, which activates when you register workers with `ModelType.Prefill`. See the [Router Guide](/docs/components/router/router-guide.md#disaggregated-serving) for the default setup.
> >
> Use this manual setup if you need explicit control over prefill routing configuration or want to manage prefill and decode routers separately. > Use this manual setup if you need explicit control over prefill routing configuration or want to manage prefill and decode routers separately.
...@@ -103,7 +103,7 @@ See [`components/src/dynamo/vllm/handlers.py`](../vllm/handlers.py) for a refere ...@@ -103,7 +103,7 @@ See [`components/src/dynamo/vllm/handlers.py`](../vllm/handlers.py) for a refere
## See Also ## See Also
- [Router Guide](/docs/pages/components/router/router-guide.md) - Configuration and tuning for KV-aware routing - [Router Guide](/docs/components/router/router-guide.md) - Configuration and tuning for KV-aware routing
- [Router Design](/docs/pages/design-docs/router-design.md) - Architecture details and event transport modes - [Router Design](/docs/design-docs/router-design.md) - Architecture details and event transport modes
- [Frontend Router](../frontend/README.md) - Main HTTP frontend with integrated routing - [Frontend Router](../frontend/README.md) - Main HTTP frontend with integrated routing
- [Router Benchmarking](/benchmarks/router/README.md) - Performance testing and tuning - [Router Benchmarking](/benchmarks/router/README.md) - Performance testing and tuning
...@@ -230,7 +230,7 @@ text-to-video-diffusion.sh # 1-2 GPUs - Text-to-video (Wan2.1) ...@@ -230,7 +230,7 @@ text-to-video-diffusion.sh # 1-2 GPUs - Text-to-video (Wan2.1)
survives parent kill. Always check `nvidia-smi` after teardown. survives parent kill. Always check `nvidia-smi` after teardown.
For troubleshooting (CuDNN, config.json errors, OOM, disagg connectivity), see For troubleshooting (CuDNN, config.json errors, OOM, disagg connectivity), see
`docs/pages/backends/sglang/sglang-examples.md#troubleshooting`. `docs/backends/sglang/sglang-examples.md#troubleshooting`.
## Adding a New Worker Type ## Adding a New Worker Type
...@@ -268,7 +268,7 @@ Checklist for adding a new worker (e.g., a new modality or serving mode): ...@@ -268,7 +268,7 @@ Checklist for adding a new worker (e.g., a new modality or serving mode):
pass engine=None. Guard any engine access in shared base class code. pass engine=None. Guard any engine access in shared base class code.
- **Rebuild after Rust changes**: If changing registration (register.py interacts with Rust - **Rebuild after Rust changes**: If changing registration (register.py interacts with Rust
bindings), rebuild: `cd lib/bindings/python && maturin develop --uv && cd <root> && uv pip install -e .` bindings), rebuild: `cd lib/bindings/python && maturin develop --uv && cd <root> && uv pip install -e .`
- **Troubleshooting**: See `docs/pages/backends/sglang/sglang-examples.md#troubleshooting` - **Troubleshooting**: See `docs/backends/sglang/sglang-examples.md#troubleshooting`
for CuDNN, config.json, OOM, and disagg connectivity issues. for CuDNN, config.json, OOM, and disagg connectivity issues.
## File Index ## File Index
......
...@@ -12,7 +12,7 @@ HuggingFace Diffusers model), eliminating the need for a --model-type flag. ...@@ -12,7 +12,7 @@ HuggingFace Diffusers model), eliminating the need for a --model-type flag.
Requirements: Requirements:
- tensorrt_llm with visual_gen support (tensorrt_llm._torch.visual_gen). - tensorrt_llm with visual_gen support (tensorrt_llm._torch.visual_gen).
See: https://github.com/NVIDIA/TensorRT-LLM See: https://github.com/NVIDIA/TensorRT-LLM
- See docs/pages/backends/trtllm/README.md for setup instructions. - See docs/backends/trtllm/README.md for setup instructions.
Note on imports: Note on imports:
tensorrt_llm._torch.visual_gen is imported lazily in initialize() because: tensorrt_llm._torch.visual_gen is imported lazily in initialize() because:
......
...@@ -133,7 +133,7 @@ async def init_video_diffusion_worker( ...@@ -133,7 +133,7 @@ async def init_video_diffusion_worker(
raise RuntimeError( raise RuntimeError(
"ModelType.Videos not available in dynamo-runtime. " "ModelType.Videos not available in dynamo-runtime. "
"Video diffusion requires a compatible dynamo-runtime version. " "Video diffusion requires a compatible dynamo-runtime version. "
"See docs/pages/backends/trtllm/README.md for setup instructions." "See docs/backends/trtllm/README.md for setup instructions."
) )
model_type = ModelType.Videos model_type = ModelType.Videos
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment