Unverified Commit d6c49779 authored by Ayush Agarwal's avatar Ayush Agarwal Committed by GitHub
Browse files

chore: unify filesystem usage across all frameworks and workers (#6391)


Signed-off-by: default avatarayushag <ayushag@nvidia.com>
parent 6dd3ce2e
...@@ -238,7 +238,7 @@ The pipeline type is **auto-detected** from the model's `model_index.json` — n ...@@ -238,7 +238,7 @@ The pipeline type is **auto-detected** from the model's `model_index.json` — n
python -m dynamo.trtllm \ python -m dynamo.trtllm \
--modality video_diffusion \ --modality video_diffusion \
--model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \ --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
--output-dir /tmp/videos --media-output-fs-url file:///tmp/dynamo_media
``` ```
### API Endpoint ### API Endpoint
...@@ -263,7 +263,7 @@ curl -X POST http://localhost:8000/v1/videos \ ...@@ -263,7 +263,7 @@ curl -X POST http://localhost:8000/v1/videos \
| Flag | Description | Default | | Flag | Description | Default |
|------|-------------|---------| |------|-------------|---------|
| `--output-dir` | Directory for generated videos | `/tmp/dynamo_videos` | | `--media-output-fs-url` | Filesystem URL for storing generated media | `file:///tmp/dynamo_media` |
| `--default-height` | Default video height | `480` | | `--default-height` | Default video height | `480` |
| `--default-width` | Default video width | `832` | | `--default-width` | Default video width | `832` |
| `--default-num-frames` | Default frame count | `81` | | `--default-num-frames` | Default frame count | `81` |
......
...@@ -139,7 +139,7 @@ The response returns a video URL or base64 data depending on `response_format`: ...@@ -139,7 +139,7 @@ The response returns a video URL or base64 data depending on `response_format`:
"object": "video", "object": "video",
"model": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers", "model": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
"status": "completed", "status": "completed",
"data": [{"url": "/tmp/generated_video.mp4"}] "data": [{"url": "file:///tmp/dynamo_media/videos/req-abc123.mp4"}]
} }
``` ```
...@@ -162,6 +162,24 @@ The `/v1/videos` endpoint also accepts NVIDIA extensions via the `nvext` field f ...@@ -162,6 +162,24 @@ The `/v1/videos` endpoint also accepts NVIDIA extensions via the `nvext` field f
| `--output-modalities <modality>` | Output modality: `text`, `image`, or `video` | | `--output-modalities <modality>` | Output modality: `text`, `image`, or `video` |
| `--stage-configs-path <path>` | Path to stage config YAML (optional; vLLM-Omni uses model defaults if omitted) | | `--stage-configs-path <path>` | Path to stage config YAML (optional; vLLM-Omni uses model defaults if omitted) |
| `--connector none` | Disable KV connector (recommended for omni workers) | | `--connector none` | Disable KV connector (recommended for omni workers) |
| `--media-output-fs-url <url>` | Filesystem URL for storing generated media (default: `file:///tmp/dynamo_media`) |
| `--media-output-http-url <url>` | Base URL for rewriting media paths in responses (optional) |
## Storage Configuration
Generated images and videos are stored via [fsspec](https://filesystem-spec.readthedocs.io/), which supports local filesystems, S3, GCS, and Azure Blob.
By default, media is written to the local filesystem at `file:///tmp/dynamo_media`. To use cloud storage:
```bash
bash examples/backends/vllm/launch/agg_omni_video.sh \
--media-output-fs-url s3://my-bucket/media \
--media-output-http-url https://cdn.example.com/media
```
When `--media-output-http-url` is set, response URLs are rewritten as `{base-url}/{storage-path}` (e.g., `https://cdn.example.com/media/videos/req-id.mp4`). When unset, the raw filesystem path is returned.
For S3 credential configuration, set the standard AWS environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`) or use IAM roles. See the [fsspec S3 docs](https://s3fs.readthedocs.io/en/latest/#credentials) for details.
## Stage Configuration ## Stage Configuration
......
...@@ -15,7 +15,7 @@ trap cleanup EXIT INT TERM ...@@ -15,7 +15,7 @@ trap cleanup EXIT INT TERM
# Defaults # Defaults
WAN_SIZE="1b" WAN_SIZE="1b"
FS_URL="file:///tmp/dynamo_videos" FS_URL="file:///tmp/dynamo_media"
HTTP_PORT="${HTTP_PORT:-8000}" HTTP_PORT="${HTTP_PORT:-8000}"
NUM_FRAMES=17 NUM_FRAMES=17
HEIGHT=480 HEIGHT=480
...@@ -61,7 +61,7 @@ while [[ $# -gt 0 ]]; do ...@@ -61,7 +61,7 @@ while [[ $# -gt 0 ]]; do
echo "" echo ""
echo "Options:" echo "Options:"
echo " --wan-size <1b|14b> Model size (default: 1b)" echo " --wan-size <1b|14b> Model size (default: 1b)"
echo " --fs-url <url> Filesystem URL for video storage (default: file:///tmp/dynamo_videos)" echo " --fs-url <url> Filesystem URL for media storage (default: file:///tmp/dynamo_media)"
echo " --http-port <port> Frontend HTTP port (default: 8000)" echo " --http-port <port> Frontend HTTP port (default: 8000)"
echo " --num-frames <n> Default frame count for health check (default: 17)" echo " --num-frames <n> Default frame count for health check (default: 17)"
echo " --height <n> Video height (default: 480)" echo " --height <n> Video height (default: 480)"
...@@ -139,7 +139,7 @@ python3 -m dynamo.sglang \ ...@@ -139,7 +139,7 @@ python3 -m dynamo.sglang \
--served-model-name "$MODEL_PATH" \ --served-model-name "$MODEL_PATH" \
--tp "$TP_SIZE" \ --tp "$TP_SIZE" \
--video-generation-worker \ --video-generation-worker \
--video-generation-fs-url "$FS_URL" \ --media-output-fs-url "$FS_URL" \
--trust-remote-code \ --trust-remote-code \
--skip-tokenizer-init \ --skip-tokenizer-init \
--enable-metrics \ --enable-metrics \
......
...@@ -40,4 +40,5 @@ DYN_SYSTEM_PORT=${DYN_SYSTEM_PORT:-8081} \ ...@@ -40,4 +40,5 @@ DYN_SYSTEM_PORT=${DYN_SYSTEM_PORT:-8081} \
--omni \ --omni \
--connector none \ --connector none \
--output-modalities image \ --output-modalities image \
--media-output-fs-url file:///tmp/dynamo_media \
"${EXTRA_ARGS[@]}" "${EXTRA_ARGS[@]}"
...@@ -42,4 +42,5 @@ DYN_SYSTEM_PORT=${DYN_SYSTEM_PORT:-8081} \ ...@@ -42,4 +42,5 @@ DYN_SYSTEM_PORT=${DYN_SYSTEM_PORT:-8081} \
--omni \ --omni \
--connector none \ --connector none \
--output-modalities video \ --output-modalities video \
--media-output-fs-url file:///tmp/dynamo_media \
"${EXTRA_ARGS[@]}" "${EXTRA_ARGS[@]}"
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment