GitLab

Signed-off-by: Yan Ru Pei <yanrpei@gmail.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>

Publish `generation_config.json` from worker to ingress, as part of Model Deployment Card. That allows ingress to read key fields out of it. Gemma 3 4B+ has some important information that's only in there.

Need to reinstall the rdma-core and libibverbs to use RDMA devices.
Also, docker container can be built using a recent version of UCX for
EFA support.
Signed-off-by: Adit Ranadive <aranadive@nvidia.com>

feat: set model specific prompt templates in the multimodal config files, add documentation for multimodal example deployment (#1366)

Signed-off-by: Kristen Kelleher <kkelleher@nvidia.com>
- Content, format, and structural changes to the Dynamo docs for 0.3.0. 
- Includes copyediting and the first batch of changes from the DMO review.

To talk to the vllm/sglang/trtllm engine we previously hardcoded an endpoint. The user never sees it so it doesn't matter which one.

However if you try to run _two_ instances of Dynamo on one machine they will conflict.

Use a UUID as the component name to resolve that.

Part of the solution for:
https://github.com/ai-dynamo/dynamo/issues/1073

This PR modifies the mistralrs engine to ensure that the maximum output token length never exceeds the context length provided.

Creates a README.md file for Connect.

The README contains and overview, examples w/ diagrams, and documents the important classes.

The README is not intended to be comprehensive.
Instead it's meant to be more of a "getting started" or "learn the basics".
More comprehensive information / documentation is available from the inline comments / documentation.

Additionally, updates the Multimodal Example:

Moves the remote and local prefill code from the generate method into remote_prefill and local_prefill respectively.
Code changes made.
Replaces reference to "agent" with "worker" for consistency reasons throughout the inline documentation.
Only comments updated. No code changes made.
The intention of this change is improve readability of the example code and to provide clearer examples to reference from within documentation.

DIS-101

Signed-off-by: Hongkuan Zhou <tedzhouhk@gmail.com>
Co-authored-by: jothomson <jwillthomson19@gmail.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

Signed-off-by: hhzhang16 <54051230+hhzhang16@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

Do not include by default as it needs libgomp1 at runtime. Add a feature to enable it at build time.

This allows building:
-  only `mistral.rs` engine: `--no-default-features --features mistralrs`  
- or only `llama.cpp` engine: `--no-default-features --features llamacpp`. 

Since llama.cpp became a default we'd only tested building both at once. The docs already said we supported that but there was some combo of Rust features that didn't build. This is the fix.