GitLab

Signed-off-by: Tianer Zhou <ezhoureal@gmail.com>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>

Signed-off-by: PeaBrane <yanrpei@gmail.com>
Signed-off-by: Yan Ru Pei <yanrpei@gmail.com>
Signed-off-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: jain-ria <riajain@NVIDIA.com>

Signed-off-by: Hongkuan Zhou <tedzhouhk@gmail.com>

Signed-off-by: Kristen Kelleher <kkelleher@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

Signed-off-by: Tianer Zhou <ezhoureal@gmail.com>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>

Publish `generation_config.json` from worker to ingress, as part of Model Deployment Card. That allows ingress to read key fields out of it. Gemma 3 4B+ has some important information that's only in there.

Signed-off-by: Kristen Kelleher <kkelleher@nvidia.com>
- Content, format, and structural changes to the Dynamo docs for 0.3.0. 
- Includes copyediting and the first batch of changes from the DMO review.

This PR modifies the mistralrs engine to ensure that the maximum output token length never exceeds the context length provided.

Signed-off-by: Hongkuan Zhou <tedzhouhk@gmail.com>
Co-authored-by: jothomson <jwillthomson19@gmail.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

Do not include by default as it needs libgomp1 at runtime. Add a feature to enable it at build time.

It was confusing to have two names for one type.

This tidy up started in #1064 , is now complete.

Unify them with all our other logs, so we can filter with DYN_LOG, they will eventually go to the log aggregation, etc.

Previously `mistral.rs` was the default engine for both safetensors and GGUF models. Now it is only the default for safetensors, `llama.cpp` becomes the default for GGUF.

Why?

- Since #1177 `llama.cpp` is built-in by default, so we can switch.
- `llama.cpp` is very very good at running GGUF (but can't run other types of model), so we should switch.

Dynamo's multi-engine support gives us a secret super-power: we can use the best engine for this specific format or model.

We can still run GGUF with mistralrs by doing `out=mistralrs`.