GitLab

"echo_core" is an engine that echoes the post-processed request back to you so you can see the template. Good for testing. It needed an extra flag set to work correctly.

It's different enough that I made a new engine vllm0_8 and renamed the previous engine to vllm0_7.

`dynamo-run out=vllm` now expects 0.8. This matches the container change in #690.

For older use `dynamo-run out=vllm0_7`.

Signed-off-by: hhzhang16 <54051230+hhzhang16@users.noreply.github.com>
Co-authored-by: mohammedabdulwahhab <furkhan324@berkeley.edu>

Signed-off-by: Maksim Khadkevich <mkhadkevich@nvidia.com>

Signed-off-by: Tanmay Verma <tanmay2592@gmail.com>
Signed-off-by: Tanmay Verma <tanmayv@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>

Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
Signed-off-by: Pavithra Vijayakrishnan <160681768+pvijayakrish@users.noreply.github.com>
Signed-off-by: Chantal D Gama Rose <cdgamarose@nvidia.com>
Signed-off-by: hhzhang16 <54051230+hhzhang16@users.noreply.github.com>
Co-authored-by: Julien Mancuso <jmancuso@nvidia.com>
Co-authored-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
Co-authored-by: mabdulwahhab <mabdulwahhab@nvidia.com>
Co-authored-by: Tushar Sharma <tusharma@nvidia.com>
Co-authored-by: Jacky <18255193+kthui@users.noreply.github.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>
Co-authored-by: Pavithra Vijayakrishnan <160681768+pvijayakrish@users.noreply.github.com>
Co-authored-by: cdgamarose-nv <cdgamarose@nvidia.com>
Co-authored-by: Anant Sharma <anants@nvidia.com>
Co-authored-by: julienmancuso <161955438+julienmancuso@users.noreply.github.com>
Co-authored-by: Suman Tatiraju <167138127+statiraju@users.noreply.github.com>
Co-authored-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Co-authored-by: Tanmay Verma <tanmayv@nvidia.com>

feat: Extract Common Configs + Log Configs on Init + Add `test_` to `sdk/tests` filenames required for pytest (#434)
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>

#### Overview:

Updated the dynamo run doc `docs/guides/dynamo_run.md`

#### Details:

- Updated the instructions to make it clear which binary to use for built backends
- Reformatted the doc to make it more readable
- Added missing cmake library for ubuntu
Signed-off-by: Chantal D Gama Rose <cdgamarose@nvidia.com>

Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>

Put the arguments in a JSON file:
```
{
    "dtype": "half",
    "trust_remote_code": true
}
```

Pass it like this:
```
dynamo-run out=sglang ~/llm_models/Llama-3.2-3B-Instruct --extra-engine-args sglang_extra.json
```

Requested here https://github.com/ai-dynamo/dynamo/issues/290 (`dtype`) and here https://github.com/ai-dynamo/dynamo/issues/360 (`trust_remote_code`).

This lets us do:
```
dynamo-run out=llamacpp <gguf_file>
```

Previously a `--model-config <hf-repo>` was also required, to configure our tokenizer.

Co-authored-by: Olga Andreeva <oandreeva@oandreeva-mlt.client.nvidia.com>

Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>

It hardly slows the build down, and it makes things run much faster. That allows us to switch to the debug (default) profile for development, and keep the release profile for, well, releasing.

Motivated by changes in https://github.com/ai-dynamo/dynamo/pull/279

Co-authored-by: mabdulwahhab <mabdulwahhab@nvidia.com>

Co-authored-by: Dmitry Tokarev <dtokarev@nvidia.com>

Co-authored-by: mabdulwahhab <mabdulwahhab@nvidia.com>
Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>

Co-authored-by: Anant Sharma <anants@nvidia.com>

Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>

Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>

Co-authored-by: Harrison Saturley-Hall <454891+saturley-hall@users.noreply.github.com>

Co-authored-by: Vikram Sharma <vsm2@illinois.edu>
Co-authored-by: Ziqi Fan <ziqif@nvidia.com>
Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>

Co-authored-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Sean <choishsean@gmail.com>
Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>