GitLab

Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>

Co-authored-by: Nicolas Noble <nicolasnoble@users.noreply.github.com>

Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>

Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>

Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>

Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>

Signed-off-by: ishandhanani <ishandhanani@gmail.com>
Co-authored-by: mabdulwahhab <mabdulwahhab@nvidia.com>

Co-authored-by: Harrison Saturley-Hall <454891+saturley-hall@users.noreply.github.com>

Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>

Co-authored-by: Vikram Sharma <vsm2@illinois.edu>
Co-authored-by: Ziqi Fan <ziqif@nvidia.com>
Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>

Co-authored-by: ptarasiewicz@nvidia.com <Piotr Tarasiewicz>

Co-authored-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Sean <choishsean@gmail.com>
Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>

Co-authored-by: Anant Sharma <anants@nvidia.com>

Previously several parts of the stack ensured max tokens (for this single request) was set.

Now only text input sets it (to 8k). Everything else leaves as is, potentially blank. The engines themselves have very small defaults, 16 for vllm and 128 for sglang.

Also fix dynamo-run CUDA startup message to only print if we're using an engine that would benefit from it (mistralrs, llamacpp).

Co-authored-by: Piotr Tarasiewicz <ptarasiewicz@nvidia.com>

Co-authored-by: hongkuanz <hongkuanz@nvidia.com>
Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>
Co-authored-by: Dmitry Tokarev <dtokarev@nvidia.com>