GitLab

feat: Add compound AI python SDK

See merge request dl/triton/triton-distributed!5

`cargo build --locked` won't let you use "1.85.0" if you only have "stable" installed, even if those are the same thing right now.

Signed-off-by: Neelay Shah <neelays@nvidia.com>

Engine, `tio` support and docs.

Proof of concept / experimental.

Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

triton-distributed-llm component and support in tio

Signed-off-by: Harrison Saturley-Hall <454891+saturley-hall@users.noreply.github.com>
Signed-off-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>
Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>

Docs in README

Co-authored-by: Piotr Tarasiewicz Nvidia <ptarasiewicznv@Piotrs-MacBook-Pro.local>
Co-authored-by: nnshah1 <neelays@nvidia.com>
Co-authored-by: alec-flowers <aflowers@nvidia.com>

Co-authored-by: NVShreyas <158103197+NVShreyas@users.noreply.github.com>

Co-authored-by: Alec <35311602+alec-flowers@users.noreply.github.com>

Co-authored-by: Graham King <grahamk@nvidia.com>

Co-authored-by: Ryan Olson <rolson@nvidia.com>

This means we don't need to explain the parts to the users until they are ready. We use what they provide and default the rest.

Allows all of this and more:
- `tio out=tdr://test`
- `tio out=tdr://llama_8b_pool`
- `tio in=tdr://corp_ai_research_group/model_next-20250226`
- `tio out=tdr://AIRE.NIM.migrate.mistralrs.1802`

Python, API, etc all untouched.

Signed-off-by: Piotr Marcinkiewicz <piotrm@nvidia.com>