GitLab

Signed-off-by: Tanmay Verma <tanmay2592@gmail.com>
Co-authored-by: Dmitry Tokarev <dtokarev@nvidia.com>

Signed-off-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>

As a first step towards KV routing:
- introduce a `--router-mode` in dynamo-run that only does random and round-robin right now. Not that interesting yet.
- Make the vllm engine publish the KV events received from our patched vllm.

Now we "just" need to connect the two. Easy right?

Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>

Co-authored-by: finofliu <finofliu@tencent.com>

Co-authored-by: mabdulwahhab <mabdulwahhab@nvidia.com>

Also upgrade the cargo resolver to v3, the default.

New clippy lints:
- `next_back()` instead of `last()` for a double-ended iterator. That avoids walking the whole list.
- ` repeat_n` instead of `repeat.take`. That avoids cloning.
- Doc indenting

Adds `@dynamo_worker(static = True)` to create a static worker which has a predictable name and hence does not require discovery or `etcd` to be running. There can only be a single static worker per namespace / component / endpoint trio.

This contrasts with the default dynamic `dynamo_worker` endpoints we have now, which get a unique random name (based on namespace/component/endpoint), and are discovered by ingress components using etcd.

Also change the hello_world example to use `dynamo_worker(static = True)` so that it is exercised and demonstrated somewhere.

For NIM.

Moved all of `lib/llm/src/engines` to their own crates as e.g. `lib/engines/mistralrs`. This will allow publishing of the `dynamo-llm` crate as it won't have any github dependencies.

The only engines in dynamo-llm will be the demo `echo` ones.
Co-authored-by: Graham King <grahamk@nvidia.com>

Co-authored-by: Neelay Shah <neelays@nvidia.com>

Signed-off-by: hanweisen <hanweisen_yewu@cmss.chinamobile.com>

Co-authored-by: mabdulwahhab <mabdulwahhab@nvidia.com>

Co-authored-by: Ryan Olson <ryanolson@users.noreply.github.com>

Co-authored-by: hongkuanz <hongkuanz@nvidia.com>