GitLab

Co-authored-by: Cursor <cursoragent@cursor.com>

Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

Signed-off-by: Ryan Olson <rolson@nvidia.com>

Signed-off-by: Anant Sharma <anants@nvidia.com>

Signed-off-by: Graham King <grahamk@nvidia.com>

Signed-off-by: pvijayakrish <pvijayakrish@nvidia.com>

Signed-off-by: Anant Sharma <anants@nvidia.com>

Signed-off-by: Anant Sharma <anants@nvidia.com>

Signed-off-by: Graham King <grahamk@nvidia.com>

Signed-off-by: Graham King <grahamk@nvidia.com>

Signed-off-by: Graham King <grahamk@nvidia.com>

Signed-off-by: Anant Sharma <anants@nvidia.com>

Signed-off-by: Graham King <grahamk@nvidia.com>

Signed-off-by: Graham King <grahamk@nvidia.com>

Signed-off-by: Tushar Sharma <tusharma@nvidia.com>

Signed-off-by: Graham King <grahamk@nvidia.com>

Signed-off-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>

Signed-off-by: Graham King <grahamk@nvidia.com>

Signed-off-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>

Signed-off-by: Graham King <grahamk@nvidia.com>

Signed-off-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com>

Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>

Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>

feat: add a hierarchical Prometheus MetricsRegistry trait for DistributedRuntime, Namespace, Components, and Endpoint (#2008)
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Ryan Olson <rolson@nvidia.com>

Allow both password or TLS auth, if none of these is provided fallback to no auth

Closes #657

Signed-off-by: Harrison Saturley-Hall <454891+saturley-hall@users.noreply.github.com>

This will allow an ingress-side pre-processor to see it without needing a model checkout.

Currently pre-processing is done in the worker, which has access to the model deployment card ("MDC") files (`config.json`, `tokenizer.json` and `tokenizer_config.json`) locally. We want to move the pre-processor to the ingress side to support KV routing. That requires ingress side (i.e the HTTP server), on a different machine than the worker to be able to see those three files.

To support that this PR makes the worker upload the contents of those files to the NATS object store, and publishes the MDC with those NATS urls to the key-value store. 

The key-value store has an interface so any store (nats, etcd, redis, etc) can be supported. Implementations for memory and NATS are provided.

Fetching the MDC from the store, doing pre-processing ingress side, and publishing a card backed by a GGUF, are all for a later commit.

Part of #743