Unverified Commit d346782c authored by Graham King's avatar Graham King Committed by GitHub
Browse files

chore: Publish Model Deployment Card to NATS (#799)

This will allow an ingress-side pre-processor to see it without needing a model checkout.

Currently pre-processing is done in the worker, which has access to the model deployment card ("MDC") files (`config.json`, `tokenizer.json` and `tokenizer_config.json`) locally. We want to move the pre-processor to the ingress side to support KV routing. That requires ingress side (i.e the HTTP server), on a different machine than the worker to be able to see those three files.

To support that this PR makes the worker upload the contents of those files to the NATS object store, and publishes the MDC with those NATS urls to the key-value store. 

The key-value store has an interface so any store (nats, etcd, redis, etc) can be supported. Implementations for memory and NATS are provided.

Fetching the MDC from the store, doing pre-processing ingress side, and publishing a card backed by a GGUF, are all for a later commit.

Part of #743 
parent 16310b26
...@@ -36,8 +36,6 @@ impl Slug { ...@@ -36,8 +36,6 @@ impl Slug {
Slug::slugify_unique(s.as_ref()) Slug::slugify_unique(s.as_ref())
} }
/* Not currently used but leave it for now
*
/// Turn the string into a valid slug, replacing any not-web-or-nats-safe characters with '-' /// Turn the string into a valid slug, replacing any not-web-or-nats-safe characters with '-'
pub fn slugify(s: &str) -> Slug { pub fn slugify(s: &str) -> Slug {
let out = s let out = s
...@@ -54,7 +52,6 @@ impl Slug { ...@@ -54,7 +52,6 @@ impl Slug {
.collect::<String>(); .collect::<String>();
Slug::new(out) Slug::new(out)
} }
*/
/// Like slugify but also add a four byte hash on the end, in case two different strings slug /// Like slugify but also add a four byte hash on the end, in case two different strings slug
/// to the same thing. /// to the same thing.
......
This diff is collapsed.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment