"examples/vllm/vscode:/vscode.git/clone" did not exist on "ac7e8882a4a5f116f587ce0b58a6f98c3dfa1c4c"
-
Graham King authored
Example: ``` dynamo-run out=<engine> <model> --kv-cache-block-size 64 ``` In a distributed system this goes on the worker node and is propagated to ingress via the model deployment card. Previously hard coded to 16, which is now the default. - Load context_length from model. Closes #1172 - Store context length and KV cache block size in Model Deployment Card #1170
183f2b32
