"vllm/vscode:/vscode.git/clone" did not exist on "53fa45739176e81b9078f7e3516a8b11fda161de"
deploying_with_kserve.rst 362 Bytes
Newer Older
1
2
3
4
5
6
7
.. _deploying_with_kserve:

Deploying with KServe
============================

vLLM can be deployed with `KServe <https://github.com/kserve/kserve>`_ on Kubernetes for highly scalable distributed model serving.

8
Please see `this guide <https://kserve.github.io/website/latest/modelserving/v1beta1/llm/huggingface/>`_ for more details on using vLLM with KServe.