Unverified Commit 49d849b3 authored by Yuan Tang's avatar Yuan Tang Committed by GitHub
Browse files

docs: Add tutorial on deploying vLLM model with KServe (#2586)


Signed-off-by: default avatarYuan Tang <terrytangyuan@gmail.com>
parent 27ca23dc
...@@ -70,6 +70,7 @@ Documentation ...@@ -70,6 +70,7 @@ Documentation
serving/distributed_serving serving/distributed_serving
serving/run_on_sky serving/run_on_sky
serving/deploying_with_kserve
serving/deploying_with_triton serving/deploying_with_triton
serving/deploying_with_docker serving/deploying_with_docker
serving/serving_with_langchain serving/serving_with_langchain
......
.. _deploying_with_kserve:
Deploying with KServe
============================
vLLM can be deployed with `KServe <https://github.com/kserve/kserve>`_ on Kubernetes for highly scalable distributed model serving.
Please see `this guide <https://kserve.github.io/website/latest/modelserving/v1beta1/llm/vllm/>`_ for more details on using vLLM with KServe.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment