"docs/vscode:/vscode.git/clone" did not exist on "28028dff2fed19e0face08a303b86273d954979a"
Unverified Commit b0925b38 authored by Sherlock Xu's avatar Sherlock Xu Committed by GitHub
Browse files

docs: Add BentoML deployment doc (#3336)


Signed-off-by: default avatarSherlock113 <sherlockxu07@gmail.com>
parent 654865e2
......@@ -73,6 +73,7 @@ Documentation
serving/run_on_sky
serving/deploying_with_kserve
serving/deploying_with_triton
serving/deploying_with_bentoml
serving/deploying_with_docker
serving/serving_with_langchain
serving/metrics
......
.. _deploying_with_bentoml:
Deploying with BentoML
======================
`BentoML <https://github.com/bentoml/BentoML>`_ allows you to deploy a large language model (LLM) server with vLLM as the backend, which exposes OpenAI-compatible endpoints. You can serve the model locally or containerize it as an OCI-complicant image and deploy it on Kubernetes.
For details, see the tutorial `vLLM inference in the BentoML documentation <https://docs.bentoml.com/en/latest/use-cases/large-language-models/vllm.html>`_.
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment