Unverified Commit 78818dd1 authored by Michael Yao's avatar Michael Yao Committed by GitHub
Browse files

[Docs] Have a try to improve frameworks/streamlit.md (#24841)


Signed-off-by: default avatarwindsonsea <haifeng.yao@daocloud.io>
parent 8e5cdcda
...@@ -6,35 +6,33 @@ It can be quickly integrated with vLLM as a backend API server, enabling powerfu ...@@ -6,35 +6,33 @@ It can be quickly integrated with vLLM as a backend API server, enabling powerfu
## Prerequisites ## Prerequisites
- Setup vLLM environment Set up the vLLM environment by installing all required packages:
## Deploy
- Start the vLLM server with the supported chat completion model, e.g.
```bash ```bash
vllm serve qwen/Qwen1.5-0.5B-Chat pip install vllm streamlit openai
``` ```
- Install streamlit and openai: ## Deploy
```bash 1. Start the vLLM server with a supported chat completion model, e.g.
pip install streamlit openai
```
- Use the script: <gh-file:examples/online_serving/streamlit_openai_chatbot_webserver.py> ```bash
vllm serve Qwen/Qwen1.5-0.5B-Chat
```
- Start the streamlit web UI and start to chat: 1. Use the script: <gh-file:examples/online_serving/streamlit_openai_chatbot_webserver.py>
```bash 1. Start the streamlit web UI and start to chat:
streamlit run streamlit_openai_chatbot_webserver.py
# or specify the VLLM_API_BASE or VLLM_API_KEY ```bash
VLLM_API_BASE="http://vllm-server-host:vllm-server-port/v1" \
streamlit run streamlit_openai_chatbot_webserver.py streamlit run streamlit_openai_chatbot_webserver.py
# start with debug mode to view more details # or specify the VLLM_API_BASE or VLLM_API_KEY
streamlit run streamlit_openai_chatbot_webserver.py --logger.level=debug VLLM_API_BASE="http://vllm-server-host:vllm-server-port/v1" \
``` streamlit run streamlit_openai_chatbot_webserver.py
# start with debug mode to view more details
streamlit run streamlit_openai_chatbot_webserver.py --logger.level=debug
```
![](../../assets/deployment/streamlit-chat.png) ![Chat with vLLM assistant in Streamlit](../../assets/deployment/streamlit-chat.png)
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment