Commit a013f41a authored by laibao's avatar laibao
Browse files

Update README.md to standardize model path usage and enhance vllm serving...

Update README.md to standardize model path usage and enhance vllm serving command with additional parameters and template details.
parent 5c8b9046
Pipeline #2984 failed with stages
in 0 seconds
......@@ -139,7 +139,8 @@ pip install gradio
2.1 启动gradio服务,根据提示操作
```
python gradio_openai_vlm_webserver.py --model "/mnt/data/llm-models/llava/llava-1.5-7b-hf" --model-url http://localhost:8000/v1 --host "0.0.0.0" --port 8001
python gradio_openai_vlm_webserver.py --model model_path --model-url http://localhost:8000/v1 --host "0.0.0.0" --port 8001"
```
2.2 更改文件权限
......@@ -161,13 +162,14 @@ ssh -L 8000:计算节点IP:8000 -L 8001:计算节点IP:8001 用户名@登录节
`cd examples`
```
vllm serve /mnt/data/llm-models/llava/llava-1.5-7b-hf --chat-template template_llava.jinja --port 8000 --host"0.0.0.0"
vllm serve model_path --trust-remote-code --port 8000 --host 0.0.0.0 --allowed-local-media-path xxxx --hf-overrides '{"architectures": ["GLM4VForCausalLM"]}' --chat-template examples/template_chatml.jinja
```
4.启动gradio服务
```
python gradio_openai_vlm_webserver.py --model "/mnt/data/llm-models/llava/llava-1.5-7b-hf" --model-url http://localhost:8000/v1 --host "0.0.0.0" --port 8001"
python gradio_openai_vlm_webserver.py --model model_path --model-url http://localhost:8000/v1 --host "0.0.0.0" --port 8001"
```
5.使用对话服务
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment