Commit 46a25a83 authored by laibao's avatar laibao
Browse files

laibao

parent 5d257366
......@@ -200,6 +200,7 @@ chmod +x frpc_linux_amd64_v0.*
```
ssh -L 8000:计算节点IP:8000 -L 8001:计算节点IP:8001 用户名@登录节点 -p 登录节点端口
```
通过跳板机(登录节点)转发端口,让你在本地访问内网计算节点上的服务(如 vLLM API)。
3.启动OpenAI兼容服务
......@@ -272,7 +273,7 @@ Prompt: 'What is deep learning?', Generated text: ' Deep learning is a subset of
## 源码仓库及问题反馈
* [ModelZoo / Qwen3_vllm · GitLab](https://developer.hpccube.com/codes/modelzoo/qwen3_vllm)
* [https://developer.hpccube.com/codes/modelzoo/qwen3_vllm](https://developer.hpccube.com/codes/modelzoo/qwen3_vllm)
## 参考资料
......
......@@ -48,5 +48,5 @@ if __name__ == "__main__":
help="Data type for model weights")
args = parser.parse_args()
main(args.model_path, args.tensor_parallel_size, args.gpu_memory_utilization, args.dtype)
main(args.model_path, args.tp, args.gpu_memory_utilization, args.dtype)
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment