Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
Qwen3_vllm
Commits
46a25a83
Commit
46a25a83
authored
Oct 30, 2025
by
laibao
Browse files
laibao
parent
5d257366
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
3 additions
and
2 deletions
+3
-2
README.md
README.md
+2
-1
examples/offline_inference/basic/basic.py
examples/offline_inference/basic/basic.py
+1
-1
No files found.
README.md
View file @
46a25a83
...
@@ -200,6 +200,7 @@ chmod +x frpc_linux_amd64_v0.*
...
@@ -200,6 +200,7 @@ chmod +x frpc_linux_amd64_v0.*
```
```
ssh -L 8000:计算节点IP:8000 -L 8001:计算节点IP:8001 用户名@登录节点 -p 登录节点端口
ssh -L 8000:计算节点IP:8000 -L 8001:计算节点IP:8001 用户名@登录节点 -p 登录节点端口
```
```
通过跳板机(登录节点)转发端口,让你在本地访问内网计算节点上的服务(如 vLLM API)。
通过跳板机(登录节点)转发端口,让你在本地访问内网计算节点上的服务(如 vLLM API)。
3.
启动OpenAI兼容服务
3.
启动OpenAI兼容服务
...
@@ -272,7 +273,7 @@ Prompt: 'What is deep learning?', Generated text: ' Deep learning is a subset of
...
@@ -272,7 +273,7 @@ Prompt: 'What is deep learning?', Generated text: ' Deep learning is a subset of
## 源码仓库及问题反馈
## 源码仓库及问题反馈
*
[
M
odel
Z
oo
/ Q
wen3_vllm
· GitLab
](
https://developer.hpccube.com/codes/modelzoo/qwen3_vllm
)
*
[
https://developer.hpccube.com/codes/m
odel
z
oo
/q
wen3_vllm
](
https://developer.hpccube.com/codes/modelzoo/qwen3_vllm
)
## 参考资料
## 参考资料
...
...
examples/offline_inference/basic/basic.py
View file @
46a25a83
...
@@ -48,5 +48,5 @@ if __name__ == "__main__":
...
@@ -48,5 +48,5 @@ if __name__ == "__main__":
help
=
"Data type for model weights"
)
help
=
"Data type for model weights"
)
args
=
parser
.
parse_args
()
args
=
parser
.
parse_args
()
main
(
args
.
model_path
,
args
.
t
ensor_parallel_size
,
args
.
gpu_memory_utilization
,
args
.
dtype
)
main
(
args
.
model_path
,
args
.
t
p
,
args
.
gpu_memory_utilization
,
args
.
dtype
)
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment