Commit 89e69f24 authored by laibao's avatar laibao
Browse files

No commit message

No commit message
parent b4c46fe7
# Internlm_vllm
![1732692191521](image/README/1732692191521.png)
## 论文
`InternLM2 Technical Report`
......@@ -77,8 +79,8 @@ conda create -n internlm_vllm python=3.10
### 模型下载
| 基座模型 | | | |
| ----------------------------------------------------------- | ----------------------------------------------------------- | ------------------------------------------------------------- | --------------------------------------------------------------- |
| 基座模型 | | | |
| ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
| [internlm2-7b](http://113.200.138.88:18080/aimodels/internlm/internlm2-7b.git)  | [internlm2-20b](http://113.200.138.88:18080/aimodels/internlm/internlm2-20b.git) | [internlm2_5-7b](http://113.200.138.88:18080/aimodels/internlm/internlm2_5-7b.git) | [internlm2_5-20b](http://113.200.138.88:18080/aimodels/internlm/internlm2_5-20b.git) |
### 离线批量推理
......@@ -199,11 +201,12 @@ python gradio_openai_chatbot_webserver.py --model "internlm/internlm2_5-7b" --m
```
chmod +x frpc_linux_amd64_v0.*
```
2.3 端口映射
```
ssh -L 8000:计算节点IP:8000 -L 8001:计算节点IP:8001 用户名@登录节点 -p 登录节点端口
```
```
3.启动OpenAI兼容服务
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment