Commit 997d1621 authored by weishb's avatar weishb
Browse files

更新部署步骤

parent 9f204d9f
...@@ -59,6 +59,12 @@ docker run -it \ ...@@ -59,6 +59,12 @@ docker run -it \
#### 单机推理 #### 单机推理
**以`Spark-Chemistry-X1-13B`为例** **以`Spark-Chemistry-X1-13B`为例**
```bash
cd /path/to/model/Spark-Chemistry-X1-13B #进入模型目录内
python transformers_run.py
```
**transformers_run.py参考:**
```python ```python
from transformers import AutoModelForCausalLM, AutoTokenizer from transformers import AutoModelForCausalLM, AutoTokenizer
import torch import torch
...@@ -119,7 +125,7 @@ vllm serve /path/to/Spark-Scilit-X1-13B \ ...@@ -119,7 +125,7 @@ vllm serve /path/to/Spark-Scilit-X1-13B \
curl http://127.0.0.1:8000/v1/chat/completions \ curl http://127.0.0.1:8000/v1/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "Spark-Formalizer-X1-7B", "model": "/path/to/Spark-Scilit-X1-13B",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment