README.md

# JIUTIAN-139MoE-Chat

## 环境配置

### Docker

拉取镜像，启动并进入容器

```
docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-ubuntu20.04-dtk24.04.1-py3.10
docker run -it  --shm-size 80g --network=host --name=jiutian --privileged  --device /dev/m--device=/dev/kfd --device=/dev/dri --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -v /opt/hyhal/:/opt/hyhal/:ro image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-ubuntu20.04-dtk24.04.1-py3.10 /bin/bash
```

下载模型权重：[JIUTIAN-139MoE-Chat · 模型库 (modelscope.cn)](https://www.modelscope.cn/models/jiutian-ai/jiutian-139moe-chat/files)

```
#克隆项目
git clone http://developer.hpccube.com/codes/modelzoo/jiutian-139moe-chat.git
cd jiutian-139moe-chat
```

### 模型推理

```
python inference.py
```

### FastAPI调用模型

```
python app.py
```

测试容器内部是否能够正常调用：

另起一个终端，输入

```
curl -X POST "http://localhost:8000/predict/" -H "Content-Type: application/json" -d '{"text": "Please introduce the Great Wall."}'
```

成功调用会在终端生成内容：

```
{"response":"Human:\nPlease introduce the Great Wall.\n\nAssistant:\n The Great Wall of China is a series of fortifications built along the northern borders of China to protect against invasions and raids from various nomadic groups. It is one of the most famous landmarks in China and is also one of the largest construction projects in human history.\n\nThe Great Wall stretches"}
```