# JIUTIAN-139MoE-Chat

## 论文

https://www.modelscope.cn/models/JiuTian-AI/JIUTIAN-139MoE-chat/file/view/master?fileName=JIUTIAN-139MOE%2520TECHNICAL%2520REPORT.pdf&status=1

## 模型结构

JIUTIAN-139MoE是一个拥有130亿参数的大型语言模型，使用解码器型的MoE架构，包含一对大专家和六个小专家。模型支持在不同GPU和NPU集群上训练，并能无损切换。在FFN层采用MoE设计，有特殊的激活和路由机制。

![](https://developer.sourcefind.cn/codes/modelzoo/jiutian-139moe-chat/-/raw/main/jiutian.png?inline=false)

## 算法原理

JIUTIAN-139MoE利用Mixture-of-Experts (MoE) 架构，通过不同规模的专家网络处理不同的数据特征，并通过门控机制智能地分配任务给最合适的专家，从而提高模型处理复杂问题的能力。

![](https://developer.sourcefind.cn/codes/modelzoo/jiutian-139moe-chat/-/raw/main/MoE.png?inline=false)

## 环境配置

### Docker（方法一）

拉取镜像，启动并进入容器

```
docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-ubuntu20.04-dtk24.04.1-py3.10
# <Image ID>用上面拉取docker镜像的ID替换
# <Host Path>主机端路径
# <Container Path>容器映射路径
docker run -it  --shm-size 80g --network=host --name=jiutian --privileged  --device /dev/m--device=/dev/kfd --device=/dev/dri --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -v /opt/hyhal/:/opt/hyhal/:ro -v <Host Path>:<Container Path> <Image ID> /bin/bash
```

### Dockerfile（方法二）

```
# <Host Path>主机端路径
# <Container Path>容器映射路径
docker build -t jiutian:latest .
docker run -it  --shm-size 80g --network=host --name=jiutian --privileged  --device /dev/m--device=/dev/kfd --device=/dev/dri --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -v /opt/hyhal/:/opt/hyhal/:ro -v <Host Path>:<Container Path> <Image ID> /bin/bash
```

### Anaconda（方法三）

```
conda create -n jiutian python=3.10
```

## 模型下载

```
#克隆项目
git clone http://developer.hpccube.com/codes/modelzoo/jiutian-139moe-chat.git
cd jiutian-139moe-chat
```

下载模型权重：[JIUTIAN-139MoE-Chat · 模型库 (modelscope.cn)](https://www.modelscope.cn/models/jiutian-ai/jiutian-139moe-chat/files)

## 数据集

无

## 训练

无

## 推理

```
python inference.py
```

### FastAPI调用模型

```
python app.py
```

测试容器内部是否能够正常调用：

另起一个终端，输入

```
curl -X POST "http://localhost:8000/predict/" -H "Content-Type: application/json" -d '{"text": "Please introduce the Great Wall."}'
```

## result

```
{"response":"Human:\nPlease introduce the Great Wall.\n\nAssistant:\n The Great Wall of China is a series of fortifications built along the northern borders of China to protect against invasions and raids from various nomadic groups. It is one of the most famous landmarks in China and is also one of the largest construction projects in human history.\n\nThe Great Wall stretches"}
```

## 应用场景

### 算法类别

对话问答

### 热点应用行业

电信、能源、交通、航空、钢铁、金融、医疗、建筑

## 源码仓库及问题反馈

http://developer.hpccube.com/codes/modelzoo/jiutian-139moe-chat.git

## 参考资料

[JIUTIAN-139MoE-Chat · 模型库 (modelscope.cn)](https://www.modelscope.cn/models/JiuTian-AI/JIUTIAN-139MoE-chat)