README.md 2.13 KB
Newer Older
zhuwenwen's avatar
zhuwenwen committed
1
2
3
# <div align="center"><strong>Colossal-AI</strong></div>
## 简介
Colossal-AI提供了一系列并行组件,主要提供以下功能:提供用户友好的工具,支持编写分布式深度学习模型,只需几行即可启动分布式训练和推理。Colossal-AI官方github地址:[https://github.com/hpcaitech/ColossalAI](https://github.com/hpcaitech/ColossalAI)
zhuwenwen's avatar
zhuwenwen committed
4

zhuwenwen's avatar
zhuwenwen committed
5
6
7
8
9
## 安装
Colossal-AI支持
+ Python 3.7.
+ Python 3.8.
+ Python 3.9.
zhuwenwen's avatar
zhuwenwen committed
10

zhuwenwen's avatar
zhuwenwen committed
11
### 使用pip安装
zhuwenwen's avatar
zhuwenwen committed
12
colossalai whl包下载目录:[https://cancon.hpccube.com:65024/4/main/colossalai/dtk23.10](https://cancon.hpccube.com:65024/4/main/colossalai/dtk23.10)
zhuwenwen's avatar
zhuwenwen committed
13
根据对应的pytorch版本和python版本,下载对应colossalai的whl包
zhuwenwen's avatar
zhuwenwen committed
14
15

```shell
zhuwenwen's avatar
zhuwenwen committed
16
pip install colossalai* (下载的colossalai的whl包)
zhuwenwen's avatar
zhuwenwen committed
17
18
```

zhuwenwen's avatar
zhuwenwen committed
19
### 使用源码编译方式安装
zhuwenwen's avatar
zhuwenwen committed
20

zhuwenwen's avatar
zhuwenwen committed
21
#### 编译环境准备
zhuwenwen's avatar
zhuwenwen committed
22
提供2种环境准备方式:
zhuwenwen's avatar
zhuwenwen committed
23

zhuwenwen's avatar
zhuwenwen committed
24
25
26
1. 基于光源pytorch基础镜像环境:镜像下载地址:[https://sourcefind.cn/#/image/dcu/pytorch](https://sourcefind.cn/#/image/dcu/pytorch),根据pytorch、python、dtk及系统下载对应的镜像版本。

2. 基于现有python环境:安装pytorch,pytorch whl包下载目录:[https://cancon.hpccube.com:65024/4/main/pytorch/dtk23.10](https://cancon.hpccube.com:65024/4/main/pytorch/dtk23.10),根据python、dtk版本,下载对应pytorch的whl包。安装命令如下:
zhuwenwen's avatar
zhuwenwen committed
27
```shell
zhuwenwen's avatar
zhuwenwen committed
28
pip install torch* (下载的torch的whl包)
zhuwenwen's avatar
zhuwenwen committed
29
pip install setuptools wheel
zhuwenwen's avatar
zhuwenwen committed
30
31
```

zhuwenwen's avatar
zhuwenwen committed
32
#### 源码编译安装
zhuwenwen's avatar
zhuwenwen committed
33
```shell
zhuwenwen's avatar
zhuwenwen committed
34
git clone https://developer.hpccube.com/codes/aicomponent/colossalai # 根据需要的分支进行切换
zhuwenwen's avatar
zhuwenwen committed
35
```
zhuwenwen's avatar
zhuwenwen committed
36

zhuwenwen's avatar
zhuwenwen committed
37
38
39
- 提供2种源码编译方式(进入colossalai目录):
```
1. 编译whl包并安装
zhuwenwen's avatar
zhuwenwen committed
40
41
python setup.py bdist_wheel 
pip install dist/colossalai*
zhuwenwen's avatar
zhuwenwen committed
42

zhuwenwen's avatar
zhuwenwen committed
43
44
2. 源码编译安装
python3 setup.py install 
zhuwenwen's avatar
zhuwenwen committed
45
46
```

zhuwenwen's avatar
zhuwenwen committed
47
## 版本号查询
zhuwenwen's avatar
zhuwenwen committed
48
49
50
51
- python -c "import colossalai; print(colossalai.\_\_version__)",版本号与官方版本同步,查询该软件的版本号,例如0.1.13;

## Known Issue
-
zhuwenwen's avatar
zhuwenwen committed
52

zhuwenwen's avatar
zhuwenwen committed
53
## Note
zhuwenwen's avatar
zhuwenwen committed
54
55
56
57
+ 若使用 pip install 下载安装过慢,可添加源:-i https://pypi.tuna.tsinghua.edu.cn/simple/

## 参考
- [README_ORIGIN](README_ORIGIN.md)
zhuwenwen's avatar
zhuwenwen committed
58
59
60
61
62
63
64
65
66
67
- [README_zh-Hans](README_zh-Hans.md)