README.md 3.07 KB
Newer Older
limm's avatar
limm committed
1
2
# <div aligh="center"><strong>PyTorch Sparse</strong></div>
## 简介
3
Pytorch Sparce 是 个包包含了一个小型扩展库,用于优化支持自动微分的稀疏矩阵操作。目前,这个包包括以下方法:Coalesce,Transpose,Sparse Dense Matrix Multiplication,Sparse Sparse Matrix Multiplication 所有包含的操作都支持不同类型的数据,并且既在CPU上也在GPU上实现了。DAS软件栈中的PyTorch Sparce版本,不仅保证了组件核心功能在DCU加速卡的可用性,还针对DCU特有的硬件架构进行了深度定制优化。这使得开发者能够以极低的成本,轻松实现应用程序在DCU加速卡上的快速迁移和性能提升。目前支持Pytorch1.13 Pyotrch2.1 Pytorch2.4.1 Pytorch2.5.1
limm's avatar
limm committed
4
5

### 使用pip方式安装
6
torch-sparce whl包下载目录:[光合开发者社区](https://das.sourcefind.cn:55011/portal/#/home). 目前只提供有python.10版本的安装包 (如果为空需要自己按照下面步骤编译)
limm's avatar
limm committed
7
8
9
10
11
12
13
14
15
16
17
```shell
pip install torch_sparse* (下载的torch_sparse的whl包)
```
### 使用源码编译方式安装

#### 编译环境准备
- 安装相关依赖
```shell
pip install numpy
pip install 'urllib3==1.26.14'
pip install pytest
limm's avatar
limm committed
18
pip install wheel
limm's avatar
limm committed
19
```
20
- 在首页 | 光合开发者社区下载 dtk25.04 解压在 /opt/ 路径下,并建立软连接,例如:
limm's avatar
limm committed
21
```shell
22
cd /opt & ln -s dtk-25.04 dtk
limm's avatar
limm committed
23
24
```

25
- 安装pytorch. pytorch whl包下载目录: [光合开发者社区](https://das.sourcefind.cn:55011/portal/#/home). 根据需求下载对应系统下的版本,安装如下:
limm's avatar
limm committed
26
27
28
```shell
pip install torch* (下载的torch的whl包)
```
29
30
31
32
33
34
- 安装fastpt. fastpt whl包下载目录: [光合开发者社区](https://das.sourcefind.cn:55011/portal/#/home). 根据需求下载对应系统下的版本,安装如下:
```shell
pip install fastpt* (下载的fastpt的whl包)
```


limm's avatar
limm committed
35
36
#### 源码下载编译安装
```shell
37
38
39
git clone -b 0.6.16-fastpt http://developer.hpccube.com/codes/aicomponent/torch-sparce.git
export FORCE_CUDA=1
source /usr/local/bin/fastpt -C
limm's avatar
limm committed
40
41
42
43
44
45
46
47
48
cd torch-sparce
python setup.py bdist_wheel
pip install dist/*.whl
```

## 单侧
```shell
cd torch-sparce
pytest
limm's avatar
limm committed
49
50
```

limm's avatar
limm committed
51
52
## Known Issue
- 完成安装进行单测时,会报错ImportError: Could not find module '_version_cpu' ~,在根目录/下查找一下,然后把库文件目录添加一下软链接即可。
limm's avatar
limm committed
53
```
limm's avatar
limm committed
54
55
find / -name "_version_cpu.so"
cd /torch-sparce/torch_sparse
limm's avatar
limm committed
56
ln -s /usr/local/lib/python3.10/site-packages/torch_sparse/* .
limm's avatar
limm committed
57
58
```

limm's avatar
limm committed
59
- 编译torch_sparse==0.6.16 目前不支持torch2.1版本的torch,如果编译torch2.1版本的torch_sparse0.6.16版本需要修改一些代码,文件路径pytorch_sparse/csrc/version.cpp
limm's avatar
limm committed
60
61
```shell
static auto registry = torch::RegisterOperators().op("torch_sparse::cuda_version", &sparse::cuda_version); -> static auto registry = torch::RegisterOperators().op("torch_sparse::cuda_version", [] { return sparse::cuda_version(); });
limm's avatar
limm committed
62
```
limm's avatar
limm committed
63
代码已修改,目前已经支持torch2.1, 如果编译torch1.13 需要把上面代码修改回去.
limm's avatar
limm committed
64

limm's avatar
limm committed
65
## 参考资料
limm's avatar
limm committed
66
[https://github.com/rusty1s/pytorch_sparse](https://github.com/rusty1s/pytorch_sparse)