README.md 2.34 KB
Newer Older
limm's avatar
limm committed
1
# <div align="center"><strong>PyTorch Scatter</strong></div>
quyuanhao123's avatar
quyuanhao123 committed
2
## 简介
limm's avatar
limm committed
3
PyTorch Scatter由一个小型扩展库组成,专为PyTorch设计,提供了一系列高度优化的稀疏更新(scatter和segment)操作,这些操作在主包中是缺失的。Scatter和segment操作可以大致描述为基于给定“组索引”张量的归约操作。Segment操作要求“组索引”张量是有序的,而scatter操作则不受此要求限制。DAS软件栈中的PyTorch Scatter版本,不仅保证了该组件 核心功能在DCU加速卡的可用性,还针对DCU特有的硬件架构进行了深度定制优化,这使得开发者能够以极低的成本,轻松实现应用程序在DCU加速卡上的快速迁移和性能提升。目前已适配支持Pytorch1.13,Pyotrch2.1,Pytorch2.3。
quyuanhao123's avatar
quyuanhao123 committed
4

limm's avatar
limm committed
5
## 安装
quyuanhao123's avatar
quyuanhao123 committed
6

limm's avatar
limm committed
7
8
9
10
11
12
### 使用pip方式安装
pytorch-scatter whl包下载目录:[http://10.6.10.68:8000/customized/torch-scatter/dtk2404](http://10.6.10.68:8000/customized/torch-scatter/dtk2404),目前只提供有python3.8版本的whl包。
```shell
pip install torch_scatter* (下载的torch_scatter的whl包)
```
### 使用源码编译方式安装
quyuanhao123's avatar
quyuanhao123 committed
13
14

#### 编译环境准备
limm's avatar
limm committed
15
- 安装相关依赖
quyuanhao123's avatar
quyuanhao123 committed
16
```shell
limm's avatar
limm committed
17
pip install -r requirements.txt 
quyuanhao123's avatar
quyuanhao123 committed
18
```
limm's avatar
limm committed
19
- 在首页 | 光合开发者社区下载 dtk24.04 解压至 /opt/ 路径下,并建立软链接
quyuanhao123's avatar
quyuanhao123 committed
20
```shell
limm's avatar
limm committed
21
22
cd /opt && ln -s dtk-24.04 dtk
source /opt/dtk/env.sh
quyuanhao123's avatar
quyuanhao123 committed
23
```
limm's avatar
limm committed
24
25
26
- 安装pytorch,pytorch whl包下载目录:[http://10.6.10.68:8000/debug/pytorch/dtk24.10/hipify/](http://10.6.10.68:8000/debug/pytorch/dtk24.04/hipify/),根据python、dtk版本,下载对应pytorch的whl包。安装命令如下:
```shell
pip install torch* (下载的torch的whl包)
quyuanhao123's avatar
quyuanhao123 committed
27
```
limm's avatar
limm committed
28
29
30
31
32
33
#### 源码编译安装
```shell
git clone -b 2.1.0-release http://developer.hpccube.com/codes/aicomponent/torch-scatter.git
cd torch-scatter
python setup.py bdist_wheel
pip install dist/*.whl
quyuanhao123's avatar
quyuanhao123 committed
34
```
limm's avatar
limm committed
35
36
37
38
## 单测
```shell
cd torch-scatter
python setup.py test
quyuanhao123's avatar
quyuanhao123 committed
39
```
limm's avatar
limm committed
40
41
## Known Issue
完成安装进行单测时,会报错ImportError: Could not find module '_version_cpu' ~,在根目录/下查找一下,然后把库文件目录添加一下软链接即可。
quyuanhao123's avatar
quyuanhao123 committed
42
43

```
limm's avatar
limm committed
44
45
46
find / -name "_version_cpu.so"
cd /torch-scatter/torch_scatter
ln -s /usr/local/lib/python3.8/site-packages/torch-scatter/* .
quyuanhao123's avatar
quyuanhao123 committed
47
48
49
```

## 参考资料
limm's avatar
limm committed
50

limm's avatar
limm committed
51
[https://github.com/rusty1s/pytorch_scatter](https://github.com/rusty1s/pytorch_scatter)