README.md

# PointPillars
## 论文
[PointPillars: Fast Encoders for Object Detection from Point Clouds](https://arxiv.org/abs/1812.05784)
## 模型结构
<div align=center>
    <img src="./asserts/model.png"/>
</div>

## 算法原理
本文提出的PointPillars是一种针对3D目标检测任务的新型点云编码器和网络结构。它采用PointNets来学习点云的垂直柱状特征表示，从而更好地捕捉点云信息，并将其应用于标准的2D卷积检测架构中。PointPillars的设计使得可以根据需要进行速度和精度之间的权衡，同时在保持高精度的同时可以实现超过100Hz的速度。

具体来说，PointPillars采用了三个块来构建网络结构，每个块都包括 Upsampling 步骤和 Concatenation 步骤。这些步骤使特征向量的维度逐渐增加，以便于后续的目标检测任务。此外，PointPillars还使用了一些损失函数来优化网络参数，如SmoothL1损失、Focal损失等。
<div align=center>
    <img src="./asserts/image.png"/>
</div>

## 环境配置
`-v 路径`、`docker_name`和`imageID`根据实际情况修改
###
### Docker（方法一）
```bash
docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.4.1-ubuntu22.04-dtk25.04-py3.10
docker run -it --shm-size 200g --network=host --name {docker_name} --privileged --device=/dev/kfd --device=/dev/dri --device=/dev/mkfd --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -u root -v /path/your_code_data/:/path/your_code_data/ -v /opt/hyhal/:/opt/hyhal/:ro {imageID} bash

cd /your_code_path/pointpillars-pytorch
pip install -r requirements.txt
```

### Dockerfile（方法二）
```bash
cd docker
docker build --no-cache -t pointpillars:latest .
docker run -it --shm-size 200g --network=host --name {docker_name} --privileged --device=/dev/kfd --device=/dev/dri --device=/dev/mkfd --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -u root -v /path/your_code_data/:/path/your_code_data/ -v /opt/hyhal/:/opt/hyhal/:ro {imageID} bash

cd /your_code_path/pointpillars-pytorch
pip install -r requirements.txt
```

### Anaconda（方法三）
关于本项目DCU显卡所需的特殊深度学习库可从[光合](https://developer.sourcefind.cn/tool/)开发者社区下载安装。
```bash
DTK: 25.04
python: 3.10
vllm: 0.8.5
torch: 2.4.1+das.opt2.dtk2504
deepspeed: 0.14.2+das.opt2.dtk2504

```
`Tips：以上dtk驱动、python、torch等DCU相关工具版本需要严格一一对应`

其它非深度学习库安装方式如下：
```bash
pip install -r requirements.txt
```
## 数据集
 下载数据集[point cloud](https://s3.eu-central-1.amazonaws.com/avg-kitti/data_object_velodyne.zip)(29GB), [images](https://s3.eu-central-1.amazonaws.com/avg-kitti/data_object_image_2.zip)(12 GB), [calibration files](https://s3.eu-central-1.amazonaws.com/avg-kitti/data_object_calib.zip)(16 MB)和[labels](https://s3.eu-central-1.amazonaws.com/avg-kitti/data_object_label_2.zip)(5 MB)。数据格式如下所示:
```
 kitti
        |- training
            |- calib (#7481 .txt)
            |- image_2 (#7481 .png)
            |- label_2 (#7481 .txt)
            |- velodyne (#7481 .bin)
            |- velodyne_reduced (#7481 .bin)
        |- testing
            |- calib (#7518 .txt)
            |- image_2 (#7518 .png)
            |- velodyne (#7518 .bin)
            |- velodyne_reduced (#7518 .bin)
        |- kitti_gt_database (# 19700 .bin)
        |- kitti_infos_train.pkl
        |- kitti_infos_val.pkl
        |- kitti_infos_trainval.pkl
        |- kitti_infos_test.pkl
        |- kitti_dbinfos_train.pkl
```
## 训练
```
cd PointPillars/
python train.py --data_root your_path_to_kitti
```

## 推理-验证
```
cd PointPillars/
python evaluate.py --ckpt pretrained/epoch_160.pth --data_root your_path_to_kitti 
```

## result
<div align=center>
    <img src="./asserts/result.png"/>
</div>

### 精度
```
与NV基本一致，部分精度高于NV
==========BBOX_2D==========
Pedestrian AP@0.5: 65.2719 61.7278 58.0838
Cyclist AP@0.5: 87.5932 75.1039 71.6308
Car AP@0.7: 90.7070 89.4133 86.6231
==========AOS==========
Pedestrian AOS@0.5: 45.5449 43.3262 40.6216
Cyclist AOS@0.5: 87.2105 72.7019 69.1393
Car AOS@0.7: 90.6199 88.9817 85.9057
==========BBOX_BEV==========
Pedestrian AP@0.5: 59.7484 55.0461 50.7703
Cyclist AP@0.5: 84.4216 68.7483 64.2096
Car AP@0.7: 90.1521 87.5884 85.7974
==========BBOX_3D==========
Pedestrian AP@0.5: 51.5879 46.5993 43.3320
Cyclist AP@0.5: 80.9012 62.8649 60.4813
Car AP@0.7: 86.0721 76.7700 74.3576

==========Overall==========
bbox_2d AP: 81.1907 75.4150 72.1126
AOS AP: 74.4584 68.3366 65.2222
bbox_bev AP: 78.1073 70.4609 66.9258
bbox_3d AP: 72.8538 62.0781 59.3903
```
## 应用场景
### 算法类别
文本理解

### 热点应用行业
制造,智能驾驶,3D点云


## 源码仓库及问题反馈
- https://developer.sourcefind.cn/codes/modelzoo/pointpillars-pytorch

## 参考资料
- https://github.com/zhulf0804/PointPillars