- 
-
--* Please note that the FPS here is measured with RTX3090 TensorRT FP16. - -# Panoptic-FlashOcc: An Efficient Baseline to Marry Semantic Occupancy with Panoptic via Instance Center - -
- 
-
--* Please note that the FPS here is measured with A100 GPU (PyTorch fp32 backend). - -## News -- **2024.09.16** Technical Report: FlashOcc can be insert to Bevdet with 1.1ms consumption while facilitating each other.[
-
-- -and [Panoptic-FlashOCC](https://arxiv.org/pdf/2406.10527) - -
- 
-
-- - -## Main Results -### 1. FlashOCC -| Config | Backbone | Input
Size | mIoU | FPS
(Hz) | Flops
(G) | Params
(M) | Model | Log | -|:---------------------------------------------------------------------------------------------------|:--------:|:----------:|:-------:|:-----------------------------:|:------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------:| -| [**BEVDetOCC (1f)**](projects/configs/bevdet_occ/bevdet-occ-r50.py) | R50 | 256x704 | 31.60 | 92.1 | [241.76](doc/mmdeploy_test.md) | [29.02](doc/mmdeploy_test.md) | [gdrive]() | [log]() | -| [**M0: FlashOCC (1f)**](projects/configs/flashocc/flashocc-r50.py) | R50 | 256x704 | 31.95 | [197.6](doc/mmdeploy_test.md) | [154.1](doc/mmdeploy_test.md) | [39.94](doc/mmdeploy_test.md) | [gdrive](https://drive.google.com/file/d/14my3jdqiIv6VIrkozQ6-ruEcBOPVlWGJ/view?usp=sharing) | [log](https://drive.google.com/file/d/1E-kaHxbTr6s3Qn70vhKpwJM8kejoNFxQ/view?usp=sharing) | -| [**M1: FlashOCC (1f)**](projects/configs/flashocc/flashocc-r50.py) | R50 | 256x704 | 32.08 | [152.7](doc/mmdeploy_test.md) | [248.57](doc/mmdeploy_test.md) | [44.74](doc/mmdeploy_test.md) | [gdrive](https://drive.google.com/file/d/1k9BzXB2nRyvXhqf7GQx3XNSej6Oq6I-B/view?usp=drive_link) | [log](https://drive.google.com/file/d/1NRm27wVZMSUylmZxsMedFSLr7729YEAV/view?usp=drive_link) | -| [**BEVDetOCC-4D-Stereo (2f)**](projects/configs/bevdet_occ/bevdet-occ-r50-4d-stereo.py) | R50 | 256x704 | 36.1 | - | - | - | [baidu](https://pan.baidu.com/s/1237QyV18zvRJ1pU3YzRItw?pwd=npe1) | [log](https://pan.baidu.com/s/1237QyV18zvRJ1pU3YzRItw?pwd=npe1) | -| [**M2:FlashOCC-4D-Stereo (2f)**](projects/configs/flashocc/flashocc-r50-4d-stereo.py) | R50 | 256x704 | 37.84 | - | - | - | [gdrive](https://drive.google.com/file/d/12WYaCdoZA8-A6_oh6vdLgOmqyEc3PNCe/view?usp=drive_link) | [log](https://drive.google.com/file/d/1eYvu9gUSQ7qk7w7lWPLrZMB0O2uKQUk3/view?usp=drive_link) | -| [**BEVDetOCC-4D-Stereo (2f)**](projects/configs/bevdet_occ/bevdet-occ-stbase-4d-stereo-512x1408.py) | Swin-B | 512x1408 | 42.0 | - | - | - | [baidu](https://pan.baidu.com/s/1237QyV18zvRJ1pU3YzRItw?pwd=npe1) | [log](https://pan.baidu.com/s/1237QyV18zvRJ1pU3YzRItw?pwd=npe1) | -| [**M3:FlashOCC-4D-Stereo (2f)**](projects/configs/flashocc/flashocc-stbase-4d-stereo-512x1408_4x4_2e-4.py) | Swin-B | 512x1408 | 43.52 | - | [1490.77](doc/mmdeploy_test.md) | [144.99](doc/mmdeploy_test.md) | [gdrive](https://drive.google.com/file/d/1f6E6Bm6enIJETSEbfXs57M0iOUU997kU/view?usp=drive_link) | [log](https://drive.google.com/file/d/1tch-YK4ROGDGNmDcN5FZnOAvsbHe-iSU/view?usp=drive_link) | - -FPS are tested via TensorRT on 3090 with FP16 precision. Please refer to Tab.2 in paper for the detail model settings for M-number. - -### 2. Panoptic-FlashOCC -**In Panoptic-FlashOCC, we have made the following 3 adjustments to FlashOCC**: -- Without using camera mask for training. This is because its use significantly improves the prediction performance in the visible region, but at the expense of prediction in the invisible region. -- Using category balancing. -- Using stronger loss settings. -- Introducing instance center for panoptic occupancy - -**More results for different configurations will be released soon.** - -| Config | Backbone | Input
Size | RayIou | RayPQ | mIoU | FPS
(Hz) | Flops
(G) | Params
(M) | Model | Log | -|:--------------------------------------------------------------------------------|:--------:|:-----------:|:-------:|:------:|:------:|:------------:|:------------------------------:|:-----------------------------:|:-----------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------:| -| [**M1: FlashOCC (1f)**](projects/configs/flashocc/flashocc-r50.py) | R50 | 256x704 | - | - | 15.41 | - | [248.57](doc/mmdeploy_test.md) | [44.74](doc/mmdeploy_test.md) | [gdrive](https://drive.google.com/file/d/14XsvjSwp-vLpy_eBZzvKo3MAh-YWRHcu/view?usp=drive_link) | [log](https://drive.google.com/file/d/1cTDoauEmjhK9fReLDcA2zPZx4a6X3U1-/view?usp=drive_link) | -| [**Panoptic-FlashOCC-Depth-tiny (1f)**](projects/configs/panoptic-flashocc/panoptic-flashocc-r50-depth-tiny.py) | R50 | 256x704 | 34.57 | - | 28.83 | 43.9 | [175.00](doc/mmdeploy_test.md) | [45.32](doc/mmdeploy_test.md) | [gdrive](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | [log](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | -| [**Panoptic-FlashOCC-Depth-tiny-Pano (1f)**](projects/configs/panoptic-flashocc/panoptic-flashocc-r50-depth-tiny-pano.py) | R50 | 256x704 | 34.81 | 12.9 | 29.14 | 39.8 | [175.00](doc/mmdeploy_test.md) | [45.32](doc/mmdeploy_test.md) | [gdrive](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | [log](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | -| [**Panoptic-FlashOCC-Depth (1f)**](projects/configs/panoptic-flashocc/panoptic-flashocc-r50-depth.py) | R50 | 256x704 | 34.93 | - | 28.91 | 38.7 | [269.47](doc/mmdeploy_test.md) | [50.12](doc/mmdeploy_test.md) | [gdrive](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | [log](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | -| [**Panoptic-FlashOCC-Depth-Pano (1f)**](projects/configs/panoptic-flashocc/panoptic-flashocc-r50-depth-pano.py) | R50 | 256x704 | 35.22 | 13.2 | 29.39 | 35.2 | [269.47](doc/mmdeploy_test.md) | [50.12](doc/mmdeploy_test.md) | [gdrive](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | [log](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | -| [**Panoptic-FlashOCC-4D-Depth (2f)**](projects/configs/panoptic-flashocc/panoptic-flashocc-r50-depth4d.py) | R50 | 256x704 | 35.99 | - | 29.57 | 35.9 | - | - | [gdrive](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | [log](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | -| [**Panoptic-FlashOCC-4D-Depth-Pano (2f)**](projects/configs/panoptic-flashocc/panoptic-flashocc-r50-depth4d-pano.py) | R50 | 256x704 | 36.76 | 14.5 | 30.31 | 30.4 | - | - | [gdrive](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | [log](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | -| [**Panoptic-FlashOCC-4DLongterm-Depth (8f)**](projects/configs/panoptic-flashocc/panoptic-flashocc-r50-depth4d-longterm8f.py) | R50 | 256x704 | 38.51 | - | 31.49 | 35.6 | - | - | [gdrive](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | [log](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | -| [**Panoptic-FlashOCC-4DLongterm-Depth-Pano (8f)**](projects/configs/panoptic-flashocc/panoptic-flashocc-r50-depth4d-longterm8f.py) | R50 | 256x704 | 38.50 | 16.0 | 31.57 | 30.2 | - | - | [gdrive](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | [log](https://drive.google.com/drive/folders/1cgCsbXgikoP10lj6DBC7Le9C-UOCIxlN?usp=sharing) | - -* Please note that the FPS here is measured with A100 GPU (PyTorch fp32 backend). - -## Get Started -1. [Environment Setup](doc/install.md) -2. [Model Training](doc/model_training.md) -3. [Quick Test Via TensorRT In MMDeploy](doc/mmdeploy_test.md) - -| Backend | mIOU | FPS(Hz) | -|----------|-------|---------| -| PyTorch-FP32 | 31.95 | - | -| TRT-FP32 | 30.78 | 96.2 | -| TRT-FP16 | 30.78 | 197.6 | -| TRT-FP16+INT8(PTQ) | 29.60 | 383.7 | -| TRT-INT8(PTQ) | 29.59 | 397.0 | - -4. [Visualization](doc/visualization.md) - * [flashocc] : A detail video can be found at [baidu](https://pan.baidu.com/s/1xfnFsj5IclpjJxIaOlI6dA?pwd=gype) -
- 
-
-- - * [panoptic-flashocc] : first row is our prediction and second row is gt. -
- 
-
-- -
- 
-
-- - -5. [TensorRT Implement Writen In C++ With Cuda Acceleration](https://github.com/drilistbox/TRT_FlashOcc) - - -## Acknowledgement -Many thanks to the authors of [BEVDet](https://github.com/HuangJunJie2017/BEVDet), [FB-BEV](https://github.com/NVlabs/FB-BEV.git), -[RenderOcc](https://github.com/pmj110119/RenderOcc.git) and [SparseBEV](https://github.com/MCG-NJU/SparseBEV.git) - -## Bibtex -If this work is helpful for your research, please consider citing the following BibTeX entry. - -``` -@article{yu2024ultimatedo, - title={UltimateDO: An Efficient Framework to Marry Occupancy Prediction with 3D Object Detection via Channel2height}, - author={Yu, Zichen and Shu, Changyong}, - journal={arXiv preprint arXiv:2409.11160}, - year={2024} -} - -@article{yu2024panoptic, - title={Panoptic-FlashOcc: An Efficient Baseline to Marry Semantic Occupancy with Panoptic via Instance Center}, - author={Yu, Zichen and Shu, Changyong and Sun, Qianpu and Linghu, Junjie and Wei, Xiaobao and Yu, Jiangyong and Liu, Zongdai and Yang, Dawei and Li, Hui and Chen, Yan}, - journal={arXiv preprint arXiv:2406.10527}, - year={2024} -} - -@article{yu2023flashocc, - title={FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugin}, - author={Zichen Yu and Changyong Shu and Jiajun Deng and Kangjie Lu and Zongdai Liu and Jiangyong Yu and Dawei Yang and Hui Li and Yan Chen}, - year={2023}, - eprint={2311.12058}, - archivePrefix={arXiv}, - primaryClass={cs.CV} -} -``` diff --git a/docker-hub/FlashOCC/Flashocc/data/nuscenes b/docker-hub/FlashOCC/Flashocc/data/nuscenes deleted file mode 120000 index 9c09f1a2ed6d85d1bcd1b7dbdd59cf4426de90be..0000000000000000000000000000000000000000 --- a/docker-hub/FlashOCC/Flashocc/data/nuscenes +++ /dev/null @@ -1 +0,0 @@ -/data/nuScenes \ No newline at end of file diff --git a/docker-hub/FlashOCC/Flashocc/doc/install.md b/docker-hub/FlashOCC/Flashocc/doc/install.md deleted file mode 100644 index b86fd71bf16a43f2444f6e46e5a05f98323c992f..0000000000000000000000000000000000000000 --- a/docker-hub/FlashOCC/Flashocc/doc/install.md +++ /dev/null @@ -1,165 +0,0 @@ -## Environment Setup -step 1. Install environment for pytorch training -``` -conda create --name FlashOcc python=3.8.5 -conda activate FlashOcc -pip install torch==1.10.0+cu111 torchvision==0.11.0+cu111 torchaudio==0.10.0 -f https://download.pytorch.org/whl/torch_stable.html -pip install mmcv-full==1.5.3 -pip install mmdet==2.25.1 -pip install mmsegmentation==0.25.0 - -sudo apt-get install python3-dev -sudo apt-get install libevent-dev -sudo apt-get groupinstall 'development tools' -export PATH=/usr/local/cuda/bin:$PATH -export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH -export CUDA_ROOT=/usr/local/cuda -pip install pycuda - -pip install lyft_dataset_sdk -pip install networkx==2.2 -pip install numba==0.53.0 -pip install numpy==1.23.5 -pip install nuscenes-devkit -pip install plyfile -pip install scikit-image -pip install tensorboard -pip install trimesh==2.35.39 -pip install setuptools==59.5.0 -pip install yapf==0.40.1 - -cd Path_to_FlashOcc -git clone git@github.com:Yzichen/FlashOCC.git - -cd Path_to_FlashOcc/FlashOcc -git clone https://github.com/open-mmlab/mmdetection3d.git - -cd Path_to_FlashOcc/FlashOcc/mmdetection3d -git checkout v1.0.0rc4 -pip install -v -e . - -cd Path_to_FlashOcc/FlashOcc/projects -pip install -v -e . -``` - -step 3. Prepare nuScenes dataset as introduced in [nuscenes_det.md](nuscenes_det.md) and create the pkl for FlashOCC by running: -```shell -python tools/create_data_bevdet.py -``` -thus, the folder will be ranged as following: -```shell script -└── Path_to_FlashOcc/ - └── data - └── nuscenes - ├── v1.0-trainval (existing) - ├── sweeps (existing) - ├── samples (existing) - ├── bevdetv2-nuscenes_infos_train.pkl (new) - └── bevdetv2-nuscenes_infos_val.pkl (new) -``` - -step 4. For Occupancy Prediction task, download (only) the 'gts' from [CVPR2023-3D-Occupancy-Prediction](https://github.com/CVPR2023-3D-Occupancy-Prediction/CVPR2023-3D-Occupancy-Prediction) and arrange the folder as: -```shell script -└── Path_to_FlashOcc/ - └── data - └── nuscenes - ├── v1.0-trainval (existing) - ├── sweeps (existing) - ├── samples (existing) - ├── gts (new) - ├── bevdetv2-nuscenes_infos_train.pkl (new) - └── bevdetv2-nuscenes_infos_val.pkl (new) -``` -(for panoptic occupancy), we follow the data setting in SparseOcc: - -(1) Download Occ3D-nuScenes occupancy GT from [gdrive](https://drive.google.com/file/d/1kiXVNSEi3UrNERPMz_CfiJXKkgts_5dY/view?usp=drive_link), unzip it, and save it to `data/nuscenes/occ3d`. - -(2) Generate the panoptic occupancy ground truth with `gen_instance_info.py`. The panoptic version of Occ3D will be saved to `data/nuscenes/occ3d_panoptic`. - - -step 5. CKPTS Preparation -(1) Download flashocc-r50-256x704.pth[https://drive.google.com/file/d/1k9BzXB2nRyvXhqf7GQx3XNSej6Oq6I-B/view] to Path_to_FlashOcc/FlashOcc/ckpts/, then run: -```shell script -bash tools/dist_test.sh projects/configs/flashocc/flashocc-r50.py ckpts/flashocc-r50-256x704.pth 4 --eval map -``` - -step 6. (Optional) Install mmdeploy for tensorrt testing -```shell script -conda activate FlashOcc -pip install Cython==0.29.24 - -### get tensorrt -wget https://developer.download.nvidia.com/compute/machine-learning/tensorrt/secure/8.4.0/tars/TensorRT-8.4.0.6.Linux.x86_64-gnu.cuda-11.6.cudnn8.3.tar.gz -export TENSORRT_DIR=Path_to_TensorRT-8.4.0.6 - -### get onnxruntime -ONNXRUNTIME_VERSION=1.8.1 -pip install onnxruntime-gpu==${ONNXRUNTIME_VERSION} -cd Path_to_your_onnxruntime -wget https://github.com/microsoft/onnxruntime/releases/download/v${ONNXRUNTIME_VERSION}/onnxruntime-linux-x64-${ONNXRUNTIME_VERSION}.tgz \ - && tar -zxvf onnxruntime-linux-x64-${ONNXRUNTIME_VERSION}.tgz -# export ONNXRUNTIME_DIR=/data01/shuchangyong/pkgs/onnxruntime-linux-x64-1.8.1 -export ONNXRUNTIME_DIR=Path_to_your_onnxruntime/onnxruntime-linux-x64-1.8.1 -cd Path_to_FlashOcc/FlashOcc/ -git clone git@github.com:drilistbox/mmdeploy.git -cd Path_to_FlashOcc/FlashOcc/mmdeploy -git submodule update --init --recursive -mkdir -p build -cd Path_to_FlashOcc/FlashOcc/mmdeploy/build -cmake -DMMDEPLOY_TARGET_BACKENDS="ort;trt" .. -make -j 16 -cd Path_to_FlashOcc/FlashOcc/mmdeploy -pip install -e . - -### build sdk -cd Path_to_pplcv/ -git clone https://github.com/openppl-public/ppl.cv.git -cd Path_to_pplcv/ppl.cv -export PPLCV_VERSION=0.7.0 -git checkout tags/v${PPLCV_VERSION} -b v${PPLCV_VERSION} -./build.sh cuda - -#pip install nvidia-tensorrt==8.4.0.6 -pip install nvidia-tensorrt==8.4.1.5 -pip install tensorrt -#pip install h5py -pip install spconv==2.3.6 - -export PATH=Path_to_TensorRT-8.4.0.6/bin:$PATH -export LD_LIBRARY_PATH=Path_to_TensorRT-8.4.0.6/lib:$LD_LIBRARY_PATH -export LIBRARY_PATH=Path_to_TensorRT-8.4.0.6/lib:$LIBRARY_PATH -``` - -## The finally overall rangement -1. Tensort -```shell script -└── Path_to_TensorRT-8.4.0.6 - └── TensorRT-8.4.0.6 -``` -2. FlashOcc -```shell script -└── Path_to_FlashOcc/ - └── data - └── nuscenes - ├── v1.0-trainval (existing) - ├── sweeps (existing) - ├── samples (existing) - ├── gts (new) - ├── bevdetv2-nuscenes_infos_train.pkl (new) - └── bevdetv2-nuscenes_infos_val.pkl (new) - └── doc - ├── install.md - └── trt_test.md - ├── figs - ├── mmdeploy (new) - ├── mmdetection3d (new) - ├── projects - ├── requirements - ├── tools - └── README.md -``` -3. ppl.cv -```shell script -└── Path_to_pplcv - └── ppl.cv -``` diff --git a/docker-hub/FlashOCC/Flashocc/doc/mmdeploy_test.md b/docker-hub/FlashOCC/Flashocc/doc/mmdeploy_test.md deleted file mode 100644 index 266da6e62b68e9b0d2fc530491173975eae42687..0000000000000000000000000000000000000000 --- a/docker-hub/FlashOCC/Flashocc/doc/mmdeploy_test.md +++ /dev/null @@ -1,143 +0,0 @@ -# trt inference speed -```shell -conda activate FlashOcc -1. cmd for M0 -exp_name=flashocc-r50-M0 -fold_name=flashocc -config=projects/configs/${fold_name}/${exp_name}-trt.py -checkpoint=ckpts/flashocc-r50-M0-256x704.pth -work_dir=work_dirs/${exp_name}/onnx_trt/ - -2. cmd for M1 -exp_name=flashocc-r50 -fold_name=flashocc -config=projects/configs/${fold_name}/${exp_name}-trt.py -checkpoint=ckpts/flashocc-r50-256x704.pth -work_dir=work_dirs/${exp_name}/onnx_trt/ - - -# int8 test. -engine=work_dirs/${exp_name}/onnx_trt/bevdet_int8_fuse.engine -python tools/convert_bevdet_to_TRT.py $config $checkpoint $work_dir --fuse-conv-bn --int8 --calib_num 256 -python tools/analysis_tools/benchmark_trt.py $config $engine --eval -# ===> per class IoU of 6019 samples: -# ===> others - IoU = 4.58 -# ===> barrier - IoU = 34.13 -# ===> bicycle - IoU = 8.68 -# ===> bus - IoU = 34.9 -# ===> car - IoU = 40.48 -# ===> construction_vehicle - IoU = 15.99 -# ===> motorcycle - IoU = 15.49 -# ===> pedestrian - IoU = 13.58 -# ===> traffic_cone - IoU = 12.83 -# ===> trailer - IoU = 25.31 -# ===> truck - IoU = 28.08 -# ===> driveable_surface - IoU = 76.7 -# ===> other_flat - IoU = 31.5 -# ===> sidewalk - IoU = 45.01 -# ===> terrain - IoU = 49.63 -# ===> manmade - IoU = 35.72 -# ===> vegetation - IoU = 30.39 -# ===> mIoU of 6019 samples: 29.59 - -# int8+fp16 test. -engine=work_dirs/${exp_name}/onnx_trt/bevdet_int8_fp16_fuse.engine -python tools/convert_bevdet_to_TRT.py $config $checkpoint $work_dir --fuse-conv-bn --fp16 --int8 --calib_num 256 -python tools/analysis_tools/benchmark_trt.py $config $engine --eval -# ===> per class IoU of 6019 samples: -# ===> others - IoU = 4.59 -# ===> barrier - IoU = 34.13 -# ===> bicycle - IoU = 8.71 -# ===> bus - IoU = 34.9 -# ===> car - IoU = 40.49 -# ===> construction_vehicle - IoU = 16.01 -# ===> motorcycle - IoU = 15.55 -# ===> pedestrian - IoU = 13.63 -# ===> traffic_cone - IoU = 12.86 -# ===> trailer - IoU = 25.33 -# ===> truck - IoU = 28.1 -# ===> driveable_surface - IoU = 76.7 -# ===> other_flat - IoU = 31.51 -# ===> sidewalk - IoU = 45.01 -# ===> terrain - IoU = 49.63 -# ===> manmade - IoU = 35.72 -# ===> vegetation - IoU = 30.39 -# ===> mIoU of 6019 samples: 29.6 - -# fp16 test -engine=work_dirs/${exp_name}/onnx_trt/bevdet_fp16_fuse.engine -python tools/convert_bevdet_to_TRT.py $config $checkpoint $work_dir --fuse-conv-bn --fp16 -python tools/analysis_tools/benchmark_trt.py $config $engine -python tools/analysis_tools/benchmark_trt.py $config $engine --eval -# ===> per class IoU of 6019 samples: -# ===> others - IoU = 5.97 -# ===> barrier - IoU = 36.37 -# ===> bicycle - IoU = 10.14 -# ===> bus - IoU = 35.47 -# ===> car - IoU = 41.57 -# ===> construction_vehicle - IoU = 15.73 -# ===> motorcycle - IoU = 14.8 -# ===> pedestrian - IoU = 15.65 -# ===> traffic_cone - IoU = 14.46 -# ===> trailer - IoU = 27.47 -# ===> truck - IoU = 29.39 -# ===> driveable_surface - IoU = 77.14 -# ===> other_flat - IoU = 34.66 -# ===> sidewalk - IoU = 46.44 -# ===> terrain - IoU = 51.05 -# ===> manmade - IoU = 35.79 -# ===> vegetation - IoU = 31.19 -# ===> mIoU of 6019 samples: 30.78 - -# fp32 test -engine=work_dirs/${exp_name}/onnx_trt/bevdet_fuse.engine -python tools/convert_bevdet_to_TRT.py $config $checkpoint $work_dir --fuse-conv-bn -python tools/analysis_tools/benchmark_trt.py $config $engine -python tools/analysis_tools/benchmark_trt.py $config $engine --eval -# ===> per class IoU of 6019 samples: -# ===> others - IoU = 5.97 -# ===> barrier - IoU = 36.37 -# ===> bicycle - IoU = 10.15 -# ===> bus - IoU = 35.46 -# ===> car - IoU = 41.56 -# ===> construction_vehicle - IoU = 15.73 -# ===> motorcycle - IoU = 14.78 -# ===> pedestrian - IoU = 15.64 -# ===> traffic_cone - IoU = 14.44 -# ===> trailer - IoU = 27.46 -# ===> truck - IoU = 29.39 -# ===> driveable_surface - IoU = 77.14 -# ===> other_flat - IoU = 34.68 -# ===> sidewalk - IoU = 46.44 -# ===> terrain - IoU = 51.05 -# ===> manmade - IoU = 35.79 -# ===> vegetation - IoU = 31.18 -# ===> mIoU of 6019 samples: 30.78 - -``` - - -3. cmd for flashoccv2 -``` -exp_name=flashoccv2-r50-depth -fold_name=flashoccv2 -config=projects/configs/${fold_name}/${exp_name}-trt.py -checkpoint=work_dirs/${exp_name}/epoch_24_ema.pth -work_dir=work_dirs/${exp_name}/onnx_trt/ - -# fp16 test -engine=work_dirs/${exp_name}/onnx_trt/bevdet_fp16_fuse.engine -python tools/convert_bevdet_to_TRT.py $config $checkpoint $work_dir --fuse-conv-bn --fp16 -python tools/analysis_tools/benchmark_trt.py $config $engine -python tools/analysis_tools/benchmark_trt.py $config $engine --eval -``` - -# Flops and params -```shell -python tools/analysis_tools/get_flops.py projects/configs/bevdet_occ/bevdet-occ-r50.py --modality image --shape 256 704 -python tools/analysis_tools/get_flops.py projects/configs/flashocc/flashocc-r50-M0.py --modality image --shape 256 704 -python tools/analysis_tools/get_flops.py projects/configs/flashocc/flashocc-r50.py --modality image --shape 256 704 -python tools/analysis_tools/get_flops.py projects/configs/flashocc/flashocc-stbase-4d-stereo-512x1408.py --modality image --shape 512 1408 -python tools/analysis_tools/get_flops.py projects/configs/flashoccv2/flashoccv2-r50-depth.py --modality image --shape 256 704 -python tools/analysis_tools/get_flops.py projects/configs/flashoccv2/flashoccv2-r50-depth-tiny.py --modality image --shape 256 704 -``` \ No newline at end of file diff --git a/docker-hub/FlashOCC/Flashocc/doc/model_training.md b/docker-hub/FlashOCC/Flashocc/doc/model_training.md deleted file mode 100644 index fee2c6cab0f8f9466691876e97c125ffda84c339..0000000000000000000000000000000000000000 --- a/docker-hub/FlashOCC/Flashocc/doc/model_training.md +++ /dev/null @@ -1,29 +0,0 @@ - -#### Train model -```shell -# single gpu -python tools/train.py $config -# multiple gpu -./tools/dist_train.sh $config num_gpu -``` - -#### Test model -```shell -# single gpu -python tools/test.py $config $checkpoint --eval mAP -# multiple gpu -./tools/dist_test.sh $config $checkpoint num_gpu --eval mAP -# ray-iou metric -./tools/dist_test.sh $config $checkpoint num_gpu --eval ray-iou -``` - -#### FPS for Panoptic-FlashOcc -```shell -# for single-frame -python tools/analysis_tools/benchmark.py config ckpt -python tools/analysis_tools/benchmark.py config ckpt --w_pano - -# for multi-frame -python tools/analysis_tools/benchmark_sequential.py config ckpt -python tools/analysis_tools/benchmark_sequential.py config ckpt --w_pano -``` diff --git a/docker-hub/FlashOCC/Flashocc/doc/visualization.md b/docker-hub/FlashOCC/Flashocc/doc/visualization.md deleted file mode 100644 index 4b4c41e9835fd66bed5fe690763c3fab9e4ea8ab..0000000000000000000000000000000000000000 --- a/docker-hub/FlashOCC/Flashocc/doc/visualization.md +++ /dev/null @@ -1,17 +0,0 @@ - -# FlashOcc -```shell -# step 1. generate result -bash tools/dist_test.sh projects/configs/flashocc/flashocc-r50.py ckpts/flashocc-r50-256x704.pth 4 --eval map --eval-options show_dir=work_dirs/flashocc_r50/results -# step 2. visualization -python tools/analysis_tools/vis_occ.py work_dirs/flashocc_r50/results/ --root_path ./data/nuscenes --save_path ./vis -``` - -# Panoptic-FlashOcc -```shell - -exp_name=panoptic-flashocc-r50-depth4d-longterm8f-pano -python tools/vis_occ.py --config projects/configs/panoptic-flashocc/${exp_name}.py --weights work_dirs/${exp_name}/epoch_24_ema.pth --viz-dir vis/${exp_name} --draw-pano-gt - -``` - diff --git a/docker-hub/FlashOCC/Flashocc/figs/overview.png b/docker-hub/FlashOCC/Flashocc/figs/overview.png deleted file mode 100644 index 43393b70c6918d9c20a49a146393244863a06989..0000000000000000000000000000000000000000 Binary files a/docker-hub/FlashOCC/Flashocc/figs/overview.png and /dev/null differ diff --git a/docker-hub/FlashOCC/Flashocc/figs/pano.png b/docker-hub/FlashOCC/Flashocc/figs/pano.png deleted file mode 100644 index a07226d18247fda799e984b317bd09689bc7447d..0000000000000000000000000000000000000000 Binary files a/docker-hub/FlashOCC/Flashocc/figs/pano.png and /dev/null differ diff --git a/docker-hub/FlashOCC/Flashocc/figs/panoptic_flashOcc.jpg b/docker-hub/FlashOCC/Flashocc/figs/panoptic_flashOcc.jpg deleted file mode 100644 index 78c8318cf5c842cc036bda093ffea103b2857ea1..0000000000000000000000000000000000000000 Binary files a/docker-hub/FlashOCC/Flashocc/figs/panoptic_flashOcc.jpg and /dev/null differ diff --git a/docker-hub/FlashOCC/Flashocc/figs/panoptic_flashOcc.png b/docker-hub/FlashOCC/Flashocc/figs/panoptic_flashOcc.png deleted file mode 100644 index f4a0b4cf573e4c47947989cd42f4610b087e643a..0000000000000000000000000000000000000000 Binary files a/docker-hub/FlashOCC/Flashocc/figs/panoptic_flashOcc.png and /dev/null differ diff --git a/docker-hub/FlashOCC/Flashocc/figs/performance.png b/docker-hub/FlashOCC/Flashocc/figs/performance.png deleted file mode 100644 index d6e6381bdb92c5583ebc8acfb482c73cda8932c8..0000000000000000000000000000000000000000 Binary files a/docker-hub/FlashOCC/Flashocc/figs/performance.png and /dev/null differ diff --git a/docker-hub/FlashOCC/Flashocc/figs/performance_flashocc.jpg b/docker-hub/FlashOCC/Flashocc/figs/performance_flashocc.jpg deleted file mode 100644 index 80e89e9965ac1278dad5cb186cdc1cc83a7d7330..0000000000000000000000000000000000000000 Binary files a/docker-hub/FlashOCC/Flashocc/figs/performance_flashocc.jpg and /dev/null differ diff --git a/docker-hub/FlashOCC/Flashocc/figs/sem.png b/docker-hub/FlashOCC/Flashocc/figs/sem.png deleted file mode 100644 index 78202c16dedfbfb8b4b6b3eb4fbb17db10dcfa0a..0000000000000000000000000000000000000000 Binary files a/docker-hub/FlashOCC/Flashocc/figs/sem.png and /dev/null differ diff --git a/docker-hub/FlashOCC/Flashocc/figs/visualization.png b/docker-hub/FlashOCC/Flashocc/figs/visualization.png deleted file mode 100644 index e6aac9691146456673a0d3cb20a9dcc209ddd876..0000000000000000000000000000000000000000 Binary files a/docker-hub/FlashOCC/Flashocc/figs/visualization.png and /dev/null differ diff --git a/docker-hub/FlashOCC/Flashocc/lib/dvr/dvr.cu b/docker-hub/FlashOCC/Flashocc/lib/dvr/dvr.cu deleted file mode 100644 index 8010eb8854a632321fc5db1643d805927bd312b7..0000000000000000000000000000000000000000 --- a/docker-hub/FlashOCC/Flashocc/lib/dvr/dvr.cu +++ /dev/null @@ -1,747 +0,0 @@ -// Acknowledgments: https://github.com/tarashakhurana/4d-occ-forecasting -// Modified by Haisong Liu - -#include
-MapTR
-
-
-#
-### News
-* **`Aug. 31th, 2023`:** initial MapTRv2 is released at ***maptrv2*** branch. Please run `git checkout maptrv2` to use it.
-* **`Aug. 14th, 2023`:** As required by many researchers, the code of MapTR-based map annotation framework (VMA) will be released at https://github.com/hustvl/VMA recently.
-* **`Aug. 10th, 2023`:** We release [MapTRv2](https://arxiv.org/abs/2308.05736) on Arxiv. MapTRv2 demonstrates much stronger performance and much faster convergence. To better meet the requirement of the downstream planner (like [PDM](https://github.com/autonomousvision/nuplan_garage)), we introduce an extra semantic——centerline (using path-wise modeling proposed by [LaneGAP](https://github.com/hustvl/LaneGAP)). Code & model will be released in late August. Please stay tuned!
-* **`May. 12th, 2023`:** MapTR now support various bevencoder, such as [BEVFormer encoder](projects/configs/maptr/maptr_tiny_r50_24e_bevformer.py) and [BEVFusion bevpool](projects\configs\maptr\maptr_tiny_r50_24e_bevpool.py). Check it out!
-* **`Apr. 20th, 2023`:** Extending MapTR to a general map annotation framework ([paper](https://arxiv.org/pdf/2304.09807.pdf), [code](https://github.com/hustvl/VMA)), with high flexibility in terms of spatial scale and element type.
-* **`Mar. 22nd, 2023`:** By leveraging MapTR, VAD ([paper](https://arxiv.org/abs/2303.12077), [code](https://github.com/hustvl/VAD)) models the driving scene as fully vectorized representation, achieving SoTA end-to-end planning performance!
-* **`Jan. 21st, 2023`:** MapTR is accepted to ICLR 2023 as **Spotlight Presentation**!
-* **`Nov. 11st, 2022`:** We release an initial version of MapTR.
-* **`Aug. 31st, 2022`:** We released our paper on Arxiv. Code/Models are coming soon. Please stay tuned! ☕️
-
-
-## Introduction
-