+ 
+ 
+
+
+**注意:**
+
+ - **PaddleYOLO** 代码库协议为 **[GPL 3.0](LICENSE)**，[YOLOv5](configs/yolov5)、[YOLOv6](configs/yolov6)、[YOLOv7](configs/yolov7)和[YOLOv8](configs/yolov8)这几类模型代码不合入[PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection)，其余YOLO模型推荐在[PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection)中使用，**会最先发布PP-YOLO系列特色检测模型的最新进展**；
+ - **PaddleYOLO**代码库**推荐使用paddlepaddle-2.4.2以上的版本**，请参考[官网](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/install/pip/linux-pip.html)下载对应适合版本，**Windows平台请安装paddle develop版本**；
+ - **PaddleYOLO 的[Roadmap](https://github.com/PaddlePaddle/PaddleYOLO/issues/44)** issue用于收集用户的需求，欢迎提出您的建议和需求；
+
+## 教程
+
+
+
+
+
+
+安装
+ +Clone 代码库和安装 [requirements.txt](./requirements.txt)，环境需要在一个 +[**Python>=3.7.0**](https://www.python.org/) 下的环境，且需要安装 +[**PaddlePaddle>=2.4.2**](https://www.paddlepaddle.org.cn/install/)。 + +```bash +git clone https://github.com/PaddlePaddle/PaddleYOLO # clone +cd PaddleYOLO +pip install -r requirements.txt # install +``` + +
+
+
+训练/验证/预测/
+将以下命令写在一个脚本文件里如```run.sh```，一键运行命令为：```sh run.sh```，也可命令行一句句去运行。 + +```bash +model_name=ppyoloe # 可修改，如 yolov7 +job_name=ppyoloe_plus_crn_s_80e_coco # 可修改，如 yolov7_tiny_300e_coco + +config=configs/${model_name}/${job_name}.yml +log_dir=log_dir/${job_name} +# weights=https://bj.bcebos.com/v1/paddledet/models/${job_name}.pdparams +weights=output/${job_name}/model_final.pdparams + +# 1.训练（单卡/多卡），加 --eval 表示边训边评估，加 --amp 表示混合精度训练 +# CUDA_VISIBLE_DEVICES=0 python tools/train.py -c ${config} --eval --amp +python -m paddle.distributed.launch --log_dir=${log_dir} --gpus 0,1,2,3,4,5,6,7 tools/train.py -c ${config} --eval --amp + +# 2.评估，加 --classwise 表示输出每一类mAP +CUDA_VISIBLE_DEVICES=0 python tools/eval.py -c ${config} -o weights=${weights} --classwise + +# 3.预测 (单张图/图片文件夹） +CUDA_VISIBLE_DEVICES=0 python tools/infer.py -c ${config} -o weights=${weights} --infer_img=demo/000000014439_640x640.jpg --draw_threshold=0.5 +# CUDA_VISIBLE_DEVICES=0 python tools/infer.py -c ${config} -o weights=${weights} --infer_dir=demo/ --draw_threshold=0.5 +``` + +
+
+
+
+部署/测速
+ +将以下命令写在一个脚本文件里如```run.sh```，一键运行命令为：```sh run.sh```，也可命令行一句句去运行。 + +```bash +model_name=ppyoloe # 可修改，如 yolov7 +job_name=ppyoloe_plus_crn_s_80e_coco # 可修改，如 yolov7_tiny_300e_coco + +config=configs/${model_name}/${job_name}.yml +log_dir=log_dir/${job_name} +# weights=https://bj.bcebos.com/v1/paddledet/models/${job_name}.pdparams +weights=output/${job_name}/model_final.pdparams + +# 4.导出模型，以下3种模式选一种 +## 普通导出，加trt表示用于trt加速，对NMS和silu激活函数提速明显 +CUDA_VISIBLE_DEVICES=0 python tools/export_model.py -c ${config} -o weights=${weights} # trt=True + +## exclude_post_process去除后处理导出，返回和YOLOv5导出ONNX时相同格式的concat后的1个Tensor，是未缩放回原图的坐标+分类置信度 +# CUDA_VISIBLE_DEVICES=0 python tools/export_model.py -c ${config} -o weights=${weights} exclude_post_process=True # trt=True + +## exclude_nms去除NMS导出，返回2个Tensor，是缩放回原图后的坐标和分类置信度 +# CUDA_VISIBLE_DEVICES=0 python tools/export_model.py -c ${config} -o weights=${weights} exclude_nms=True # trt=True + +# 5.部署预测，注意不能使用 去除后处理 或 去除NMS 导出后的模型去预测 +CUDA_VISIBLE_DEVICES=0 python deploy/python/infer.py --model_dir=output_inference/${job_name} --image_file=demo/000000014439_640x640.jpg --device=GPU + +# 6.部署测速，加 “--run_mode=trt_fp16” 表示在TensorRT FP16模式下测速，注意如需用到 trt_fp16 则必须为加 trt=True 导出的模型 +CUDA_VISIBLE_DEVICES=0 python deploy/python/infer.py --model_dir=output_inference/${job_name} --image_file=demo/000000014439_640x640.jpg --device=GPU --run_benchmark=True # --run_mode=trt_fp16 + +# 7.onnx导出，一般结合 exclude_post_process去除后处理导出的模型 +paddle2onnx --model_dir output_inference/${job_name} --model_filename model.pdmodel --params_filename model.pdiparams --opset_version 12 --save_file ${job_name}.onnx + +# 8.onnx trt测速 +/usr/local/TensorRT-8.0.3.4/bin/trtexec --onnx=${job_name}.onnx --workspace=4096 --avgRuns=10 --shapes=input:1x3x640x640 --fp16 +/usr/local/TensorRT-8.0.3.4/bin/trtexec --onnx=${job_name}.onnx --workspace=4096 --avgRuns=10 --shapes=input:1x3x640x640 --fp32 +``` + +- 如果想切换模型，只要修改开头两行即可，如: + ``` + model_name=yolov7 + job_name=yolov7_tiny_300e_coco + ``` +- 导出**onnx**，首先安装[Paddle2ONNX](https://github.com/PaddlePaddle/Paddle2ONNX)，`pip install paddle2onnx`； +- **统计FLOPs(G)和Params(M)**，首先安装[PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim)，`pip install paddleslim`，然后设置[runtime.yml](configs/runtime.yml)里`print_flops: True`和`print_params: True`，并且注意确保是**单尺度**下如640x640，**打印的是MACs，FLOPs=2*MACs**。 + +
+
+
+
+## 更新日志
+
+* 【2023/05/21】支持[RT-DETR](configs/rtdetr)、[YOLOv8](configs/yolov8)、[YOLOv5u](configs/yolov5/yolov5u)和[YOLOv7u](configs/yolov7/yolov7u)训练全流程，支持[YOLOv6Lite](configs/yolov6/yolov6lite)预测和部署；
+* 【2023/03/13】支持[YOLOv5u](configs/yolov5/yolov5u)和[YOLOv7u](configs/yolov7/yolov7u)预测和部署；
+* 【2023/01/10】支持[YOLOv8](configs/yolov8)预测和部署；
+* 【2022/09/29】支持[RTMDet](configs/rtmdet)预测和部署；
+* 【2022/09/26】发布[PaddleYOLO](https://github.com/PaddlePaddle/PaddleYOLO)模型套件，请参照[ModelZoo](docs/MODEL_ZOO_cn.md)；
+* 【2022/09/19】支持[YOLOv6](configs/yolov6)新版，包括n/t/s/m/l模型；
+* 【2022/08/23】发布`YOLOSeries`代码库: 支持`YOLOv3`,`PP-YOLOE`,`PP-YOLOE+`,`YOLOX`,`YOLOv5`,`YOLOv6`,`YOLOv7`等YOLO模型，支持`ConvNeXt`骨干网络高精度版`PP-YOLOE`,`YOLOX`和`YOLOv5`等模型，支持PaddleSlim无损加速量化训练`PP-YOLOE`,`YOLOv5`,`YOLOv6`和`YOLOv7`等模型，详情可阅读[此文章](https://mp.weixin.qq.com/s/Hki01Zs2lQgvLSLWS0btrA)；
+
+
+## [训练自定义数据集](https://github.com/PaddlePaddle/PaddleYOLO/issues/43)
+ +- 请参照[文档](docs/MODEL_ZOO_cn.md#自定义数据集)和[issue](https://github.com/PaddlePaddle/PaddleYOLO/issues/43)； +- PaddleDetection团队提供了**基于PP-YOLOE的各种垂类检测模型**的配置文件和权重，用户也可以作为参考去使用自定义数据集。请参考 [PP-YOLOE application](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.6/configs/ppyoloe/application)、[pphuman](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.6/configs/pphuman)、[ppvehicle](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.6/configs/ppvehicle)、[visdrone](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.6/configs/visdrone) 和 [smalldet](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.6/configs/smalldet)。 +- PaddleDetection团队也提供了**VOC数据集的各种YOLO模型**的配置文件和权重，用户也可以作为参考去使用自定义数据集。请参考 [voc](configs/voc)。 +- 训练自定义数据集之前请先**确保加载了对应COCO权重作为预训练**，将配置文件中的`pretrain_weights: `设置为对应COCO模型训好的权重，一般会提示head分类层卷积的通道数没对应上，属于正常现象，是由于自定义数据集一般和COCO数据集种类数不一致； +- YOLO检测模型建议**总`batch_size`至少大于`64`**去训练，如果资源不够请**换小模型**或**减小模型的输入尺度**，为了保障较高检测精度，**尽量不要尝试单卡训和总`batch_size`小于`64`训**； + +
产品动态
+
+- 🔥 **2023.3.14：PaddleYOLO发布[release/2.6版本](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.6)**
+ - 💡 模型套件：
+ - 支持`YOLOv8`,`YOLOv5u`,`YOLOv7u`等YOLO模型预测和部署；
+ - 支持`Swin-Transformer`、`ViT`、`FocalNet`骨干网络高精度版`PP-YOLOE+`等模型；
+ - 支持`YOLOv8`在[FastDeploy](https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/vision/detection/paddledetection)中多硬件快速部署；
+
+- 🔥 **2022.9.26：PaddleYOLO发布[release/2.5版本](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5)**
+ - 💡 模型套件：
+ - 发布[PaddleYOLO](https://github.com/PaddlePaddle/PaddleYOLO)模型套件: 支持`YOLOv3`,`PP-YOLOE`,`PP-YOLOE+`,`YOLOX`,`YOLOv5`,`YOLOv6`,`YOLOv7`等YOLO模型，支持`ConvNeXt`骨干网络高精度版`PP-YOLOE`,`YOLOX`和`YOLOv5`等模型，支持PaddleSlim无损加速量化训练`PP-YOLOE`,`YOLOv5`,`YOLOv6`和`YOLOv7`等模型；
+
+- 🔥 **2022.8.26：PaddleDetection发布[release/2.5版本](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.5)**
+ - 🗳 特色模型：
+ - 发布[PP-YOLOE+](configs/ppyoloe)，最高精度提升2.4% mAP，达到54.9% mAP，模型训练收敛速度提升3.75倍，端到端预测速度最高提升2.3倍；多个下游任务泛化性提升
+ - 发布[PicoDet-NPU](configs/picodet)模型，支持模型全量化部署；新增[PicoDet](configs/picodet)版面分析模型
+ - 发布[PP-TinyPose升级版](./configs/keypoint/tiny_pose/)增强版，在健身、舞蹈等场景精度提升9.1% AP，支持侧身、卧躺、跳跃、高抬腿等非常规动作
+ - 🔮 场景能力：
+ - 发布行人分析工具[PP-Human v2](./deploy/pipeline)，新增打架、打电话、抽烟、闯入四大行为识别，底层算法性能升级，覆盖行人检测、跟踪、属性三类核心算法能力，提供保姆级全流程开发及模型优化策略，支持在线视频流输入
+ - 首次发布[PP-Vehicle](./deploy/pipeline)，提供车牌识别、车辆属性分析（颜色、车型）、车流量统计以及违章检测四大功能，兼容图片、在线视频流、视频输入，提供完善的二次开发文档教程
+ - 💡 前沿算法：
+ - 全面覆盖的[YOLO家族](https://github.com/PaddlePaddle/PaddleYOLO)经典与最新模型: 包括YOLOv3，百度飞桨自研的实时高精度目标检测检测模型PP-YOLOE，以及前沿检测算法YOLOv4、YOLOv5、YOLOX，YOLOv6及YOLOv7
+ - 新增基于[ViT](configs/vitdet)骨干网络高精度检测模型，COCO数据集精度达到55.7% mAP；新增[OC-SORT](configs/mot/ocsort)多目标跟踪模型；新增[ConvNeXt](configs/convnext)骨干网络
+ - 📋 产业范例：新增[智能健身](https://aistudio.baidu.com/aistudio/projectdetail/4385813)、[打架识别](https://aistudio.baidu.com/aistudio/projectdetail/4086987?channelType=0&channel=0)、[来客分析](https://aistudio.baidu.com/aistudio/projectdetail/4230123?channelType=0&channel=0)、车辆结构化范例
+
+- 2022.3.24：PaddleDetection发布[release/2.4版本](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.4)
+ - 发布高精度云边一体SOTA目标检测模型[PP-YOLOE](configs/ppyoloe)，提供s/m/l/x版本，l版本COCO test2017数据集精度51.6%，V100预测速度78.1 FPS，支持混合精度训练，训练较PP-YOLOv2加速33%，全系列多尺度模型，满足不同硬件算力需求，可适配服务器、边缘端GPU及其他服务器端AI加速卡。
+ - 发布边缘端和CPU端超轻量SOTA目标检测模型[PP-PicoDet增强版](configs/picodet)，精度提升2%左右，CPU预测速度提升63%，新增参数量0.7M的PicoDet-XS模型，提供模型稀疏化和量化功能，便于模型加速，各类硬件无需单独开发后处理模块，降低部署门槛。
+ - 发布实时行人分析工具[PP-Human](deploy/pipeline)，支持行人跟踪、人流量统计、人体属性识别与摔倒检测四大能力，基于真实场景数据特殊优化，精准识别各类摔倒姿势，适应不同环境背景、光线及摄像角度。
+ - 新增[YOLOX](configs/yolox)目标检测模型，支持nano/tiny/s/m/l/x版本，x版本COCO val2017数据集精度51.8%。
+
+- [更多版本发布](https://github.com/PaddlePaddle/PaddleDetection/releases)
+
+## 
简介
+
+**PaddleDetection**为基于飞桨PaddlePaddle的端到端目标检测套件，内置**30+模型算法**及**250+预训练模型**，覆盖**目标检测、实例分割、跟踪、关键点检测**等方向，其中包括**服务器端和移动端高精度、轻量级**产业级SOTA模型、冠军方案和学术前沿算法，并提供配置化的网络模块组件、十余种数据增强策略和损失函数等高阶优化支持和多种部署方案，在打通数据处理、模型开发、训练、压缩、部署全流程的基础上，提供丰富的案例及教程，加速算法产业落地应用。
+
+
+ 
+
+
+##
+
特性
+
+- **模型丰富**: 包含**目标检测**、**实例分割**、**人脸检测**、****关键点检测****、**多目标跟踪**等**250+个预训练模型**，涵盖多种**全球竞赛冠军**方案。
+- **使用简洁**：模块化设计，解耦各个网络组件，开发者轻松搭建、试用各种检测模型及优化策略，快速得到高性能、定制化的算法。
+- **端到端打通**: 从数据增强、组网、训练、压缩、部署端到端打通，并完备支持**云端**/**边缘端**多架构、多设备部署。
+- **高性能**: 基于飞桨的高性能内核，模型训练速度及显存占用优势明显。支持FP16训练, 支持多机训练。
+
+
+ 
+
+
+##
+
技术交流
+
+- 如果你发现任何PaddleDetection存在的问题或者是建议, 欢迎通过[GitHub Issues](https://github.com/PaddlePaddle/PaddleDetection/issues)给我们提issues。
+
+- **欢迎加入PaddleDetection 微信用户群（扫码填写问卷即可入群）**
+ - **入群福利 💎：获取PaddleDetection团队整理的重磅学习大礼包🎁**
+ - 📊 福利一：获取飞桨联合业界企业整理的开源数据集
+ - 👨🏫 福利二：获取PaddleDetection历次发版直播视频与最新直播咨询
+ - 🗳 福利三：获取垂类场景预训练模型集合，包括工业、安防、交通等5+行业场景
+ - 🗂 福利四：获取10+全流程产业实操范例，覆盖火灾烟雾检测、人流量计数等产业高频场景
+
+ 
+
+
+##
+ 
套件结构概览
+
+| + Architectures + | ++ Backbones + | ++ Components + | ++ Data Augmentation + | +
+
Object Detection+
|
+
+ Details+
|
+
+ Common+
FPN+
Loss+
Post-processing+
Speed+
|
+
+ Details+
|
+
模型性能概览
+
+
+
+
+云端模型性能对比
+ +各模型结构和骨干网络的代表模型在COCO数据集上精度mAP和单卡Tesla V100上预测速度(FPS)对比图。 + +
+ 
+
+
+**说明：**
+
+- `PP-YOLOE`是对`PP-YOLO v2`模型的进一步优化，在COCO数据集精度51.6%，Tesla V100预测速度78.1FPS
+- `PP-YOLOE+`是对`PPOLOE`模型的进一步优化，在COCO数据集精度53.3%，Tesla V100预测速度78.1FPS
+- 图中模型均可在[模型库](#模型库)中获取
+
+
+
+
+
+## 移动端模型性能对比
+ +各移动端模型在COCO数据集上精度mAP和高通骁龙865处理器上预测速度(FPS)对比图。 + +
+ 
+
+
+**说明：**
+
+- 测试数据均使用高通骁龙865(4\*A77 + 4\*A55)处理器batch size为1, 开启4线程测试，测试使用NCNN预测库，测试脚本见[MobileDetBenchmark](https://github.com/JiweiMaster/MobileDetBenchmark)
+- [PP-PicoDet](configs/picodet)及[PP-YOLO-Tiny](configs/ppyolo)为PaddleDetection自研模型，其余模型PaddleDetection暂未提供
+
+
+
模型库
+
+
+
+
+
+## 1. 通用检测
+ +#### [PP-YOLOE+](./configs/ppyoloe)系列 推荐场景：Nvidia V100, T4等云端GPU和Jetson系列等边缘端设备 + +| 模型名称 | COCO精度（mAP） | V100 TensorRT FP16速度(FPS) | 配置文件 | 模型下载 | +|:---------- |:-----------:|:-------------------------:|:-----------------------------------------------------:|:------------------------------------------------------------------------------------:| +| PP-YOLOE+_s | 43.9 | 333.3 | [链接](configs/ppyoloe/ppyoloe_plus_crn_s_80e_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/ppyoloe_plus_crn_s_80e_coco.pdparams) | +| PP-YOLOE+_m | 50.0 | 208.3 | [链接](configs/ppyoloe/ppyoloe_plus_crn_m_80e_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/ppyoloe_plus_crn_m_80e_coco.pdparams) | +| PP-YOLOE+_l | 53.3 | 149.2 | [链接](configs/ppyoloe/ppyoloe_plus_crn_l_80e_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/ppyoloe_plus_crn_m_80e_coco.pdparams) | +| PP-YOLOE+_x | 54.9 | 95.2 | [链接](configs/ppyoloe/ppyoloe_plus_crn_x_80e_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/ppyoloe_plus_crn_x_80e_coco.pdparams) | + +#### 前沿检测算法 + +| 模型名称 | COCO精度（mAP） | V100 TensorRT FP16速度(FPS) | 配置文件 | 模型下载 | +|:------------------------------------------------------------------ |:-----------:|:-------------------------:|:------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------:| +| [YOLOX-l](configs/yolox) | 50.1 | 107.5 | [链接](configs/yolox/yolox_l_300e_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/yolox_l_300e_coco.pdparams) | +| [YOLOv5-l](configs/yolov5) | 48.6 | 136.0 | [链接](configs/yolov5/yolov5_l_300e_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/yolov5_l_300e_coco.pdparams) | +| [YOLOv7-l](configs/yolov7) | 51.0 | 135.0 | [链接](configs/yolov7/yolov7_l_300e_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/yolov7_l_300e_coco.pdparams) | + +
文档教程
+
+### 入门教程
+
+- [安装说明](docs/tutorials/INSTALL_cn.md)
+- [快速体验](docs/tutorials/QUICK_STARTED_cn.md)
+- [数据准备](docs/tutorials/data/README.md)
+- [PaddleDetection全流程使用](docs/tutorials/GETTING_STARTED_cn.md)
+- [FAQ/常见问题汇总](docs/tutorials/FAQ)
+
+### 进阶教程
+
+- 参数配置
+
+ - [PP-YOLO参数说明](docs/tutorials/config_annotation/ppyolo_r50vd_dcn_1x_coco_annotation.md)
+
+- 模型压缩(基于[PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim))
+
+ - [剪裁/量化/蒸馏教程](configs/slim)
+
+- [推理部署](deploy/README.md)
+
+ - [模型导出教程](deploy/EXPORT_MODEL.md)
+ - [Paddle Inference部署](deploy/README.md)
+ - [Python端推理部署](deploy/python)
+ - [C++端推理部署](deploy/cpp)
+ - [Paddle-Lite部署](deploy/lite)
+ - [Paddle Serving部署](deploy/serving)
+ - [ONNX模型导出](deploy/EXPORT_ONNX_MODEL.md)
+ - [推理benchmark](deploy/BENCHMARK_INFER.md)
+
+- 进阶开发
+
+ - [数据处理模块](docs/advanced_tutorials/READER.md)
+ - [新增检测模型](docs/advanced_tutorials/MODEL_TECHNICAL.md)
+ - 二次开发教程
+ - [目标检测](docs/advanced_tutorials/customization/detection.md)
+
+
+## 
版本更新
+
+版本更新内容请参考[版本更新文档](docs/CHANGELOG.md)
+
+
+## 
许可证书
+
+本项目的发布受[GPL-3.0 license](LICENSE)许可认证。
+
+
+## 
引用
+
+```
+@misc{ppdet2019,
+title={PaddleDetection, Object detection and instance segmentation toolkit based on PaddlePaddle.},
+author={PaddlePaddle Authors},
+howpublished = {\url{https://github.com/PaddlePaddle/PaddleDetection}},
+year={2019}
+}
+```
diff --git a/README_en.md b/README_en.md
new file mode 100644
index 0000000000000000000000000000000000000000..947c8fe33fb37db0f49ba8ed9c0be98cebea2907
--- /dev/null
+++ b/README_en.md
@@ -0,0 +1,446 @@
+[简体中文](README_cn.md) | English
+
+## Introduction
+
+**PaddleYOLO** is a YOLO series toolbox based on [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection), **only relevant codes of YOLO series models are included**. It supports `YOLOv3`,`PP-YOLO`,`PP-YOLOv2`,`PP-YOLOE`,`PP-YOLOE+`,`YOLOX`,`YOLOv5`,`YOLOv6`,`YOLOv7`,`YOLOv8`,`YOLOv5u`,`YOLOv7u`,`RTMDet` and so on, see COCO dataset ModelZoo in [ModelZoo](docs/MODEL_ZOO_en.md) and [configs](configs/).
+
+
+
+
+
+
+**Notes：**
+ - The Licence of **PaddleYOLO** is **[GPL 3.0](LICENSE)**, the codes of [YOLOv5](configs/yolov5),[YOLOv6](configs/yolov6),[YOLOv7](configs/yolov7) and [YOLOv8](configs/yolov8) will not be merged into [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection). Except for these YOLO models, other YOLO models are recommended to use in [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection), **which will be the first to release the latest progress of PP-YOLO series detection model**;
+ - To use **PaddleYOLO**, **PaddlePaddle-2.3.2 or above is recommended**，please refer to the [official website](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/install/pip/linux-pip.html) to download the appropriate version. **For Windows platforms, please install the paddle develop version**;
+ - **PaddleYOLO's [Roadmap](https://github.com/PaddlePaddle/PaddleYOLO/issues/44)** issue collects feature requests from user, welcome to put forward any opinions and suggestions.
+
+## Tutorials
+
+
+
+
+
+
+
+Install
+ +Clone repo and install [requirements.txt](./requirements.txt) in a +[**Python>=3.7.0**](https://www.python.org/) environment, including +[**PaddlePaddle>=2.3.2**](https://www.paddlepaddle.org.cn/install/). + +```bash +git clone https://github.com/PaddlePaddle/PaddleYOLO # clone +cd PaddleYOLO +pip install -r requirements.txt # install +``` + +
+
+
+
+Training/Evaluation/Inference
+ +Write the following commands in a script file, such as ```run.sh```, and run as：```sh run.sh```. You can also run the command line sentence by sentence. + +```bash +model_name=ppyoloe # yolov7 +job_name=ppyoloe_plus_crn_s_80e_coco # yolov7_tiny_300e_coco + +config=configs/${model_name}/${job_name}.yml +log_dir=log_dir/${job_name} +# weights=https://bj.bcebos.com/v1/paddledet/models/${job_name}.pdparams +weights=output/${job_name}/model_final.pdparams + +# 1.training（single GPU / multi GPU） +# CUDA_VISIBLE_DEVICES=0 python tools/train.py -c ${config} --eval --amp +python -m paddle.distributed.launch --log_dir=${log_dir} --gpus 0,1,2,3,4,5,6,7 tools/train.py -c ${config} --eval --amp + +# 2.eval +CUDA_VISIBLE_DEVICES=0 python tools/eval.py -c ${config} -o weights=${weights} --classwise + +# 3.infer +CUDA_VISIBLE_DEVICES=0 python tools/infer.py -c ${config} -o weights=${weights} --infer_img=demo/000000014439_640x640.jpg --draw_threshold=0.5 +# CUDA_VISIBLE_DEVICES=0 python tools/infer.py -c ${config} -o weights=${weights} --infer_dir=demo/ --draw_threshold=0.5 +``` + +
+
+
+
+Deployment/Speed
+ +Write the following commands in a script file, such as ```run.sh```, and run as：```sh run.sh```. You can also run the command line sentence by sentence. + +```bash +model_name=ppyoloe # yolov7 +job_name=ppyoloe_plus_crn_s_80e_coco # yolov7_tiny_300e_coco + +config=configs/${model_name}/${job_name}.yml +log_dir=log_dir/${job_name} +# weights=https://bj.bcebos.com/v1/paddledet/models/${job_name}.pdparams +weights=output/${job_name}/model_final.pdparams + +# 4.export +CUDA_VISIBLE_DEVICES=0 python tools/export_model.py -c ${config} -o weights=${weights} # trt=True + +# CUDA_VISIBLE_DEVICES=0 python tools/export_model.py -c ${config} -o weights=${weights} exclude_post_process=True # trt=True + +# CUDA_VISIBLE_DEVICES=0 python tools/export_model.py -c ${config} -o weights=${weights} exclude_nms=True # trt=True + +# 5.deploy infer +CUDA_VISIBLE_DEVICES=0 python deploy/python/infer.py --model_dir=output_inference/${job_name} --image_file=demo/000000014439_640x640.jpg --device=GPU + +# 6.deploy speed, add '--run_mode=trt_fp16' to test in TensorRT FP16 mode +CUDA_VISIBLE_DEVICES=0 python deploy/python/infer.py --model_dir=output_inference/${job_name} --image_file=demo/000000014439_640x640.jpg --device=GPU --run_benchmark=True # --run_mode=trt_fp16 + +# 7.export onnx +paddle2onnx --model_dir output_inference/${job_name} --model_filename model.pdmodel --params_filename model.pdiparams --opset_version 12 --save_file ${job_name}.onnx + +# 8.onnx speed +/usr/local/TensorRT-8.0.3.4/bin/trtexec --onnx=${job_name}.onnx --workspace=4096 --avgRuns=10 --shapes=input:1x3x640x640 --fp16 +/usr/local/TensorRT-8.0.3.4/bin/trtexec --onnx=${job_name}.onnx --workspace=4096 --avgRuns=10 --shapes=input:1x3x640x640 --fp32 +``` + +**Note：** +- If you want to switch models, just modify the first two lines, such as: + ``` + model_name=yolov7 + job_name=yolov7_tiny_300e_coco + ``` +- For **exporting onnx**, you should install [Paddle2ONNX](https://github.com/PaddlePaddle/Paddle2ONNX) by `pip install paddle2onnx` at first. +- For **FLOPs(G) and Params(M)**, you should install [PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim) by `pip install paddleslim` at first, then set `print_flops: True` and `print_params: True` in [runtime.yml](configs/runtime.yml). Make sure **single scale** like 640x640, **MACs are printed，FLOPs=2*MACs**. + +
+
+
+
+## Updates
+
+* 【2023/03/13】Support [YOLOv5u](configs/yolov5/yolov5u) and [YOLOv7u](configs/yolov7/yolov7u) inference and deploy;
+* 【2023/01/10】Support [YOLOv8](configs/yolov8) inference and deploy;
+* 【2022/09/29】Support [RTMDet](configs/rtmdet) inference and deploy;
+* 【2022/09/26】Release [PaddleYOLO](https://github.com/PaddlePaddle/PaddleYOLO), see [ModelZoo](docs/MODEL_ZOO_en.md);
+* 【2022/09/19】Support the new version of [YOLOv6](configs/yolov6), including n/t/s/m/l model;
+* 【2022/08/23】Release `YOLOSeries` codebase: support `YOLOv3`,`PP-YOLOE`,`PP-YOLOE+`,`YOLOX`,`YOLOv5`,`YOLOv6` and `YOLOv7`; support using `ConvNeXt` backbone to get high-precision version of `PP-YOLOE`,`YOLOX` and `YOLOv5`; support PaddleSlim accelerated quantitative training `PP-YOLOE`,`YOLOv5`,`YOLOv6` and `YOLOv7`. For details, please read this [article](https://mp.weixin.qq.com/s/Hki01Zs2lQgvLSLWS0btrA)；
+
+
+## [Training Custom dataset](https://github.com/PaddlePaddle/PaddleYOLO/issues/43)
+ +- Please refer to [doc](docs/MODEL_ZOO_en.md#CustomDataset) and [issue](https://github.com/PaddlePaddle/PaddleYOLO/issues/43). +- PaddleDetection team provides various **feature detection models based on PP-YOLOE** , which can also be used as a reference to modify on your custom dataset. Please refer to [PP-YOLOE application](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.6/configs/ppyoloe/application), [pphuman](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.6/configs/pphuman), [ppvehicle](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.6/configs/ppvehicle), [visdrone](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.6/configs/visdrone) and [smalldet](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.6/configs/smalldet). +- PaddleDetection also provides **various YOLO models for VOC dataset** , which can also be used as a reference to modify on your custom dataset. Please refer to [voc](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.6/configs/voc). +- Please **ensure the corresponding COCO trained weights are loaded as pre-train weights** at first. Set the `pretrain_weights: ` with corresponding COCO trained weights in the config file, and it will generally prompt that the number of channels convolved by the head classification layer does not correspond, which is a normal phenomenon, because the number of types of user-defined data sets is generally inconsistent with that of COCO data sets. +- We recommend to use YOLO detection model **with a total `batch_size` at least greater than `64` to train**. If the resources are insufficient, please **use the smaller model** or **reduce the input size of the model**. To ensure high detection accuracy, **you'd better not try to using single GPU or total `batch_size` less than `64` for training**; + + Product Update
+
+- 🔥 **2023.3.14：Release PaddleYOLO [release/2.6](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.6)**
+ - 💡 Model kit：
+ - Support `YOLOv8`,`YOLOv5u`,`YOLOv7u` inference and deploy.
+ - Support `Swin-Transformer`、`ViT`、`FocalNet` backbone to get high-precision version of `PP-YOLOE+`.
+ - Support `YOLOv8` in [FastDeploy](https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/vision/detection/paddledetection).
+
+- 🔥 **2022.9.26：Release PaddleYOLO [release/2.5](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5)**
+ - 💡 Model kit：
+ - Release [PaddleYOLO](https://github.com/PaddlePaddle/PaddleYOLO): support `YOLOv3`,`PP-YOLOE`,`PP-YOLOE+`,`YOLOX`,`YOLOv5`,`YOLOv6` and `YOLOv7`; support using `ConvNeXt` backbone to get high-precision version of `PP-YOLOE`,`YOLOX` and `YOLOv5`; support PaddleSlim accelerated quantitative training `PP-YOLOE`,`YOLOv5`,`YOLOv6` and `YOLOv7`.
+
+- 🔥 **2022.8.26：PaddleDetection [release/2.5 version](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.5)**
+
+ - 🗳 Model features：
+
+ - Release [PP-YOLOE+](configs/ppyoloe): Increased accuracy by a maximum of 2.4% mAP to 54.9% mAP, 3.75 times faster model training convergence rate, and up to 2.3 times faster end-to-end inference speed; improved generalization for multiple downstream tasks
+ - Release [PicoDet-NPU](configs/picodet) model which supports full quantization deployment of models; add [PicoDet](configs/picodet) layout analysis model
+ - Release [PP-TinyPose Plus](./configs/keypoint/tiny_pose/). With 9.1% AP accuracy improvement in physical exercise, dance, and other scenarios, our PP-TinyPose Plus supports unconventional movements such as turning to one side, lying down, jumping, and high lifts
+
+ - 🔮 Functions in different scenarios
+
+ - Release the pedestrian analysis tool [PP-Human v2](./deploy/pipeline). It introduces four new behavior recognition: fighting, telephoning, smoking, and trespassing. The underlying algorithm performance is optimized, covering three core algorithm capabilities: detection, tracking, and attributes of pedestrians. Our model provides end-to-end development and model optimization strategies for beginners and supports online video streaming input.
+ - First release [PP-Vehicle](./deploy/pipeline), which has four major functions: license plate recognition, vehicle attribute analysis (color, model), traffic flow statistics, and violation detection. It is compatible with input formats, including pictures, online video streaming, and video. And we also offer our users a comprehensive set of tutorials for customization.
+
+ - 💡 Cutting-edge algorithms：
+
+ - Covers [YOLO family](https://github.com/PaddlePaddle/PaddleYOLO) classic and latest models: YOLOv3, PP-YOLOE (a real-time high-precision object detection model developed by Baidu PaddlePaddle), and cutting-edge detection algorithms such as YOLOv4, YOLOv5, YOLOX, YOLOv6, and YOLOv7
+ - Newly add high precision detection model based on [ViT](configs/vitdet) backbone network, with a 55.7% mAP accuracy on COCO dataset; newly add multi-object tracking model [OC-SORT](configs/mot/ocsort); newly add [ConvNeXt](configs/convnext) backbone network.
+
+ - 📋 Industrial applications: Newly add [Smart Fitness](https://aistudio.baidu.com/aistudio/projectdetail/4385813), [Fighting recognition](https://aistudio.baidu.com/aistudio/projectdetail/4086987?channelType=0&channel=0),[ and Visitor Analysis](https://aistudio.baidu.com/aistudio/projectdetail/4230123?channelType=0&channel=0).
+
+- 2022.3.24：PaddleDetection released[release/2.4 version](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.4)
+ - Release high-performanace SOTA object detection model [PP-YOLOE](configs/ppyoloe). It integrates cloud and edge devices and provides S/M/L/X versions. In particular, Verson L has the accuracy as 51.4% on COCO test 2017 dataset, inference speed as 78.1 FPS on a single Test V100. It supports mixed precision training, 33% faster than PP-YOLOv2. Its full range of multi-sized models can meet different hardware arithmetic requirements, and adaptable to server, edge-device GPU and other AI accelerator cards on servers.
+ - Release ultra-lightweight SOTA object detection model [PP-PicoDet Plus](configs/picodet) with 2% improvement in accuracy and 63% improvement in CPU inference speed. Add PicoDet-XS model with a 0.7M parameter, providing model sparsification and quantization functions for model acceleration. No specific post processing module is required for all the hardware, simplifying the deployment.
+ - Release the real-time pedestrian analysis tool [PP-Human](deploy/pphuman). It has four major functions: pedestrian tracking, visitor flow statistics, human attribute recognition and falling detection. For falling detection, it is optimized based on real-life data with accurate recognition of various types of falling posture. It can adapt to different environmental background, light and camera angle.
+ - Add [YOLOX](configs/yolox) object detection model with nano/tiny/S/M/L/X. X version has the accuracy as 51.8% on COCO Val2017 dataset.
+
+- [More releases](https://github.com/PaddlePaddle/PaddleDetection/releases)
+
+## Brief Introduction
+
+**PaddleDetection** is an end-to-end object detection development kit based on PaddlePaddle. Providing **over 30 model algorithm** and **over 250 pre-trained models**, it covers object detection, instance segmentation, keypoint detection, multi-object tracking. In particular, PaddleDetection offers **high- performance & light-weight** industrial SOTA models on **servers and mobile** devices, champion solution and cutting-edge algorithm. PaddleDetection provides various data augmentation methods, configurable network components, loss functions and other advanced optimization & deployment schemes. In addition to running through the whole process of data processing, model development, training, compression and deployment, PaddlePaddle also provides rich cases and tutorials to accelerate the industrial application of algorithm.
+
+
+ 
+
+
+##
+ Features
+
+- **Rich model library**: PaddleDetection provides over 250 pre-trained models including **object detection, instance segmentation, face recognition, multi-object tracking**. It covers a variety of **global competition champion** schemes.
+- **Simple to use**: Modular design, decoupling each network component, easy for developers to build and try various detection models and optimization strategies, quick access to high-performance, customized algorithm.
+- **Getting Through End to End**: PaddlePaddle gets through end to end from data augmentation, constructing models, training, compression, depolyment. It also supports multi-architecture, multi-device deployment for **cloud and edge** device.
+- **High Performance**: Due to the high performance core, PaddlePaddle has clear advantages in training speed and memory occupation. It also supports FP16 training and multi-machine training.
+
+
+ 
+
+
+##
+ Exchanges
+
+- If you have any question or suggestion, please give us your valuable input via [GitHub Issues](https://github.com/PaddlePaddle/PaddleDetection/issues)
+
+ Welcome to join PaddleDetection user groups on WeChat (scan the QR code, add and reply "D" to the assistant)
+
+
+
+
+
+##
+ Kit Structure
+
+| + Architectures + | ++ Backbones + | ++ Components + | ++ Data Augmentation + | +
+
Object Detection+
|
+
+ Details+
|
+
+ Common+
FPN+
Loss+
Post-processing+
Speed+
|
+
+ Details+
|
+
Model Performance
+
+
+
+
+Performance comparison of Cloud models
+ +The comparison between COCO mAP and FPS on Tesla V100 of representative models of each architectures and backbones. + +
+
+
+
+**Clarification：**
+
+- `PP-YOLOE` are optimized `PP-YOLO v2`. It reached accuracy as 51.4% on COCO dataset, inference speed as 78.1 FPS on Tesla V100
+- `PP-YOLOE+` are optimized `PP-YOLOE`. It reached accuracy as 53.3% on COCO dataset, inference speed as 78.1 FPS on Tesla V100
+- The models in the figure are available in the[ model library](#模型库)
+
+
+
+
+
+## Performance omparison on mobiles
+ +The comparison between COCO mAP and FPS on Qualcomm Snapdragon 865 processor of models on mobile devices. + +
+
+
+
+**Clarification：**
+
+- Tests were conducted on Qualcomm Snapdragon 865 (4 \*A77 + 4 \*A55) batch_size=1, 4 thread, and NCNN inference library, test script see [MobileDetBenchmark](https://github.com/JiweiMaster/MobileDetBenchmark)
+- [PP-PicoDet](configs/picodet) and [PP-YOLO-Tiny](configs/ppyolo) are self-developed models of PaddleDetection, and other models are not tested yet.
+
+
+ Model libraries
+
+
+
+
+## 1. General detection
+ +#### PP-YOLOE series Recommended scenarios: Cloud GPU such as Nvidia V100, T4 and edge devices such as Jetson series + +| Model | COCO Accuracy（mAP） | V100 TensorRT FP16 Speed(FPS) | Configuration | Download | +|:---------- |:------------------:|:-----------------------------:|:-------------------------------------------------------:|:----------------------------------------------------------------------------------------:| +| PP-YOLOE+_s | 43.9 | 333.3 | [link](configs/ppyoloe/ppyoloe_plus_crn_s_80e_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/ppyoloe_plus_crn_s_80e_coco.pdparams) | +| PP-YOLOE+_m | 50.0 | 208.3 | [link](configs/ppyoloe/ppyoloe_plus_crn_m_80e_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/ppyoloe_plus_crn_m_80e_coco.pdparams) | +| PP-YOLOE+_l | 53.3 | 149.2 | [link](configs/ppyoloe/ppyoloe_plus_crn_l_80e_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/ppyoloe_plus_crn_m_80e_coco.pdparams) | +| PP-YOLOE+_x | 54.9 | 95.2 | [link](configs/ppyoloe/ppyoloe_plus_crn_x_80e_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/ppyoloe_plus_crn_x_80e_coco.pdparams) | + +#### Frontier detection algorithm + +| Model | COCO Accuracy（mAP） | V100 TensorRT FP16 speed(FPS) | Configuration | Download | +|:-------- |:------------------:|:-----------------------------:|:--------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------:| +| [YOLOX-l](configs/yolox) | 50.1 | 107.5 | [Link](configs/yolox/yolox_l_300e_coco.yml) | [Download](https://paddledet.bj.bcebos.com/models/yolox_l_300e_coco.pdparams) | +| [YOLOv5-l](configs/yolov5) | 48.6 | 136.0 | [Link](configs/yolov5/yolov5_l_300e_coco.yml) | [Download](https://paddledet.bj.bcebos.com/models/yolov5_l_300e_coco.pdparams) | +| [YOLOv7-l](configs/yolov7) | 51.0 | 135.0 | [链接](configs/yolov7/yolov7_l_300e_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/yolov7_l_300e_coco.pdparams) | + +Document tutorials
+
+### Introductory tutorials
+
+- [Installation](docs/tutorials/INSTALL_cn.md)
+- [Quick start](docs/tutorials/QUICK_STARTED_cn.md)
+- [Data preparation](docs/tutorials/data/README.md)
+- [Geting Started on PaddleDetection](docs/tutorials/GETTING_STARTED_cn.md)
+- [FAQ]((docs/tutorials/FAQ)
+
+### Advanced tutorials
+
+- Configuration
+
+ - [PP-YOLO Configuration](docs/tutorials/config_annotation/ppyolo_r50vd_dcn_1x_coco_annotation.md)
+
+- Compression based on [PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim)
+
+ - [Pruning/Quantization/Distillation Tutorial](configs/slim)
+
+- [Inference deployment](deploy/README.md)
+
+ - [Export model for inference](deploy/EXPORT_MODEL.md)
+
+ - [Paddle Inference deployment](deploy/README.md)
+
+ - [Inference deployment with Python](deploy/python)
+ - [Inference deployment with C++](deploy/cpp)
+
+ - [Paddle-Lite deployment](deploy/lite)
+
+ - [Paddle Serving deployment](deploy/serving)
+
+ - [ONNX model export](deploy/EXPORT_ONNX_MODEL.md)
+
+ - [Inference benchmark](deploy/BENCHMARK_INFER.md)
+
+- Advanced development
+
+ - [Data processing module](docs/advanced_tutorials/READER.md)
+ - [New object detection models](docs/advanced_tutorials/MODEL_TECHNICAL.md)
+ - Custumization
+ - [Object detection](docs/advanced_tutorials/customization/detection.md)
+
+
+## Version updates
+
+Please refer to the[ Release note ](https://github.com/PaddlePaddle/Paddle/wiki/PaddlePaddle-2.3.0-Release-Note-EN)for more details about the updates
+
+## License
+
+PaddleYOLO is provided under the [GPL-3.0 license](LICENSE)
+
+## Quote
+
+```
+@misc{ppdet2019,
+title={PaddleDetection, Object detection and instance segmentation toolkit based on PaddlePaddle.},
+author={PaddlePaddle Authors},
+howpublished = {\url{https://github.com/PaddlePaddle/PaddleDetection}},
+year={2019}
+}
+```
diff --git a/asserts/Algorithm_principle.png b/asserts/Algorithm_principle.png
new file mode 100644
index 0000000000000000000000000000000000000000..f89b410451c34db2ef69520311beec9f8b47e2ac
Binary files /dev/null and b/asserts/Algorithm_principle.png differ
diff --git a/asserts/Backbone.png b/asserts/Backbone.png
new file mode 100644
index 0000000000000000000000000000000000000000..ebf87bbfa7004f447b8bab8a2f0b41f3b2bc7d9d
Binary files /dev/null and b/asserts/Backbone.png differ
diff --git a/asserts/model.properties b/asserts/model.properties
new file mode 100644
index 0000000000000000000000000000000000000000..7d058ca71c47ff11dd8cb085b62cce3a7670e1db
--- /dev/null
+++ b/asserts/model.properties
@@ -0,0 +1,10 @@
+# 模型唯一标识
+modelCode = 469
+# 模型名称
+modelName=yolov5_paddle
+# 模型描述
+modelDescription=yolov5是一种基于深度学习的目标检测算法,可以广泛应用于各种计算机视觉和人工智能领域的应用中
+# 应用场景
+appScenario=推理,训练,金融,交通,教育
+# 框架类型
+frameType=paddle
\ No newline at end of file
diff --git a/asserts/result_000000014439_640x640.jpg b/asserts/result_000000014439_640x640.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..d1f46e9a256c3b4c714203a15706946c2043f877
Binary files /dev/null and b/asserts/result_000000014439_640x640.jpg differ
diff --git a/configs/datasets/coco_detection.yml b/configs/datasets/coco_detection.yml
new file mode 100644
index 0000000000000000000000000000000000000000..df50dab1bc6f3f72a186bcb7bcecf4b7134c14ef
--- /dev/null
+++ b/configs/datasets/coco_detection.yml
@@ -0,0 +1,20 @@
+metric: COCO
+num_classes: 80
+
+TrainDataset:
+ name: COCODataSet
+ image_dir: train2017
+ anno_path: annotations/instances_train2017.json
+ dataset_dir: dataset/coco
+ data_fields: ['image', 'gt_bbox', 'gt_class', 'is_crowd']
+
+EvalDataset:
+ name: COCODataSet
+ image_dir: val2017
+ anno_path: annotations/instances_val2017.json
+ dataset_dir: dataset/coco
+
+TestDataset:
+ name: ImageFolder
+ anno_path: annotations/instances_val2017.json # also support txt (like VOC's label_list.txt)
+ dataset_dir: dataset/coco # if set, anno_path will be 'dataset_dir/anno_path'
diff --git a/configs/datasets/objects365_detection.yml b/configs/datasets/objects365_detection.yml
new file mode 100644
index 0000000000000000000000000000000000000000..735ebf96dcea828459428016bad764c8461e8ee8
--- /dev/null
+++ b/configs/datasets/objects365_detection.yml
@@ -0,0 +1,21 @@
+metric: COCO
+num_classes: 365
+
+TrainDataset:
+ !COCODataSet
+ image_dir: train
+ anno_path: annotations/zhiyuan_objv2_train.json
+ dataset_dir: dataset/objects365
+ data_fields: ['image', 'gt_bbox', 'gt_class', 'is_crowd']
+
+EvalDataset:
+ !COCODataSet
+ image_dir: val
+ anno_path: annotations/zhiyuan_objv2_val.json
+ dataset_dir: dataset/objects365
+ allow_empty: true
+
+TestDataset:
+ !ImageFolder
+ anno_path: annotations/zhiyuan_objv2_val.json
+ dataset_dir: dataset/objects365/
diff --git a/configs/datasets/roadsign_voc.yml b/configs/datasets/roadsign_voc.yml
new file mode 100644
index 0000000000000000000000000000000000000000..9a081611aa8dafef5d5c6f1af1476cc038db5702
--- /dev/null
+++ b/configs/datasets/roadsign_voc.yml
@@ -0,0 +1,21 @@
+metric: VOC
+map_type: integral
+num_classes: 4
+
+TrainDataset:
+ name: VOCDataSet
+ dataset_dir: dataset/roadsign_voc
+ anno_path: train.txt
+ label_list: label_list.txt
+ data_fields: ['image', 'gt_bbox', 'gt_class', 'difficult']
+
+EvalDataset:
+ name: VOCDataSet
+ dataset_dir: dataset/roadsign_voc
+ anno_path: valid.txt
+ label_list: label_list.txt
+ data_fields: ['image', 'gt_bbox', 'gt_class', 'difficult']
+
+TestDataset:
+ name: ImageFolder
+ anno_path: dataset/roadsign_voc/label_list.txt
diff --git a/configs/datasets/visdrone_detection.yml b/configs/datasets/visdrone_detection.yml
new file mode 100644
index 0000000000000000000000000000000000000000..37feb6e2618ff9d83ce2842a9e581dcfd31efc78
--- /dev/null
+++ b/configs/datasets/visdrone_detection.yml
@@ -0,0 +1,22 @@
+metric: COCO
+num_classes: 10
+
+TrainDataset:
+ !COCODataSet
+ image_dir: VisDrone2019-DET-train
+ anno_path: train.json
+ dataset_dir: dataset/visdrone
+ data_fields: ['image', 'gt_bbox', 'gt_class', 'is_crowd']
+
+EvalDataset:
+ !COCODataSet
+ image_dir: VisDrone2019-DET-val
+ anno_path: val.json
+ # image_dir: test_dev
+ # anno_path: test_dev.json
+ dataset_dir: dataset/visdrone
+
+TestDataset:
+ !ImageFolder
+ anno_path: val.json
+ dataset_dir: dataset/visdrone
diff --git a/configs/datasets/voc.yml b/configs/datasets/voc.yml
new file mode 100644
index 0000000000000000000000000000000000000000..72182bed9d17ca076c94a1872613ce7ad29d36d9
--- /dev/null
+++ b/configs/datasets/voc.yml
@@ -0,0 +1,21 @@
+metric: VOC
+map_type: 11point
+num_classes: 20
+
+TrainDataset:
+ name: VOCDataSet
+ dataset_dir: dataset/voc
+ anno_path: trainval.txt
+ label_list: label_list.txt
+ data_fields: ['image', 'gt_bbox', 'gt_class', 'difficult']
+
+EvalDataset:
+ name: VOCDataSet
+ dataset_dir: dataset/voc
+ anno_path: test.txt
+ label_list: label_list.txt
+ data_fields: ['image', 'gt_bbox', 'gt_class', 'difficult']
+
+TestDataset:
+ name: ImageFolder
+ anno_path: dataset/voc/label_list.txt
diff --git a/configs/runtime.yml b/configs/runtime.yml
new file mode 100644
index 0000000000000000000000000000000000000000..a58b171ce774e045f4db2e0894a6781a25e0ec03
--- /dev/null
+++ b/configs/runtime.yml
@@ -0,0 +1,16 @@
+use_gpu: true
+use_xpu: false
+use_mlu: false
+use_npu: false
+log_iter: 20
+save_dir: output
+snapshot_epoch: 1
+print_flops: false
+print_params: false
+
+# Exporting the model
+export:
+ post_process: True # Whether post-processing is included in the network when export model.
+ nms: True # Whether NMS is included in the network when export model.
+ benchmark: False # It is used to testing model performance, if set `True`, post-process and NMS will not be exported.
+ fuse_conv_bn: False
diff --git a/configs/yolov5/README.md b/configs/yolov5/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..7a002d8ac76df8a29cc6ad722445a4df033c1f95
--- /dev/null
+++ b/configs/yolov5/README.md
@@ -0,0 +1,245 @@
+# YOLOv5
+
+## 内容
+- [模型库](#模型库)
+- [使用说明](#使用说明)
+- [速度测试](#速度测试)
+
+## 模型库
+
+### 基础模型
+
+| 网络网络 | 输入尺寸 | 图片数/GPU | 学习率策略 | 模型推理耗时(ms) | mAPval0.5:0.95 | mAPval
0.5 | Params(M) | FLOPs(G) | 下载链接 | 配置文件 | +| :------------- | :------- | :-------: | :------: | :------------: | :---------------------: | :----------------: |:---------: | :------: |:---------------: |:-----: | +| YOLOv5-n | 640 | 16 | 300e | 1.5 | 28.0 | 45.7 | 1.87 | 4.52 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5_n_300e_coco.pdparams) | [配置文件](./yolov5_n_300e_coco.yml) | +| YOLOv5-s | 640 | 16 | 300e | 2.6 | 37.6 | 56.7 | 7.24 | 16.54 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5_s_300e_coco.pdparams) | [配置文件](./yolov5_s_300e_coco.yml) | +| YOLOv5-m | 640 | 16 | 300e | 5.2 | 45.4 | 64.1 | 21.19 | 49.08 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5_m_300e_coco.pdparams) | [配置文件](./yolov5_m_300e_coco.yml) | +| YOLOv5-l | 640 | 16 | 300e | 7.9 | 48.9 | 67.1 | 46.56 | 109.32 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5_l_300e_coco.pdparams) | [配置文件](./yolov5_l_300e_coco.yml) | +| YOLOv5-x | 640 | 16 | 300e | 13.7 | 50.6 | 68.7 | 86.75 | 205.92 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5_x_300e_coco.pdparams) | [配置文件](./yolov5_x_300e_coco.yml) | +| YOLOv5-s ConvNeXt| 640 | 8 | 36e | - | 42.4 | 65.3 | 34.54 | 17.96 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5_convnext_s_36e_coco.pdparams) | [配置文件](../convnext/yolov5_convnext_s_36e_coco.yml) | + +### SSOD预训练模型 + +| 网络网络 | 输入尺寸 | 图片数/GPU | 学习率策略 | 模型推理耗时(ms) | mAPval
0.5:0.95 | mAPval
0.5 | Params(M) | FLOPs(G) | 下载链接 | 配置文件 | +| :------------- | :------- | :-------: | :------: | :------------: | :---------------------: | :----------------: |:---------: | :------: |:---------------: |:-----: | +| YOLOv5-s | 640 | 16 | 80e | 2.6 | 38.8 | 58.2 | 7.24 | 16.54 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5_s_80e_ssod_finetune_coco.pdparams) | [配置文件](./yolov5_s_80e_ssod_finetune_coco.yml) | + +### Objects-365预训练模型 + +| 网络网络 | 输入尺寸 | 图片数/GPU | 学习率策略 | 模型推理耗时(ms) | mAPval
0.5:0.95 | mAPval
0.5 | Params(M) | FLOPs(G) | 下载链接 | 配置文件 | +| :------------- | :------- | :-------: | :------: | :------------: | :---------------------: | :----------------: |:---------: | :------: |:---------------: |:-----: | +| YOLOv5-l | 640 | 16 | 30e | 7.9 | 49.3 | 67.8 | 46.56 | 109.32 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5_l_30e_obj365_finetune_coco.pdparams) | [配置文件](./yolov5_l_30e_obj365_finetune_coco.yml) | + +### P6大尺度模型 + +| 网络网络 | 输入尺寸 | 图片数/GPU | 学习率策略 | 模型推理耗时(ms) | mAPval
0.5:0.95 | mAPval
0.5 | Params(M) | FLOPs(G) | 下载链接 | 配置文件 | +| :------------- | :------- | :-------: | :------: | :------------: | :---------------------: | :----------------: |:---------: | :------: |:---------------: |:-----: | +| YOLOv5p6-n | 1280 | 16 | 300e | - | 35.9 | 54.2 | 3.25 | 9.23 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5p6_n_300e_coco.pdparams) | [配置文件](./yolov5p6_n_300e_coco.yml) | +| YOLOv5p6-s | 1280 | 16 | 300e | - | 44.5 | 63.3 | 12.63 | 33.81 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5p6_s_300e_coco.pdparams) | [配置文件](./yolov5p6_s_300e_coco.yml) | +| YOLOv5p6-m | 1280 | 16 | 300e | - | 51.1 | 69.0 | 35.73 | 100.21 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5p6_m_300e_coco.pdparams) | [配置文件](./yolov5p6_m_300e_coco.yml) | +| YOLOv5p6-l | 1280 | 8 | 300e | - | 53.4 | 71.0 | 76.77 | 223.09 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5p6_l_300e_coco.pdparams) | [配置文件](./yolov5p6_l_300e_coco.yml) | +| YOLOv5p6-x | 1280 | 8 | 300e | - | 54.7 | 72.4 | 140.80 | 420.03 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5p6_x_300e_coco.pdparams) | [配置文件](./yolov5p6_x_300e_coco.yml) | + +### [YOLOv5u](../yolov5u) + +| 网络网络 | 输入尺寸 | 图片数/GPU | 学习率策略 | 模型推理耗时(ms) | mAPval
0.5:0.95 | mAPval
0.5 | Params(M) | FLOPs(G) | 下载链接 | 配置文件 | +| :------------- | :------- | :-------: | :------: | :------------: | :---------------------: | :----------------: |:---------: | :------: |:---------------: |:-----: | +| YOLOv5u-n | 640 | 16 | 300e | 1.61 | 34.5 | 49.7 | 2.65 | 7.79 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5u_n_300e_coco.pdparams) | [配置文件](./yolov5u/yolov5u_n_300e_coco.yml) | +| YOLOv5u-s | 640 | 16 | 300e | 2.66 | 43.0 | 59.7 | 9.15 | 24.12 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5u_s_300e_coco.pdparams) | [配置文件](./yolov5u/yolov5u_s_300e_coco.yml) | +| YOLOv5u-m | 640 | 16 | 300e | 5.50 | 49.0 | 65.7 | 25.11 | 64.42 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5u_m_300e_coco.pdparams) | [配置文件](./yolov5u/yolov5u_m_300e_coco.yml) | +| YOLOv5u-l | 640 | 16 | 300e | 8.73 | 52.2 | 69.0 | 53.23 | 135.34 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5u_l_300e_coco.pdparams) | [配置文件](./yolov5u/yolov5u_l_300e_coco.yml) | +| YOLOv5u-x | 640 | 16 | 300e | 15.49 | 53.1 | 69.9 | 97.28 | 246.89 | [下载链接](https://paddledet.bj.bcebos.com/models/yolov5u_x_300e_coco.pdparams) | [配置文件](./yolov5u/yolov5u_x_300e_coco.yml) | + + +**注意:** + - YOLOv5模型训练使用COCO train2017作为训练集，Box AP为在COCO val2017上的`mAP(IoU=0.5:0.95)`结果； + - YOLOv5u 模型表示YOLOv5结构使用YOLOv8的head和loss，是Anchor Free的检测方案，具体可参照[YOLOv5u](../yolov5u)； + - YOLOv5模型训练过程中默认使用8 GPUs进行混合精度训练，默认lr为0.01为8卡总batch_size的设置，如果**GPU卡数**或者每卡**batch size**发生改动，也不需要改动学习率，但为了保证高精度最好使用**总batch size大于64**的配置去训练； + - 模型推理耗时(ms)为TensorRT-FP16下测试的耗时，不包含数据预处理和模型输出后处理(NMS)的耗时。测试采用单卡Tesla T4 GPU，batch size=1，测试环境为**paddlepaddle-2.3.2**, **CUDA 11.2**, **CUDNN 8.2**, **GCC-8.2**, **TensorRT 8.0.3.4**，具体请参考[速度测试](#速度测试)。 + - 如果你设置了`--run_benchmark=True`, 你首先需要安装以下依赖`pip install pynvml psutil GPUtil`。 + +### 部署模型 + +| 网络模型 | 输入尺寸 | 导出后的权重(w/o NMS) | ONNX(w/o NMS) | +| :-------- | :--------: | :---------------------: | :----------------: | +| YOLOv5-n | 640 | [( w/ nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_n_300e_coco_w_nms.zip) | [( w/o nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_n_300e_coco_wo_nms.zip) | [( w/ nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_n_300e_coco_w_nms.onnx) | [( w/o nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_n_300e_coco_wo_nms.onnx) | +| YOLOv5-s | 640 | [( w/ nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_s_300e_coco_w_nms.zip) | [( w/o nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_s_300e_coco_wo_nms.zip) | [( w/ nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_s_300e_coco_w_nms.onnx) | [( w/o nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_s_300e_coco_wo_nms.onnx) | +| YOLOv5-m | 640 | [( w/ nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_m_300e_coco_w_nms.zip) | [( w/o nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_m_300e_coco_wo_nms.zip) | [( w/ nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_m_300e_coco_w_nms.onnx) | [( w/o nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_m_300e_coco_wo_nms.onnx) | +| YOLOv5-l | 640 | [( w/ nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_l_300e_coco_w_nms.zip) | [( w/o nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_l_300e_coco_wo_nms.zip) | [( w/ nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_l_300e_coco_w_nms.onnx) | [( w/o nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_l_300e_coco_wo_nms.onnx) | +| YOLOv5-x | 640 | [( w/ nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_x_300e_coco_w_nms.zip) | [( w/o nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_x_300e_coco_wo_nms.zip) | [( w/ nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_x_300e_coco_w_nms.onnx) | [( w/o nms)](https://paddledet.bj.bcebos.com/deploy/yoloseries/yolov5/yolov5_x_300e_coco_wo_nms.onnx) | + +## 使用教程 + +### 0. **一键运行全流程** + +将以下命令写在一个脚本文件里如```run.sh```，一键运行命令为：```sh run.sh```，也可命令行一句句去运行。 + +```bash +model_name=yolov5 # 可修改，如 ppyoloe +job_name=yolov5_s_300e_coco # 可修改，如 ppyoloe_plus_crn_s_80e_coco + +config=configs/${model_name}/${job_name}.yml +log_dir=log_dir/${job_name} +# weights=https://bj.bcebos.com/v1/paddledet/models/${job_name}.pdparams +weights=output/${job_name}/model_final.pdparams + +# 1.训练（单卡/多卡），加 --eval 表示边训边评估，加 --amp 表示混合精度训练 +# CUDA_VISIBLE_DEVICES=0 python tools/train.py -c ${config} --eval --amp +python -m paddle.distributed.launch --log_dir=${log_dir} --gpus 0,1,2,3,4,5,6,7 tools/train.py -c ${config} --eval --amp + +# 2.评估，加 --classwise 表示输出每一类mAP +CUDA_VISIBLE_DEVICES=0 python tools/eval.py -c ${config} -o weights=${weights} --classwise + +# 3.预测 (单张图/图片文件夹） +CUDA_VISIBLE_DEVICES=0 python tools/infer.py -c ${config} -o weights=${weights} --infer_img=demo/000000014439_640x640.jpg --draw_threshold=0.5 +# CUDA_VISIBLE_DEVICES=0 python tools/infer.py -c ${config} -o weights=${weights} --infer_dir=demo/ --draw_threshold=0.5 + +# 4.导出模型，以下3种模式选一种 +## 普通导出，加trt表示用于trt加速，对NMS和silu激活函数提速明显 +CUDA_VISIBLE_DEVICES=0 python tools/export_model.py -c ${config} -o weights=${weights} # trt=True + +## exclude_post_process去除后处理导出，返回和YOLOv5导出ONNX时相同格式的concat后的1个Tensor，是未缩放回原图的坐标+分类置信度 +# CUDA_VISIBLE_DEVICES=0 python tools/export_model.py -c ${config} -o weights=${weights} exclude_post_process=True # trt=True + +## exclude_nms去除NMS导出，返回2个Tensor，是缩放回原图后的坐标和分类置信度 +# CUDA_VISIBLE_DEVICES=0 python tools/export_model.py -c ${config} -o weights=${weights} exclude_nms=True # trt=True + +# 5.部署预测，注意不能使用 去除后处理 或 去除NMS 导出后的模型去预测 +CUDA_VISIBLE_DEVICES=0 python deploy/python/infer.py --model_dir=output_inference/${job_name} --image_file=demo/000000014439_640x640.jpg --device=GPU + +# 6.部署测速，加 “--run_mode=trt_fp16” 表示在TensorRT FP16模式下测速，注意如需用到 trt_fp16 则必须为加 trt=True 导出的模型 +CUDA_VISIBLE_DEVICES=0 python deploy/python/infer.py --model_dir=output_inference/${job_name} --image_file=demo/000000014439_640x640.jpg --device=GPU --run_benchmark=True # --run_mode=trt_fp16 + +# 7.onnx导出，一般结合 exclude_post_process去除后处理导出的模型 +paddle2onnx --model_dir output_inference/${job_name} --model_filename model.pdmodel --params_filename model.pdiparams --opset_version 12 --save_file ${job_name}.onnx + +# 8.onnx trt测速 +/usr/local/TensorRT-8.0.3.4/bin/trtexec --onnx=${job_name}.onnx --workspace=4096 --avgRuns=10 --shapes=input:1x3x640x640 --fp16 +/usr/local/TensorRT-8.0.3.4/bin/trtexec --onnx=${job_name}.onnx --workspace=4096 --avgRuns=10 --shapes=input:1x3x640x640 --fp32 +``` + +### 1. 训练 +执行以下指令使用混合精度训练YOLOv5 +```bash +python -m paddle.distributed.launch --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/yolov5/yolov5_s_300e_coco.yml --amp --eval +``` +**注意:** +- `--amp`表示开启混合精度训练以避免显存溢出，`--eval`表示边训边验证。 + +### 2. 评估 +执行以下命令在单个GPU上评估COCO val2017数据集 +```bash +CUDA_VISIBLE_DEVICES=0 python tools/eval.py -c configs/yolov5/yolov5_s_300e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/yolov5_s_300e_coco.pdparams +``` + +### 3. 推理 +使用以下命令在单张GPU上预测图片，使用`--infer_img`推理单张图片以及使用`--infer_dir`推理文件中的所有图片。 +```bash +# 推理单张图片 +CUDA_VISIBLE_DEVICES=0 python tools/infer.py -c configs/yolov5/yolov5_s_300e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/yolov5_s_300e_coco.pdparams --infer_img=demo/000000014439_640x640.jpg + +# 推理文件中的所有图片 +CUDA_VISIBLE_DEVICES=0 python tools/infer.py -c configs/yolov5/yolov5_s_300e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/yolov5_s_300e_coco.pdparams --infer_dir=demo +``` + +### 4.导出模型 +YOLOv5在GPU上推理部署或benchmark测速等需要通过`tools/export_model.py`导出模型。 + +当你**使用Paddle Inference但不使用TensorRT**时，运行以下的命令导出模型 + +```bash +python tools/export_model.py -c configs/yolov5/yolov5_s_300e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/yolov5_s_300e_coco.pdparams +``` + +当你**使用Paddle Inference且使用TensorRT**时，需要指定`-o trt=True`来导出模型。 + +```bash +python tools/export_model.py -c configs/yolov5/yolov5_s_300e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/yolov5_s_300e_coco.pdparams trt=True +``` + +如果你想将YOLOv5模型导出为**ONNX格式**，参考 +[PaddleDetection模型导出为ONNX格式教程](../../deploy/EXPORT_ONNX_MODEL.md)，运行以下命令： + +```bash + +# 导出推理模型 +python tools/export_model.py -c configs/yolov5/yolov5_s_300e_coco.yml --output_dir=output_inference -o weights=https://paddledet.bj.bcebos.com/models/yolov5_s_300e_coco.pdparams + +# 安装paddle2onnx +pip install paddle2onnx + +# 转换成onnx格式 +paddle2onnx --model_dir output_inference/yolov5_s_300e_coco --model_filename model.pdmodel --params_filename model.pdiparams --opset_version 11 --save_file yolov5_s_300e_coco.onnx +``` + +**注意：** ONNX模型目前只支持batch_size=1 + + +### 5.推理部署 +YOLOv5可以使用以下方式进行部署： + - Paddle Inference [Python](../../deploy/python) & [C++](../../deploy/cpp) + - [Paddle-TensorRT](../../deploy/TENSOR_RT.md) + - [PaddleServing](https://github.com/PaddlePaddle/Serving) + - [PaddleSlim模型量化](../slim) + +运行以下命令导出模型 + +```bash +python tools/export_model.py -c configs/yolov5/yolov5_s_300e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/yolov5_s_300e_coco.pdparams trt=True +``` + +**注意：** +- trt=True表示**使用Paddle Inference且使用TensorRT**进行测速，速度会更快，默认不加即为False，表示**使用Paddle Inference但不使用TensorRT**进行测速。 +- 如果是使用Paddle Inference在TensorRT FP16模式下部署，需要参考[Paddle Inference文档](https://www.paddlepaddle.org.cn/inference/master/user_guides/download_lib.html#python)，下载并安装与你的CUDA, CUDNN和TensorRT相应的wheel包。 + +#### 5.1.Python部署 +`deploy/python/infer.py`使用上述导出后的Paddle Inference模型用于推理和benchnark测速，如果设置了`--run_benchmark=True`, 首先需要安装以下依赖`pip install pynvml psutil GPUtil`。 + +```bash +# Python部署推理单张图片 +python deploy/python/infer.py --model_dir=output_inference/yolov5_s_300e_coco --image_file=demo/000000014439_640x640.jpg --device=gpu + +# 推理文件夹下的所有图片 +python deploy/python/infer.py --model_dir=output_inference/yolov5_s_300e_coco --image_dir=demo/ --device=gpu +``` + +#### 5.2. C++部署 +`deploy/cpp/build/main`使用上述导出后的Paddle Inference模型用于C++推理部署, 首先按照[docs](../../deploy/cpp/docs)编译安装环境。 +```bash +# C++部署推理单张图片 +./deploy/cpp/build/main --model_dir=output_inference/yolov5_s_300e_coco/ --image_file=demo/000000014439_640x640.jpg --run_mode=paddle --device=GPU --threshold=0.5 --output_dir=cpp_infer_output/yolov5_s_300e_coco +``` + + +## 速度测试 + +为了公平起见，在[模型库](#模型库)中的速度测试结果均为不包含数据预处理和模型输出后处理(NMS)的数据(与[YOLOv4(AlexyAB)](https://github.com/AlexeyAB/darknet)测试方法一致)，需要在导出模型时指定`-o exclude_nms=True`。测速需设置`--run_benchmark=True`, 首先需要安装以下依赖`pip install pynvml psutil GPUtil`。 + +**使用Paddle Inference但不使用TensorRT**进行测速，执行以下命令： + +```bash +# 导出模型 +python tools/export_model.py -c configs/yolov5/yolov5_s_300e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/yolov5_s_300e_coco.pdparams exclude_nms=True + +# 速度测试，使用run_benchmark=True +python deploy/python/infer.py --model_dir=output_inference/yolov5_s_300e_coco --image_file=demo/000000014439_640x640.jpg --run_mode=paddle --device=gpu --run_benchmark=True +``` + +**使用Paddle Inference且使用TensorRT**进行测速，执行以下命令： + +```bash +# 导出模型，使用trt=True +python tools/export_model.py -c configs/yolov5/yolov5_s_300e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/yolov5_s_300e_coco.pdparams exclude_nms=True trt=True + +# 速度测试，使用run_benchmark=True +python deploy/python/infer.py --model_dir=output_inference/yolov5_s_300e_coco --image_file=demo/000000014439_640x640.jpg --device=gpu --run_benchmark=True + +# tensorRT-FP32测速 +python deploy/python/infer.py --model_dir=output_inference/yolov5_s_300e_coco --image_file=demo/000000014439_640x640.jpg --device=gpu --run_benchmark=True --run_mode=trt_fp32 + +# tensorRT-FP16测速 +python deploy/python/infer.py --model_dir=output_inference/yolov5_s_300e_coco --image_file=demo/000000014439_640x640.jpg --device=gpu --run_benchmark=True --run_mode=trt_fp16 +``` +**注意:** +- 导出模型时指定`-o exclude_nms=True`仅作为测速时用，这样导出的模型其推理部署预测的结果不是最终检出框的结果。 +- [模型库](#模型库)中的速度测试结果为tensorRT-FP16测速后的最快速度，为不包含数据预处理和模型输出后处理(NMS)的耗时。 diff --git a/configs/yolov5/_base_/optimizer_300e.yml b/configs/yolov5/_base_/optimizer_300e.yml new file mode 100644 index 0000000000000000000000000000000000000000..d5ba562e9a9ab943e9a469a2c7f883c40bfc2d3e --- /dev/null +++ b/configs/yolov5/_base_/optimizer_300e.yml @@ -0,0 +1,19 @@ +epoch: 300 + +LearningRate: + base_lr: 0.01 + schedulers: + - !YOLOv5LRDecay + max_epochs: 300 + min_lr_ratio: 0.01 + - !ExpWarmup + epochs: 3 + +OptimizerBuilder: + optimizer: + type: Momentum + momentum: 0.937 + use_nesterov: True + regularizer: + factor: 0.0005 + type: L2 diff --git a/configs/yolov5/_base_/optimizer_300e_high.yml b/configs/yolov5/_base_/optimizer_300e_high.yml new file mode 100644 index 0000000000000000000000000000000000000000..2d5b7c0b81e5469db3d70d69655dc3031b22f834 --- /dev/null +++ b/configs/yolov5/_base_/optimizer_300e_high.yml @@ -0,0 +1,19 @@ +epoch: 300 + +LearningRate: + base_lr: 0.01 + schedulers: + - !YOLOv5LRDecay + max_epochs: 300 + min_lr_ratio: 0.1 # + - !ExpWarmup + epochs: 3 + +OptimizerBuilder: + optimizer: + type: Momentum + momentum: 0.937 + use_nesterov: True + regularizer: + factor: 0.0005 + type: L2 diff --git a/configs/yolov5/_base_/yolov5_cspdarknet.yml b/configs/yolov5/_base_/yolov5_cspdarknet.yml new file mode 100644 index 0000000000000000000000000000000000000000..5fa1e496ed1e251c53aa3e3dccb96b899139187a --- /dev/null +++ b/configs/yolov5/_base_/yolov5_cspdarknet.yml @@ -0,0 +1,46 @@ +architecture: YOLOv5 +norm_type: sync_bn +use_ema: True +ema_decay: 0.9999 +ema_decay_type: "exponential" +act: silu +find_unused_parameters: True + +depth_mult: 1.0 +width_mult: 1.0 + + +YOLOv5: + backbone: CSPDarkNet + neck: YOLOCSPPAN + yolo_head: YOLOv5Head + post_process: ~ + +CSPDarkNet: + arch: "P5" + return_idx: [2, 3, 4] + depthwise: false + +YOLOCSPPAN: + depthwise: false + +YOLOv5Head: + anchors: [[10, 13], [16, 30], [33, 23], + [30, 61], [62, 45], [59, 119], + [116, 90], [156, 198], [373, 326]] + anchor_masks: [[0, 1, 2], [3, 4, 5], [6, 7, 8]] + stride: [8, 16, 32] + loss: YOLOv5Loss + nms: + name: MultiClassNMS + nms_top_k: 3000 + keep_top_k: 300 + score_threshold: 0.001 + nms_threshold: 0.65 + +YOLOv5Loss: + downsample_ratios: [8, 16, 32] + balance: [4.0, 1.0, 0.4] + box_weight: 0.05 + obj_weight: 1.0 + cls_weght: 0.5 diff --git a/configs/yolov5/_base_/yolov5_cspresnet.yml b/configs/yolov5/_base_/yolov5_cspresnet.yml new file mode 100644 index 0000000000000000000000000000000000000000..ae1394f9456cb0d992f16d9061038edf2e9ad8ec --- /dev/null +++ b/configs/yolov5/_base_/yolov5_cspresnet.yml @@ -0,0 +1,22 @@ +_BASE_: [ + 'yolov5_cspdarknet.yml', +] + +YOLOv5: + backbone: CSPResNet + neck: CustomCSPPAN + yolo_head: YOLOv5Head + post_process: ~ + +CSPResNet: + layers: [3, 6, 6, 3] + channels: [64, 128, 256, 512, 1024] + return_idx: [1, 2, 3] + use_large_stem: True + +CustomCSPPAN: + out_channels: [768, 384, 192] + stage_num: 1 + block_num: 3 + act: 'swish' + spp: true diff --git a/configs/yolov5/_base_/yolov5_reader.yml b/configs/yolov5/_base_/yolov5_reader.yml new file mode 100644 index 0000000000000000000000000000000000000000..c99e4d83888b57587a11d55ceb067f9cfa621bcd --- /dev/null +++ b/configs/yolov5/_base_/yolov5_reader.yml @@ -0,0 +1,46 @@ +input_height: &input_height 640 +input_width: &input_width 640 +input_size: &input_size [*input_height, *input_width] +mosaic_epoch: &mosaic_epoch 300 + +worker_num: 4 +TrainReader: + sample_transforms: + - DecodeNormResize: {target_size: *input_size, mosaic: True} + - MosaicPerspective: {mosaic_prob: 1.0, target_size: *input_size} + - RandomHSV: {hgain: 0.015, sgain: 0.7, vgain: 0.4} + - RandomFlip: {} + - BboxXYXY2XYWH: {} + - NormalizeBox: {} + - NormalizeImage: {mean: [0., 0., 0.], std: [1., 1., 1.], norm_type: none} + - RGBReverse: {} # bgr->rgb + - Permute: {} + batch_size: 8 + shuffle: True + drop_last: False + use_shared_memory: False + collate_batch: False + mosaic_epoch: *mosaic_epoch + + +EvalReader: + sample_transforms: + - Decode: {} + - Resize: {target_size: *input_size, keep_ratio: True, interp: 1} + - Pad: {size: *input_size, fill_value: [114., 114., 114.]} + - NormalizeImage: {mean: [0., 0., 0.], std: [1., 1., 1.], norm_type: none} + - Permute: {} + batch_size: 1 + + +TestReader: + inputs_def: + image_shape: [3, 640, 640] + sample_transforms: + - Decode: {} + - Resize: {target_size: *input_size, keep_ratio: True, interp: 1} + - Pad: {size: *input_size, fill_value: [114., 114., 114.]} + - NormalizeImage: {mean: [0., 0., 0.], std: [1., 1., 1.], norm_type: none} + - Permute: {} + batch_size: 1 + fuse_normalize: False diff --git a/configs/yolov5/_base_/yolov5_reader_high_aug.yml b/configs/yolov5/_base_/yolov5_reader_high_aug.yml new file mode 100644 index 0000000000000000000000000000000000000000..f44882dddc970f632b3c30b65da8a547ac3d6186 --- /dev/null +++ b/configs/yolov5/_base_/yolov5_reader_high_aug.yml @@ -0,0 +1,46 @@ +input_height: &input_height 640 +input_width: &input_width 640 +input_size: &input_size [*input_height, *input_width] +mosaic_epoch: &mosaic_epoch 300 + +worker_num: 4 +TrainReader: + sample_transforms: + - DecodeNormResize: {target_size: *input_size, mosaic: True} + - MosaicPerspective: {mosaic_prob: 1.0, target_size: *input_size, scale: 0.9, mixup_prob: 0.1, copy_paste_prob: 0.1} + - RandomHSV: {hgain: 0.015, sgain: 0.7, vgain: 0.4} + - RandomFlip: {} + - BboxXYXY2XYWH: {} + - NormalizeBox: {} + - NormalizeImage: {mean: [0., 0., 0.], std: [1., 1., 1.], norm_type: none} + - RGBReverse: {} # bgr->rgb + - Permute: {} + batch_size: 8 + shuffle: True + drop_last: False + use_shared_memory: False + collate_batch: False + mosaic_epoch: *mosaic_epoch + + +EvalReader: + sample_transforms: + - Decode: {} + - Resize: {target_size: *input_size, keep_ratio: True, interp: 1} + - Pad: {size: *input_size, fill_value: [114., 114., 114.]} + - NormalizeImage: {mean: [0., 0., 0.], std: [1., 1., 1.], norm_type: none} + - Permute: {} + batch_size: 1 + + +TestReader: + inputs_def: + image_shape: [3, 640, 640] + sample_transforms: + - Decode: {} + - Resize: {target_size: *input_size, keep_ratio: True, interp: 1} + - Pad: {size: *input_size, fill_value: [114., 114., 114.]} + - NormalizeImage: {mean: [0., 0., 0.], std: [1., 1., 1.], norm_type: none} + - Permute: {} + batch_size: 1 + fuse_normalize: False diff --git a/configs/yolov5/_base_/yolov5p2_cspdarknet.yml b/configs/yolov5/_base_/yolov5p2_cspdarknet.yml new file mode 100644 index 0000000000000000000000000000000000000000..99103dc4ff8e1111359392b8ae410fa09ed34e96 --- /dev/null +++ b/configs/yolov5/_base_/yolov5p2_cspdarknet.yml @@ -0,0 +1,19 @@ +_BASE_: [ + 'yolov5_cspdarknet.yml', +] + +CSPDarkNet: + arch: "P5" + return_idx: [1, 2, 3, 4] + +YOLOv5Head: + anchors: [[5, 6], [8, 14], [15, 11], + [10, 13], [16, 30], [33, 23], + [30, 61], [62, 45], [59, 119], + [116, 90], [156, 198], [373, 326]] + anchor_masks: [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10, 11]] + stride: [4, 8, 16, 32] + +YOLOv5Loss: + downsample_ratios: [4, 8, 16, 32] + balance: [4.0, 1.0, 0.25, 0.06] diff --git a/configs/yolov5/_base_/yolov5p6_cspdarknet.yml b/configs/yolov5/_base_/yolov5p6_cspdarknet.yml new file mode 100644 index 0000000000000000000000000000000000000000..e1043e0bb7faffeda10292c249ddf930de2f9e13 --- /dev/null +++ b/configs/yolov5/_base_/yolov5p6_cspdarknet.yml @@ -0,0 +1,22 @@ +_BASE_: [ + 'yolov5_cspdarknet.yml', +] + +CSPDarkNet: + arch: "P6" + return_idx: [2, 3, 4, 5] + +YOLOv5Head: + anchors: [[19, 27], [44, 40], [38, 94], + [96, 68], [86, 152], [180, 137], + [140, 301], [303, 264], [238, 542], + [436, 615], [739, 380], [925, 792]] + anchor_masks: [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10, 11]] + stride: [8, 16, 32, 64] + +YOLOv5Loss: + downsample_ratios: [8, 16, 32, 64] + balance: [4.0, 1.0, 0.25, 0.06] + box_weight: 0.05 + obj_weight: 0.7 + cls_weght: 0.3 diff --git a/configs/yolov5/_base_/yolov5p6_reader.yml b/configs/yolov5/_base_/yolov5p6_reader.yml new file mode 100644 index 0000000000000000000000000000000000000000..cacd30d268bbb5fe5fd5035c76f17a26e36aa3c2 --- /dev/null +++ b/configs/yolov5/_base_/yolov5p6_reader.yml @@ -0,0 +1,45 @@ +input_height: &input_height 1280 +input_width: &input_width 1280 +input_size: &input_size [*input_height, *input_width] +mosaic_epoch: &mosaic_epoch 300 + +worker_num: 4 +TrainReader: + sample_transforms: + - DecodeNormResize: {target_size: *input_size, mosaic: True} + - MosaicPerspective: {mosaic_prob: 1.0, target_size: *input_size, scale: 0.9, mixup_prob: 0.1, copy_paste_prob: 0.1} + - RandomHSV: {hgain: 0.015, sgain: 0.7, vgain: 0.4} + - RandomFlip: {} + - BboxXYXY2XYWH: {} + - NormalizeBox: {} + - NormalizeImage: {mean: [0., 0., 0.], std: [1., 1., 1.], norm_type: none} + - Permute: {} + batch_size: 8 + shuffle: True + drop_last: False + use_shared_memory: False + collate_batch: False + mosaic_epoch: *mosaic_epoch + + +EvalReader: + sample_transforms: + - Decode: {} + - Resize: {target_size: *input_size, keep_ratio: True, interp: 1} + - Pad: {size: *input_size, fill_value: [114., 114., 114.]} + - NormalizeImage: {mean: [0., 0., 0.], std: [1., 1., 1.], norm_type: none} + - Permute: {} + batch_size: 2 + + +TestReader: + inputs_def: + image_shape: [3, 1280, 1280] + sample_transforms: + - Decode: {} + - Resize: {target_size: *input_size, keep_ratio: True, interp: 1} + - Pad: {size: *input_size, fill_value: [114., 114., 114.]} + - NormalizeImage: {mean: [0., 0., 0.], std: [1., 1., 1.], norm_type: none} + - Permute: {} + batch_size: 1 + fuse_normalize: False diff --git a/configs/yolov5/yolov5_l_300e_coco.yml b/configs/yolov5/yolov5_l_300e_coco.yml new file mode 100644 index 0000000000000000000000000000000000000000..2a4179b47d823cdb352a2f10d1ada8ed3e17bd1e --- /dev/null +++ b/configs/yolov5/yolov5_l_300e_coco.yml @@ -0,0 +1,22 @@ +_BASE_: [ + '../datasets/coco_detection.yml', + '../runtime.yml', + '_base_/optimizer_300e_high.yml', + '_base_/yolov5_cspdarknet.yml', + '_base_/yolov5_reader_high_aug.yml', +] +depth_mult: 1.0 +width_mult: 1.0 + +log_iter: 100 +snapshot_epoch: 10 +weights: output/yolov5_l_300e_coco/model_final + + +TrainReader: + batch_size: 16 # default 8 gpus, total bs = 128 + + +YOLOv5Loss: + obj_weight: 0.7 + cls_weght: 0.3 diff --git a/configs/yolov5/yolov5_l_30e_obj365_finetune_coco.yml b/configs/yolov5/yolov5_l_30e_obj365_finetune_coco.yml new file mode 100644 index 0000000000000000000000000000000000000000..b802e6f011debac07edc9301d824655c95115a66 --- /dev/null +++ b/configs/yolov5/yolov5_l_30e_obj365_finetune_coco.yml @@ -0,0 +1,43 @@ +_BASE_: [ + '../datasets/coco_detection.yml', + '../runtime.yml', + '_base_/optimizer_300e_high.yml', + '_base_/yolov5_cspdarknet.yml', + '_base_/yolov5_reader_high_aug.yml', +] +depth_mult: 1.0 +width_mult: 1.0 + +log_iter: 100 +snapshot_epoch: 5 +weights: output/yolov5_l_30e_obj365_finetune_coco/model_final +pretrain_weights: https://paddledet.bj.bcebos.com/models/pretrained/yolov5_l_300e_obj365.pdparams + + +TrainReader: + batch_size: 16 # default 8 gpus, total bs = 128 + + +YOLOv5Loss: + obj_weight: 0.7 + cls_weght: 0.3 + + +epoch: 30 +LearningRate: + base_lr: 0.001 + schedulers: + - !YOLOv5LRDecay + max_epochs: 30 + min_lr_ratio: 0.1 + - !ExpWarmup + epochs: 3 + +OptimizerBuilder: + optimizer: + type: Momentum + momentum: 0.937 + use_nesterov: True + regularizer: + factor: 0.0005 + type: L2 diff --git a/configs/yolov5/yolov5_m_300e_coco.yml b/configs/yolov5/yolov5_m_300e_coco.yml new file mode 100644 index 0000000000000000000000000000000000000000..0b82c764f1f1a693747249abbb01255d478f6dfb --- /dev/null +++ b/configs/yolov5/yolov5_m_300e_coco.yml @@ -0,0 +1,22 @@ +_BASE_: [ + '../datasets/coco_detection.yml', + '../runtime.yml', + '_base_/optimizer_300e_high.yml', + '_base_/yolov5_cspdarknet.yml', + '_base_/yolov5_reader_high_aug.yml', +] +depth_mult: 0.67 +width_mult: 0.75 + +log_iter: 100 +snapshot_epoch: 10 +weights: output/yolov5_m_300e_coco/model_final + + +TrainReader: + batch_size: 16 # default 8 gpus, total bs = 128 + + +YOLOv5Loss: + obj_weight: 0.7 + cls_weght: 0.3 diff --git a/configs/yolov5/yolov5_n_300e_coco.yml b/configs/yolov5/yolov5_n_300e_coco.yml new file mode 100644 index 0000000000000000000000000000000000000000..aa4d21693ad6e2e1200dc072582456fbf6f2c76d --- /dev/null +++ b/configs/yolov5/yolov5_n_300e_coco.yml @@ -0,0 +1,17 @@ +_BASE_: [ + '../datasets/coco_detection.yml', + '../runtime.yml', + '_base_/optimizer_300e.yml', + '_base_/yolov5_cspdarknet.yml', + '_base_/yolov5_reader.yml', +] +depth_mult: 0.33 +width_mult: 0.25 + +log_iter: 100 +snapshot_epoch: 10 +weights: output/yolov5_n_300e_coco/model_final + + +TrainReader: + batch_size: 16 # default 8 gpus, total bs = 128 diff --git a/configs/yolov5/yolov5_s_300e_coco.yml b/configs/yolov5/yolov5_s_300e_coco.yml new file mode 100644 index 0000000000000000000000000000000000000000..5b4b8edf03e3518dfb2789e639079340750b67e9 --- /dev/null +++ b/configs/yolov5/yolov5_s_300e_coco.yml @@ -0,0 +1,17 @@ +_BASE_: [ + '../datasets/coco_detection.yml', + '../runtime.yml', + '_base_/optimizer_300e.yml', + '_base_/yolov5_cspdarknet.yml', + '_base_/yolov5_reader.yml', +] +depth_mult: 0.33 +width_mult: 0.50 + +log_iter: 100 +snapshot_epoch: 10 +weights: output/yolov5_s_300e_coco/model_final + + +TrainReader: + batch_size: 16 # default 8 gpus, total bs = 128 diff --git a/configs/yolov5/yolov5_s_80e_ssod_finetune_coco.yml b/configs/yolov5/yolov5_s_80e_ssod_finetune_coco.yml new file mode 100644 index 0000000000000000000000000000000000000000..24b4b6725ac8e457216bc2a262d0c3d71ca3771d --- /dev/null +++ b/configs/yolov5/yolov5_s_80e_ssod_finetune_coco.yml @@ -0,0 +1,38 @@ +_BASE_: [ + '../datasets/coco_detection.yml', + '../runtime.yml', + '_base_/optimizer_300e.yml', + '_base_/yolov5_cspdarknet.yml', + '_base_/yolov5_reader.yml', +] +depth_mult: 0.33 +width_mult: 0.50 + +log_iter: 100 +snapshot_epoch: 10 +weights: output/yolov5_s_80e_ssod_finetune_coco/model_final +pretrain_weights: https://paddledet.bj.bcebos.com/models/pretrained/yolov5_s_300e_ssod_coco.pdparams + + +TrainReader: + batch_size: 16 # default 8 gpus, total bs = 128 + + +epoch: 80 +LearningRate: + base_lr: 0.001 + schedulers: + - !YOLOv5LRDecay + max_epochs: 80 + min_lr_ratio: 0.01 + - !ExpWarmup + epochs: 3 + +OptimizerBuilder: + optimizer: + type: Momentum + momentum: 0.937 + use_nesterov: True + regularizer: + factor: 0.0005 + type: L2 diff --git a/configs/yolov5/yolov5_x_300e_coco.yml b/configs/yolov5/yolov5_x_300e_coco.yml new file mode 100644 index 0000000000000000000000000000000000000000..7817960fecd634d61cd3f845a08ea9e87782d098 --- /dev/null +++ b/configs/yolov5/yolov5_x_300e_coco.yml @@ -0,0 +1,22 @@ +_BASE_: [ + '../datasets/coco_detection.yml', + '../runtime.yml', + '_base_/optimizer_300e_high.yml', + '_base_/yolov5_cspdarknet.yml', + '_base_/yolov5_reader_high_aug.yml', +] +depth_mult: 1.33 +width_mult: 1.25 + +log_iter: 100 +snapshot_epoch: 10 +weights: output/yolov5_x_300e_coco/model_final + + +TrainReader: + batch_size: 16 # default 8 gpus, total bs = 128 + + +YOLOv5Loss: + obj_weight: 0.7 + cls_weght: 0.3 diff --git a/configs/yolov5/yolov5p6_l_300e_coco.yml b/configs/yolov5/yolov5p6_l_300e_coco.yml new file mode 100644 index 0000000000000000000000000000000000000000..aec6ecdc63f656cfcca7b7b870280c226db7bde4 --- /dev/null +++ b/configs/yolov5/yolov5p6_l_300e_coco.yml @@ -0,0 +1,17 @@ +_BASE_: [ + '../datasets/coco_detection.yml', + '../runtime.yml', + '_base_/optimizer_300e_high.yml', + '_base_/yolov5p6_cspdarknet.yml', + '_base_/yolov5p6_reader.yml', +] +depth_mult: 1.0 +width_mult: 1.0 + +log_iter: 100 +snapshot_epoch: 10 +weights: output/yolov5p6_l_300e_coco/model_final + + +TrainReader: + batch_size: 8 # default 8 gpus, total bs = 64 diff --git a/configs/yolov5/yolov5p6_m_300e_coco.yml b/configs/yolov5/yolov5p6_m_300e_coco.yml new file mode 100644 index 0000000000000000000000000000000000000000..d46634b453d45229cf1b9da8414040b4e2d89141 --- /dev/null +++ b/configs/yolov5/yolov5p6_m_300e_coco.yml @@ -0,0 +1,17 @@ +_BASE_: [ + '../datasets/coco_detection.yml', + '../runtime.yml', + '_base_/optimizer_300e_high.yml', + '_base_/yolov5p6_cspdarknet.yml', + '_base_/yolov5p6_reader.yml', +] +depth_mult: 0.67 +width_mult: 0.75 + +log_iter: 100 +snapshot_epoch: 10 +weights: output/yolov5p6_m_300e_coco/model_final + + +TrainReader: + batch_size: 16 # default 8 gpus, total bs = 128 diff --git a/configs/yolov5/yolov5p6_n_300e_coco.yml b/configs/yolov5/yolov5p6_n_300e_coco.yml new file mode 100644 index 0000000000000000000000000000000000000000..6522d8d124027567dae7e71d96e9f54a9d5840a6 --- /dev/null +++ b/configs/yolov5/yolov5p6_n_300e_coco.yml @@ -0,0 +1,17 @@ +_BASE_: [ + '../datasets/coco_detection.yml', + '../runtime.yml', + '_base_/optimizer_300e_high.yml', + '_base_/yolov5p6_cspdarknet.yml', + '_base_/yolov5p6_reader.yml', +] +depth_mult: 0.33 +width_mult: 0.25 + +log_iter: 100 +snapshot_epoch: 10 +weights: output/yolov5p6_n_300e_coco/model_final + + +TrainReader: + batch_size: 16 # default 8 gpus, total bs = 128 diff --git a/configs/yolov5/yolov5p6_s_300e_coco.yml b/configs/yolov5/yolov5p6_s_300e_coco.yml new file mode 100644 index 0000000000000000000000000000000000000000..e086784e477cbbe7ad2a19246e78f59c8b76f31d --- /dev/null +++ b/configs/yolov5/yolov5p6_s_300e_coco.yml @@ -0,0 +1,17 @@ +_BASE_: [ + '../datasets/coco_detection.yml', + '../runtime.yml', + '_base_/optimizer_300e_high.yml', + '_base_/yolov5p6_cspdarknet.yml', + '_base_/yolov5p6_reader.yml', +] +depth_mult: 0.33 +width_mult: 0.50 + +log_iter: 100 +snapshot_epoch: 10 +weights: output/yolov5p6_s_300e_coco/model_final + + +TrainReader: + batch_size: 16 # default 8 gpus, total bs = 128 diff --git a/configs/yolov5/yolov5p6_x_300e_coco.yml b/configs/yolov5/yolov5p6_x_300e_coco.yml new file mode 100644 index 0000000000000000000000000000000000000000..628b5e5c67f5cd8039d636a1b95ed90750419b4c --- /dev/null +++ b/configs/yolov5/yolov5p6_x_300e_coco.yml @@ -0,0 +1,17 @@ +_BASE_: [ + '../datasets/coco_detection.yml', + '../runtime.yml', + '_base_/optimizer_300e_high.yml', + '_base_/yolov5p6_cspdarknet.yml', + '_base_/yolov5p6_reader.yml', +] +depth_mult: 1.33 +width_mult: 1.25 + +log_iter: 100 +snapshot_epoch: 10 +weights: output/yolov5p6_x_300e_coco/model_final + + +TrainReader: + batch_size: 8 # default 8 gpus, total bs = 64 diff --git a/dataset/coco/download_coco.py b/dataset/coco/download_coco.py new file mode 100644 index 0000000000000000000000000000000000000000..47659fa76dd2c1183404667efac3a48de9b099c2 --- /dev/null +++ b/dataset/coco/download_coco.py @@ -0,0 +1,28 @@ +# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import os.path as osp +import logging +# add python path of PadleDetection to sys.path +parent_path = osp.abspath(osp.join(__file__, *(['..'] * 3))) +if parent_path not in sys.path: + sys.path.append(parent_path) + +from ppdet.utils.download import download_dataset + +logging.basicConfig(level=logging.INFO) + +download_path = osp.split(osp.realpath(sys.argv[0]))[0] +download_dataset(download_path, 'coco') diff --git a/dataset/roadsign_voc/download_roadsign_voc.py b/dataset/roadsign_voc/download_roadsign_voc.py new file mode 100644 index 0000000000000000000000000000000000000000..3cb517d3cf362e3ad2ec7b4ebf3bff54acb244d4 --- /dev/null +++ b/dataset/roadsign_voc/download_roadsign_voc.py @@ -0,0 +1,28 @@ +# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import os.path as osp +import logging +# add python path of PadleDetection to sys.path +parent_path = osp.abspath(osp.join(__file__, *(['..'] * 3))) +if parent_path not in sys.path: + sys.path.append(parent_path) + +from ppdet.utils.download import download_dataset + +logging.basicConfig(level=logging.INFO) + +download_path = osp.split(osp.realpath(sys.argv[0]))[0] +download_dataset(download_path, 'roadsign_voc') diff --git a/dataset/roadsign_voc/label_list.txt b/dataset/roadsign_voc/label_list.txt new file mode 100644 index 0000000000000000000000000000000000000000..1be460f457a2fdbec91d3a69377c232ae4a6beb0 --- /dev/null +++ b/dataset/roadsign_voc/label_list.txt @@ -0,0 +1,4 @@ +speedlimit +crosswalk +trafficlight +stop \ No newline at end of file diff --git a/dataset/voc/create_list.py b/dataset/voc/create_list.py new file mode 100644 index 0000000000000000000000000000000000000000..5ab804222aa14cce988fd0aa083a460372afb484 --- /dev/null +++ b/dataset/voc/create_list.py @@ -0,0 +1,28 @@ +# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import os.path as osp +import logging +# add python path of PadleDetection to sys.path +parent_path = osp.abspath(osp.join(__file__, *(['..'] * 3))) +if parent_path not in sys.path: + sys.path.append(parent_path) + +from ppdet.utils.download import create_voc_list + +logging.basicConfig(level=logging.INFO) + +voc_path = osp.split(osp.realpath(sys.argv[0]))[0] +create_voc_list(voc_path) diff --git a/dataset/voc/download_voc.py b/dataset/voc/download_voc.py new file mode 100644 index 0000000000000000000000000000000000000000..e4c449ce207f27f07379a1e93cebcb27ace0b8fe --- /dev/null +++ b/dataset/voc/download_voc.py @@ -0,0 +1,28 @@ +# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import os.path as osp +import logging +# add python path of PadleDetection to sys.path +parent_path = osp.abspath(osp.join(__file__, *(['..'] * 3))) +if parent_path not in sys.path: + sys.path.append(parent_path) + +from ppdet.utils.download import download_dataset + +logging.basicConfig(level=logging.INFO) + +download_path = osp.split(osp.realpath(sys.argv[0]))[0] +download_dataset(download_path, 'voc') diff --git a/dataset/voc/label_list.txt b/dataset/voc/label_list.txt new file mode 100644 index 0000000000000000000000000000000000000000..8420ab35ede7400974f25836a6bb543024686a0e --- /dev/null +++ b/dataset/voc/label_list.txt @@ -0,0 +1,20 @@ +aeroplane +bicycle +bird +boat +bottle +bus +car +cat +chair +cow +diningtable +dog +horse +motorbike +person +pottedplant +sheep +sofa +train +tvmonitor diff --git a/demo/000000014439.jpg b/demo/000000014439.jpg new file mode 100644 index 0000000000000000000000000000000000000000..0abbdab06eb5950b93908cc91adfa640e8a3ac78 Binary files /dev/null and b/demo/000000014439.jpg differ diff --git a/demo/000000014439_640x640.jpg b/demo/000000014439_640x640.jpg new file mode 100644 index 0000000000000000000000000000000000000000..58e9d3e228af43c9b55d8d0cb385ce82ebb8b996 Binary files /dev/null and b/demo/000000014439_640x640.jpg differ diff --git a/demo/000000087038.jpg b/demo/000000087038.jpg new file mode 100644 index 0000000000000000000000000000000000000000..9f77f5d5f057b6f92dc096da704ecb8dee99bdf5 Binary files /dev/null and b/demo/000000087038.jpg differ diff --git a/demo/000000570688.jpg b/demo/000000570688.jpg new file mode 100644 index 0000000000000000000000000000000000000000..cb304bd56c4010c08611a30dcca58ea9140cea54 Binary files /dev/null and b/demo/000000570688.jpg differ diff --git a/demo/road554.png b/demo/road554.png new file mode 100644 index 0000000000000000000000000000000000000000..7733e57f922b0fee893775da4f698c202804966f Binary files /dev/null and b/demo/road554.png differ diff --git a/demo/visdrone_0000315_01601_d_0000509.jpg b/demo/visdrone_0000315_01601_d_0000509.jpg new file mode 100644 index 0000000000000000000000000000000000000000..cc7a3602c1c015213ca1f7e27b0d006e827ee935 Binary files /dev/null and b/demo/visdrone_0000315_01601_d_0000509.jpg differ diff --git a/deploy/BENCHMARK_INFER.md b/deploy/BENCHMARK_INFER.md new file mode 100644 index 0000000000000000000000000000000000000000..988cf30f6c672195d4b3833fe9a186b497a11c2e --- /dev/null +++ b/deploy/BENCHMARK_INFER.md @@ -0,0 +1,60 @@ +# 推理Benchmark + +## 一、环境准备 +- 1、测试环境: + - CUDA 10.1 + - CUDNN 7.6 + - TensorRT-6.0.1 + - PaddlePaddle v2.0.1 + - GPU分别为: Tesla V100和GTX 1080Ti和Jetson AGX Xavier +- 2、测试方式: + - 为了方便比较不同模型的推理速度，输入采用同样大小的图片，为 3x640x640，采用 `demo/000000014439_640x640.jpg` 图片。 + - Batch Size=1 + - 去掉前100轮warmup时间，测试100轮的平均时间，单位ms/image，包括网络计算时间、数据拷贝至CPU的时间。 + - 采用Fluid C++预测引擎: 包含Fluid C++预测、Fluid-TensorRT预测，下面同时测试了Float32 (FP32) 和Float16 (FP16)的推理速度。 + +**注意：** TensorRT中固定尺寸和动态尺寸区别请参考文档[TENSOR教程](TENSOR_RT.md)。由于固定尺寸下对两阶段模型支持不完善，所以faster rcnn模型采用动态尺寸测试。固定尺寸和动态尺寸支持融合的OP不完全一样，因此同一个模型在固定尺寸和动态尺寸下测试的性能可能会有一点差异。 + +## 二、推理速度 + +### 1、Linux系统 +#### （1）Tesla V100 + +| 模型 | backbone | 是否固定尺寸 | 入网尺寸 | paddle_inference | trt_fp32 | trt_fp16 | +|-------------------------------|--------------|--------|----------|------------------|----------|----------| +| Faster RCNN FPN | ResNet50 | 否 | 640x640 | 27.99 | 26.15 | 21.92 | +| Faster RCNN FPN | ResNet50 | 否 | 800x1312 | 32.49 | 25.54 | 21.70 | +| YOLOv3 | Mobilenet\_v1 | 是 | 608x608 | 9.74 | 8.61 | 6.28 | +| YOLOv3 | Darknet53 | 是 | 608x608 | 17.84 | 15.43 | 9.86 | +| PPYOLO | ResNet50 | 是 | 608x608 | 20.77 | 18.40 | 13.53 | +| SSD | Mobilenet\_v1 | 是 | 300x300 | 5.17 | 4.43 | 4.29 | +| TTFNet | Darknet53 | 是 | 512x512 | 10.14 | 8.71 | 5.55 | +| FCOS | ResNet50 | 是 | 640x640 | 35.47 | 35.02 | 34.24 | + + +#### （2）Jetson AGX Xavier + +| 模型 | backbone | 是否固定尺寸 | 入网尺寸 | paddle_inference | trt_fp32 | trt_fp16 | +|-------------------------------|--------------|--------|----------|------------------|----------|----------| +| Faster RCNN FPN | ResNet50 | 否 | 640x640 | 169.45 | 158.92 | 119.25 | +| Faster RCNN FPN | ResNet50 | 否 | 800x1312 | 228.07 | 156.39 | 117.03 | +| YOLOv3 | Mobilenet\_v1 | 是 | 608x608 | 48.76 | 43.83 | 18.41 | +| YOLOv3 | Darknet53 | 是 | 608x608 | 121.61 | 110.30 | 42.38 | +| PPYOLO | ResNet50 | 是 | 608x608 | 111.80 | 99.40 | 48.05 | +| SSD | Mobilenet\_v1 | 是 | 300x300 | 10.52 | 8.84 | 8.77 | +| TTFNet | Darknet53 | 是 | 512x512 | 73.77 | 64.03 | 31.46 | +| FCOS | ResNet50 | 是 | 640x640 | 217.11 | 214.38 | 205.78 | + +### 2、Windows系统 +#### （1）GTX 1080Ti + +| 模型 | backbone | 是否固定尺寸 | 入网尺寸 | paddle_inference | trt_fp32 | trt_fp16 | +|-------------------------------|--------------|--------|----------|------------------|----------|----------| +| Faster RCNN FPN | ResNet50 | 否 | 640x640 | 50.74 | 57.17 | 62.08 | +| Faster RCNN FPN | ResNet50 | 否 | 800x1312 | 50.31 | 57.61 | 62.05 | +| YOLOv3 | Mobilenet\_v1 | 是 | 608x608 | 14.51 | 11.23 | 11.13 | +| YOLOv3 | Darknet53 | 是 | 608x608 | 30.26 | 23.92 | 24.02 | +| PPYOLO | ResNet50 | 是 | 608x608 | 38.06 | 31.40 | 31.94 | +| SSD | Mobilenet\_v1 | 是 | 300x300 | 16.47 | 13.87 | 13.76 | +| TTFNet | Darknet53 | 是 | 512x512 | 21.83 | 17.14 | 17.09 | +| FCOS | ResNet50 | 是 | 640x640 | 71.88 | 69.93 | 69.52 | diff --git a/deploy/BENCHMARK_INFER_en.md b/deploy/BENCHMARK_INFER_en.md new file mode 100644 index 0000000000000000000000000000000000000000..b0b92b6cc142bb6b07a703ccadc5a017f8080956 --- /dev/null +++ b/deploy/BENCHMARK_INFER_en.md @@ -0,0 +1,61 @@ +# Inference Benchmark + +## 一、Prepare the Environment +- 1、Test Environment: + - CUDA 10.1 + - CUDNN 7.6 + - TensorRT-6.0.1 + - PaddlePaddle v2.0.1 + - The GPUS are Tesla V100 and GTX 1080 Ti and Jetson AGX Xavier +- 2、Test Method: + - In order to compare the inference speed of different models, the input shape is 3x640x640, use `demo/000000014439_640x640.jpg`. + - Batch_size=1 + - Delete the warmup time of the first 100 rounds and test the average time of 100 rounds in ms/image, including network calculation time and data copy time to CPU. + - Using Fluid C++ prediction engine: including Fluid C++ prediction, Fluid TensorRT prediction, the following test Float32 (FP32) and Float16 (FP16) inference speed. + +**Attention:** For TensorRT, please refer to the [TENSOR tutorial](TENSOR_RT.md) for the difference between fixed and dynamic dimensions. Due to the imperfect support for the two-stage model under fixed size, dynamic size test was adopted for the Faster RCNN model. Fixed size and dynamic size do not support exactly the same OP for fusion, so the performance of the same model tested at fixed size and dynamic size may differ slightly. + + +## 二、Inferring Speed + +### 1、Linux System +#### （1）Tesla V100 + +| Model | backbone | Fixed size or not | The net size | paddle_inference | trt_fp32 | trt_fp16 | +| --------------- | ------------- | ----------------- | ------------ | ---------------- | -------- | -------- | +| Faster RCNN FPN | ResNet50 | no | 640x640 | 27.99 | 26.15 | 21.92 | +| Faster RCNN FPN | ResNet50 | no | 800x1312 | 32.49 | 25.54 | 21.70 | +| YOLOv3 | Mobilenet\_v1 | yes | 608x608 | 9.74 | 8.61 | 6.28 | +| YOLOv3 | Darknet53 | yes | 608x608 | 17.84 | 15.43 | 9.86 | +| PPYOLO | ResNet50 | yes | 608x608 | 20.77 | 18.40 | 13.53 | +| SSD | Mobilenet\_v1 | yes | 300x300 | 5.17 | 4.43 | 4.29 | +| TTFNet | Darknet53 | yes | 512x512 | 10.14 | 8.71 | 5.55 | +| FCOS | ResNet50 | yes | 640x640 | 35.47 | 35.02 | 34.24 | + + +#### （2）Jetson AGX Xavier + +| Model | backbone | Fixed size or not | The net size | paddle_inference | trt_fp32 | trt_fp16 | +| --------------- | ------------- | ----------------- | ------------ | ---------------- | -------- | -------- | +| Faster RCNN FPN | ResNet50 | no | 640x640 | 169.45 | 158.92 | 119.25 | +| Faster RCNN FPN | ResNet50 | no | 800x1312 | 228.07 | 156.39 | 117.03 | +| YOLOv3 | Mobilenet\_v1 | yes | 608x608 | 48.76 | 43.83 | 18.41 | +| YOLOv3 | Darknet53 | yes | 608x608 | 121.61 | 110.30 | 42.38 | +| PPYOLO | ResNet50 | yes | 608x608 | 111.80 | 99.40 | 48.05 | +| SSD | Mobilenet\_v1 | yes | 300x300 | 10.52 | 8.84 | 8.77 | +| TTFNet | Darknet53 | yes | 512x512 | 73.77 | 64.03 | 31.46 | +| FCOS | ResNet50 | yes | 640x640 | 217.11 | 214.38 | 205.78 | + +### 2、Windows System +#### （1）GTX 1080Ti + +| Model | backbone | Fixed size or not | The net size | paddle_inference | trt_fp32 | trt_fp16 | +| --------------- | ------------- | ----------------- | ------------ | ---------------- | -------- | -------- | +| Faster RCNN FPN | ResNet50 | no | 640x640 | 50.74 | 57.17 | 62.08 | +| Faster RCNN FPN | ResNet50 | no | 800x1312 | 50.31 | 57.61 | 62.05 | +| YOLOv3 | Mobilenet\_v1 | yes | 608x608 | 14.51 | 11.23 | 11.13 | +| YOLOv3 | Darknet53 | yes | 608x608 | 30.26 | 23.92 | 24.02 | +| PPYOLO | ResNet50 | yes | 608x608 | 38.06 | 31.40 | 31.94 | +| SSD | Mobilenet\_v1 | yes | 300x300 | 16.47 | 13.87 | 13.76 | +| TTFNet | Darknet53 | yes | 512x512 | 21.83 | 17.14 | 17.09 | +| FCOS | ResNet50 | yes | 640x640 | 71.88 | 69.93 | 69.52 | diff --git a/deploy/EXPORT_MODEL.md b/deploy/EXPORT_MODEL.md new file mode 100644 index 0000000000000000000000000000000000000000..91f34b5860d6384baf773e71a39ffa4ec773dee6 --- /dev/null +++ b/deploy/EXPORT_MODEL.md @@ -0,0 +1,54 @@ +# PaddleDetection模型导出教程 + +## 一、模型导出 +本章节介绍如何使用`tools/export_model.py`脚本导出模型。 +### 1、导出模输入输出说明 +- 输入变量以及输入形状如下： + + | 输入名称 | 输入形状 | 表示含义 | + | :---------: | ----------- | ---------- | + | image | [None, 3, H, W] | 输入网络的图像，None表示batch维度，如果输入图像大小为变长，则H,W为None | + | im_shape | [None, 2] | 图像经过resize后的大小，表示为H,W, None表示batch维度 | + | scale_factor | [None, 2] | 输入图像大小比真实图像大小，表示为scale_y, scale_x | + +**注意**具体预处理方式可参考配置文件中TestReader部分。 + + +- PaddleDetection中动转静导出模型输出统一为： + + - bbox, NMS的输出，形状为[N, 6], 其中N为预测框的个数，6为[class_id, score, x1, y1, x2, y2]。 + - bbox\_num, 每张图片对应预测框的个数，例如batch_size为2，输出为[N1, N2], 表示第一张图包含N1个预测框，第二张图包含N2个预测框，并且预测框的总个数和NMS输出的第一维N相同 + - mask，如果网络中包含mask，则会输出mask分支 + +**注意**模型动转静导出不支持模型结构中包含numpy相关操作的情况。 + + +### 2、启动参数说明 + +| FLAG | 用途 | 默认值 | 备注 | +|:--------------:|:--------------:|:------------:|:-----------------------------------------:| +| -c | 指定配置文件 | None | | +| --output_dir | 模型保存路径 | `./output_inference` | 模型默认保存在`output/配置文件名/`路径下 | + +### 3、使用示例 + +使用训练得到的模型进行试用，脚本如下 + +```bash +# 导出YOLOv3模型 +python tools/export_model.py -c configs/yolov3/yolov3_darknet53_270e_coco.yml --output_dir=./inference_model \ + -o weights=weights/yolov3_darknet53_270e_coco.pdparams +``` + +预测模型会导出到`inference_model/yolov3_darknet53_270e_coco`目录下，分别为`infer_cfg.yml`, `model.pdiparams`, `model.pdiparams.info`, `model.pdmodel`。 + + +### 4、设置导出模型的输入大小 + +使用Fluid-TensorRT进行预测时，由于<=TensorRT 5.1的版本仅支持定长输入，保存模型的`data`层的图片大小需要和实际输入图片大小一致。而Fluid C++预测引擎没有此限制。设置TestReader中的`image_shape`可以修改保存模型中的输入图片大小。示例如下: + +```bash +# 导出YOLOv3模型，输入是3x640x640 +python tools/export_model.py -c configs/yolov3/yolov3_darknet53_270e_coco.yml --output_dir=./inference_model \ + -o weights=weights/yolov3_darknet53_270e_coco.pdparams TestReader.inputs_def.image_shape=[3,640,640] +``` diff --git a/deploy/EXPORT_MODEL_en.md b/deploy/EXPORT_MODEL_en.md new file mode 100644 index 0000000000000000000000000000000000000000..d2828edeb8388b4633b7d8489923a059ef96321c --- /dev/null +++ b/deploy/EXPORT_MODEL_en.md @@ -0,0 +1,53 @@ +# PaddleDetection Model Export Tutorial + +## 一、Model Export +This section describes how to use the `tools/export_model.py` script to export models. +### Export model input and output description +- Input variables and input shapes are as follows: + + | Input Name | Input Shape | Meaning | + | :----------: | --------------- | ------------------------------------------------------------------------------------------------------------------------- | + | image | [None, 3, H, W] | Enter the network image. None indicates the Batch dimension. If the input image size is variable length, H and W are None | + | im_shape | [None, 2] | The size of the image after resize is expressed as H,W, and None represents the Batch dimension | + | scale_factor | [None, 2] | The input image size is larger than the real image size, denoted byscale_y, scale_x | + +**Attention**For details about the preprocessing method, see the Test Reader section in the configuration file. + + +-The output of the dynamic and static derived model in Paddle Detection is unified as follows: + + - bbox, the output of NMS, in the shape of [N, 6], where N is the number of prediction boxes, and 6 is [class_id, score, x1, y1, x2, y2]. + - bbox\_num, Each picture corresponds to the number of prediction boxes. For example, batch size is 2 and the output is [N1, N2], indicating that the first picture contains N1 prediction boxes and the second picture contains N2 prediction boxes, and the total number of prediction boxes is the same as the first dimension N output by NMS + - mask, If the network contains a mask, the mask branch is printed + +**Attention**The model-to-static export does not support cases where numpy operations are included in the model structure. + + +### 2、Start Parameters + +| FLAG | USE | DEFAULT | NOTE | +| :----------: | :-----------------------------: | :------------------: | :-------------------------------------------------------------------: | +| -c | Specifying a configuration file | None | | +| --output_dir | Model save path | `./output_inference` | The model is saved in the `output/default_file_name/` path by default | + +### 3、Example + +Using the trained model for trial use, the script is as follows: + +```bash +# The YOLOv3 model is exported +python tools/export_model.py -c configs/yolov3/yolov3_darknet53_270e_coco.yml --output_dir=./inference_model \ + -o weights=weights/yolov3_darknet53_270e_coco.pdparams +``` +The prediction model will be exported to the `inference_model/yolov3_darknet53_270e_coco` directory. `infer_cfg.yml`, `model.pdiparams`, `model.pdiparams.info`, `model.pdmodel` respectively. + + +### 4、Sets the input size of the export model +When using Fluid TensorRT for prediction, since <= TensorRT 5.1 only supports fixed-length input, the image size of the `data` layer of the saved model needs to be the same as the actual input image size. Fluid C++ prediction engine does not have this limitation. Setting `image_shape` in Test Reader changes the size of the input image in the saved model. The following is an example: + + +```bash +#Export the YOLOv3 model with the input 3x640x640 +python tools/export_model.py -c configs/yolov3/yolov3_darknet53_270e_coco.yml --output_dir=./inference_model \ + -o weights=weights/yolov3_darknet53_270e_coco.pdparams TestReader.inputs_def.image_shape=[3,640,640] +``` diff --git a/deploy/EXPORT_ONNX_MODEL.md b/deploy/EXPORT_ONNX_MODEL.md new file mode 100644 index 0000000000000000000000000000000000000000..e1f4027833973a9c37fb9f144e77beeead3acb41 --- /dev/null +++ b/deploy/EXPORT_ONNX_MODEL.md @@ -0,0 +1,112 @@ +# PaddleDetection模型导出为ONNX格式教程 + +PaddleDetection模型支持保存为ONNX格式，目前测试支持的列表如下 +| 模型 | OP版本 | 备注 | +| :---- | :----- | :--- | +| YOLOv3 | 11 | 仅支持batch=1推理；模型导出需固定shape | +| PP-YOLO | 11 | 仅支持batch=1推理；MatrixNMS将被转换NMS，精度略有变化；模型导出需固定shape | +| PP-YOLOv2 | 11 | 仅支持batch=1推理；MatrixNMS将被转换NMS，精度略有变化；模型导出需固定shape | +| PP-YOLO Tiny | 11 | 仅支持batch=1推理；模型导出需固定shape | +| PP-YOLOE | 11 | 仅支持batch=1推理；模型导出需固定shape | +| PP-PicoDet | 11 | 仅支持batch=1推理；模型导出需固定shape | +| FCOS | 11 |仅支持batch=1推理 | +| PAFNet | 11 |- | +| TTFNet | 11 |-| +| SSD | 11 |仅支持batch=1推理 | +| PP-TinyPose | 11 | - | +| Faster RCNN | 16 | 仅支持batch=1推理, 依赖0.9.7及以上版本| +| Mask RCNN | 16 | 仅支持batch=1推理, 依赖0.9.7及以上版本| +| Cascade RCNN | 16 | 仅支持batch=1推理, 依赖0.9.7及以上版本| +| Cascade Mask RCNN | 16 | 仅支持batch=1推理, 依赖0.9.7及以上版本| + +保存ONNX的功能由[Paddle2ONNX](https://github.com/PaddlePaddle/Paddle2ONNX)提供，如在转换中有相关问题反馈，可在Paddle2ONNX的Github项目中通过[ISSUE](https://github.com/PaddlePaddle/Paddle2ONNX/issues)与工程师交流。 + +## 导出教程 + +### 步骤一、导出PaddlePaddle部署模型 + + +导出步骤参考文档[PaddleDetection部署模型导出教程](./EXPORT_MODEL.md), 导出示例如下 + +- 非RCNN系列模型, 以YOLOv3为例 +``` +cd PaddleDetection +python tools/export_model.py -c configs/yolov3/yolov3_darknet53_270e_coco.yml \ + -o weights=https://paddledet.bj.bcebos.com/models/yolov3_darknet53_270e_coco.pdparams \ + TestReader.inputs_def.image_shape=[3,608,608] \ + --output_dir inference_model +``` +导出后的模型保存在`inference_model/yolov3_darknet53_270e_coco/`目录中，结构如下 +``` +yolov3_darknet + ├── infer_cfg.yml # 模型配置文件信息 + ├── model.pdiparams # 静态图模型参数 + ├── model.pdiparams.info # 参数额外信息，一般无需关注 + └── model.pdmodel # 静态图模型文件 +``` +> 注意导出时的参数`TestReader.inputs_def.image_shape`，对于YOLO系列模型注意导出时指定该参数，否则无法转换成功 + +- RCNN系列模型，以Faster RCNN为例 + +RCNN系列模型导出ONNX模型时，需要去除模型中的控制流，因此需要额外添加`export_onnx=True` 字段 +``` +cd PaddleDetection +python tools/export_model.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml \ + -o weights=https://paddledet.bj.bcebos.com/models/faster_rcnn_r50_fpn_1x_coco.pdparams \ + export_onnx=True \ + --output_dir inference_model +``` + +导出的模型保存在`inference_model/faster_rcnn_r50_fpn_1x_coco/`目录中，结构如下 +``` +faster_rcnn_r50_fpn_1x_coco + ├── infer_cfg.yml # 模型配置文件信息 + ├── model.pdiparams # 静态图模型参数 + ├── model.pdiparams.info # 参数额外信息，一般无需关注 + └── model.pdmodel # 静态图模型文件 +``` + +### 步骤二、将部署模型转为ONNX格式 +安装Paddle2ONNX（高于或等于0.9.7版本) +``` +pip install paddle2onnx +``` +使用如下命令转换 +``` +# YOLOv3 +paddle2onnx --model_dir inference_model/yolov3_darknet53_270e_coco \ + --model_filename model.pdmodel \ + --params_filename model.pdiparams \ + --opset_version 11 \ + --save_file yolov3.onnx + +# Faster RCNN +paddle2onnx --model_dir inference_model/faster_rcnn_r50_fpn_1x_coco \ + --model_filename model.pdmodel \ + --params_filename model.pdiparams \ + --opset_version 16 \ + --save_file faster_rcnn.onnx +``` +转换后的模型即为在当前路径下的`yolov3.onnx`和`faster_rcnn.onnx` + +### 步骤三、使用onnxruntime进行推理 +安装onnxruntime +``` +pip install onnxruntime +``` +推理代码示例在[deploy/third_engine/onnx](./third_engine/onnx)下 + +使用如下命令进行推理： +``` +# YOLOv3 +python deploy/third_engine/onnx/infer.py + --infer_cfg inference_model/yolov3_darknet53_270e_coco/infer_cfg.yml \ + --onnx_file yolov3.onnx \ + --image_file demo/000000014439.jpg + +# Faster RCNN +python deploy/third_engine/onnx/infer.py + --infer_cfg inference_model/faster_rcnn_r50_fpn_1x_coco/infer_cfg.yml \ + --onnx_file faster_rcnn.onnx \ + --image_file demo/000000014439.jpg +``` diff --git a/deploy/EXPORT_ONNX_MODEL_en.md b/deploy/EXPORT_ONNX_MODEL_en.md new file mode 100644 index 0000000000000000000000000000000000000000..750959062dc20cc68600bbd89e9264468c11e4d6 --- /dev/null +++ b/deploy/EXPORT_ONNX_MODEL_en.md @@ -0,0 +1,110 @@ +# PaddleDetection Model Export as ONNX Format Tutorial + +PaddleDetection Model support is saved in ONNX format and the list of current test support is as follows +| Model | OP Version | NOTE | +| :---- | :----- | :--- | +| YOLOv3 | 11 | Only batch=1 inferring is supported. Model export needs fixed shape | +| PP-YOLO | 11 | Only batch=1 inferring is supported. A MatrixNMS will be converted to an NMS with slightly different precision; Model export needs fixed shape | +| PP-YOLOv2 | 11 | Only batch=1 inferring is supported. MatrixNMS will be converted to NMS with slightly different precision; Model export needs fixed shape | +| PP-YOLO Tiny | 11 | Only batch=1 inferring is supported. Model export needs fixed shape | +| PP-YOLOE | 11 | Only batch=1 inferring is supported. Model export needs fixed shape | +| PP-PicoDet | 11 | Only batch=1 inferring is supported. Model export needs fixed shape | +| FCOS | 11 |Only batch=1 inferring is supported | +| PAFNet | 11 |- | +| TTFNet | 11 |-| +| SSD | 11 |Only batch=1 inferring is supported | +| PP-TinyPose | 11 | - | +| Faster RCNN | 16 | Only batch=1 inferring is supported, require paddle2onnx>=0.9.7| +| Mask RCNN | 16 | Only batch=1 inferring is supported, require paddle2onnx>=0.9.7| +| Cascade RCNN | 16 | Only batch=1 inferring is supported, require paddle2onnx>=0.9.7| +| Cascade Mask RCNN | 16 | Only batch=1 inferring is supported, require paddle2onnx>=0.9.7| + + +The function of saving ONNX is provided by [Paddle2ONNX](https://github.com/PaddlePaddle/Paddle2ONNX). If there is feedback on related problems during conversion, Communicate with engineers in Paddle2ONNX's Github project via [ISSUE](https://github.com/PaddlePaddle/Paddle2ONNX/issues). + +## Export Tutorial + +### Step 1. Export the Paddle deployment model +Export procedure reference document[Tutorial on PaddleDetection deployment model export](./EXPORT_MODEL_en.md), for example: + +- Models except RCNN series, take YOLOv3 as example +``` +cd PaddleDetection +python tools/export_model.py -c configs/yolov3/yolov3_darknet53_270e_coco.yml \ + -o weights=https://paddledet.bj.bcebos.com/models/yolov3_darknet53_270e_coco.pdparams \ + TestReader.inputs_def.image_shape=[3,608,608] \ + --output_dir inference_model +``` +The derived models were saved in `inference_model/yolov3_darknet53_270e_coco/`, with the structure as follows +``` +yolov3_darknet + ├── infer_cfg.yml # Model configuration file information + ├── model.pdiparams # Static diagram model parameters + ├── model.pdiparams.info # Parameter Information is not required + └── model.pdmodel # Static diagram model file +``` +> check`TestReader.inputs_def.image_shape`, For YOLO series models, specify this parameter when exporting; otherwise, the conversion fails + +- RCNN series models, take Faster RCNN as example + +The conditional block needs to be removed in RCNN series when export ONNX model. Add `export_onnx=True` in command line +``` +cd PaddleDetection +python tools/export_model.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml \ + -o weights=https://paddledet.bj.bcebos.com/models/faster_rcnn_r50_fpn_1x_coco.pdparams \ + export_onnx=True \ + --output_dir inference_model +``` +The derived models were saved in `inference_model/faster_rcnn_r50_fpn_1x_coco/`, with the structure as follows +``` +faster_rcnn_r50_fpn_1x_coco + ├── infer_cfg.yml # Model configuration file information + ├── model.pdiparams # Static diagram model parameters + ├── model.pdiparams.info # Parameter Information is not required + └── model.pdmodel # Static diagram model file +``` + +### Step 2. Convert the deployment model to ONNX format +Install Paddle2ONNX (version 0.9.7 or higher) +``` +pip install paddle2onnx +``` +Use the following command to convert +``` +# YOLOv3 +paddle2onnx --model_dir inference_model/yolov3_darknet53_270e_coco \ + --model_filename model.pdmodel \ + --params_filename model.pdiparams \ + --opset_version 11 \ + --save_file yolov3.onnx + +# Faster RCNN +paddle2onnx --model_dir inference_model/faster_rcnn_r50_fpn_1x_coco \ + --model_filename model.pdmodel \ + --params_filename model.pdiparams \ + --opset_version 16 \ + --save_file faster_rcnn.onnx +``` +The transformed model is under the current path`yolov3.onnx` and `faster_rcnn.onnx` + +### Step 3. Inference with onnxruntime +Install onnxruntime +``` +pip install onnxruntime +``` +Inference code examples are in [deploy/third_engine/onnx](./third_engine/onnx) + +Use the following commands for inference： +``` +# YOLOv3 +python deploy/third_engine/onnx/infer.py + --infer_cfg inference_model/yolov3_darknet53_270e_coco/infer_cfg.yml \ + --onnx_file yolov3.onnx \ + --image_file demo/000000014439.jpg + +# Faster RCNN +python deploy/third_engine/onnx/infer.py + --infer_cfg inference_model/faster_rcnn_r50_fpn_1x_coco/infer_cfg.yml \ + --onnx_file faster_rcnn.onnx \ + --image_file demo/000000014439.jpg +``` diff --git a/deploy/README.md b/deploy/README.md new file mode 100644 index 0000000000000000000000000000000000000000..ac1ba72f61c760d04376a510af55ed6bd4ac75b7 --- /dev/null +++ b/deploy/README.md @@ -0,0 +1,84 @@ +# PaddleDetection 预测部署 + +PaddleDetection提供了Paddle Inference、Paddle Serving、Paddle-Lite多种部署形式，支持服务端、移动端、嵌入式等多种平台，提供了完善的Python和C++部署方案。 + +## PaddleDetection支持的部署形式说明 +|形式|语言|教程|设备/平台| +|-|-|-|-| +|Paddle Inference|Python|已完善|Linux(ARM\X86)、Windows +|Paddle Inference|C++|已完善|Linux(ARM\X86)、Windows| +|Paddle Serving|Python|已完善|Linux(ARM\X86)、Windows| +|Paddle-Lite|C++|已完善|Android、IOS、FPGA、RK... + + +## 1.Paddle Inference部署 + +### 1.1 导出模型 + +使用`tools/export_model.py`脚本导出模型以及部署时使用的配置文件，配置文件名字为`infer_cfg.yml`。模型导出脚本如下： +```bash +# 导出YOLOv3模型 +python tools/export_model.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml -o weights=output/yolov3_mobilenet_v1_roadsign/best_model.pdparams +``` +预测模型会导出到`output_inference/yolov3_mobilenet_v1_roadsign`目录下，分别为`infer_cfg.yml`, `model.pdiparams`, `model.pdiparams.info`, `model.pdmodel`。 +模型导出具体请参考文档[PaddleDetection模型导出教程](EXPORT_MODEL.md)。 + +### 1.2 使用PaddleInference进行预测 +* Python部署 支持`CPU`、`GPU`和`XPU`环境，支持，windows、linux系统，支持NV Jetson嵌入式设备上部署。参考文档[python部署](python/README.md) +* C++部署 支持`CPU`、`GPU`和`XPU`环境，支持，windows、linux系统，支持NV Jetson嵌入式设备上部署。参考文档[C++部署](cpp/README.md) +* PaddleDetection支持TensorRT加速,相关文档请参考[TensorRT预测部署教程](TENSOR_RT.md) + +**注意:** Paddle预测库版本需要>=2.1，batch_size>1仅支持YOLOv3和PP-YOLO。 + +## 2.PaddleServing部署 +### 2.1 导出模型 + +如果需要导出`PaddleServing`格式的模型，需要设置`export_serving_model=True`: +```buildoutcfg +python tools/export_model.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml -o weights=output/yolov3_mobilenet_v1_roadsign/best_model.pdparams --export_serving_model=True +``` +预测模型会导出到`output_inference/yolov3_darknet53_270e_coco`目录下，分别为`infer_cfg.yml`, `model.pdiparams`, `model.pdiparams.info`, `model.pdmodel`, `serving_client/`文件夹, `serving_server/`文件夹。 + +模型导出具体请参考文档[PaddleDetection模型导出教程](EXPORT_MODEL.md)。 + +### 2.2 使用PaddleServing进行预测 +* [安装PaddleServing](https://github.com/PaddlePaddle/Serving/blob/develop/README.md#installation) +* [使用PaddleServing](./serving/README.md) + + +## 3.PaddleLite部署 +- [使用PaddleLite部署PaddleDetection模型](./lite/README.md) +- 详细案例请参考[Paddle-Lite-Demo](https://github.com/PaddlePaddle/Paddle-Lite-Demo)部署。更多内容，请参考[Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite) + + +## 4.第三方部署（MNN、NCNN、Openvino） +- 第三方部署提供PicoDet、TinyPose案例，其他模型请参考修改 +- TinyPose部署推荐工具：Intel CPU端推荐使用Openvino，GPU端推荐使用PaddleInference，ARM/ANDROID端推荐使用PaddleLite或者MNN + +| Third_Engine | MNN | NCNN | OPENVINO | +| ------------ | ---- | ----- | ---------- | +| PicoDet | [PicoDet_MNN](./third_engine/demo_mnn/README.md) | [PicoDet_NCNN](./third_engine/demo_ncnn/README.md) | [PicoDet_OPENVINO](./third_engine/demo_openvino/README.md) | +| TinyPose | [TinyPose_MNN](./third_engine/demo_mnn_kpts/README.md) | - | [TinyPose_OPENVINO](./third_engine/demo_openvino_kpts/README.md) | + + + +## 5.Benchmark测试 +- 使用导出的模型，运行Benchmark批量测试脚本： +```shell +sh deploy/benchmark/benchmark.sh {model_dir} {model_name} +``` +**注意** 如果是量化模型，请使用`deploy/benchmark/benchmark_quant.sh`脚本。 +- 将测试结果log导出至Excel中： +``` +python deploy/benchmark/log_parser_excel.py --log_path=./output_pipeline --output_name=benchmark_excel.xlsx +``` + +## 6.常见问题QA +- 1、`Paddle 1.8.4`训练的模型，可以用`Paddle2.0`部署吗？ + Paddle 2.0是兼容Paddle 1.8.4的，因此是可以的。但是部分模型(如SOLOv2)使用到了Paddle 2.0中新增OP，这类模型不可以。 + +- 2、Windows编译时，预测库是VS2015编译的，选择VS2017或VS2019会有问题吗？ + 关于VS兼容性问题请参考：[C++Visual Studio 2015、2017和2019之间的二进制兼容性](https://docs.microsoft.com/zh-cn/cpp/porting/binary-compat-2015-2017?view=msvc-160) + +- 3、cuDNN 8.0.4连续预测会发生内存泄漏吗？ + 经QA测试，发现cuDNN 8系列连续预测时都有内存泄漏问题，且cuDNN 8性能差于cuDNN 7，推荐使用CUDA + cuDNN7.6.4的方式进行部署。 diff --git a/deploy/README_en.md b/deploy/README_en.md new file mode 100644 index 0000000000000000000000000000000000000000..f587b56b99e7a6b7c7ed31c5ae6307ade6e18126 --- /dev/null +++ b/deploy/README_en.md @@ -0,0 +1,83 @@ +# PaddleDetection Predict deployment + +PaddleDetection provides multiple deployment forms of Paddle Inference, Paddle Serving and Paddle-Lite, supports multiple platforms such as server, mobile and embedded, and provides a complete Python and C++ deployment solution + +## PaddleDetection This section describes the supported deployment modes +| formalization | language | Tutorial | Equipment/Platform | +| ---------------- | -------- | ----------- | ------------------------- | +| Paddle Inference | Python | Has perfect | Linux(ARM\X86)、Windows | +| Paddle Inference | C++ | Has perfect | Linux(ARM\X86)、Windows | +| Paddle Serving | Python | Has perfect | Linux(ARM\X86)、Windows | +| Paddle-Lite | C++ | Has perfect | Android、IOS、FPGA、RK... | + + +## 1.Paddle Inference Deployment + +### 1.1 The export model + +Use the `tools/export_model.py` script to export the model and the configuration file used during deployment. The configuration file name is `infer_cfg.yml`. The model export script is as follows + +```bash +# The YOLOv3 model is derived +python tools/export_model.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml -o weights=output/yolov3_mobilenet_v1_roadsign/best_model.pdparams +``` +The prediction model will be exported to the `output_inference/yolov3_mobilenet_v1_roadsign` directory `infer_cfg.yml`, `model.pdiparams`, `model.pdiparams.info`, `model.pdmodel`. For details on model export, please refer to the documentation [Tutorial on Paddle Detection MODEL EXPORT](./EXPORT_MODEL_en.md). + +### 1.2 Use Paddle Inference to Make Predictions +* Python deployment supports `CPU`, `GPU` and `XPU` environments, Windows, Linux, and NV Jetson embedded devices. Reference Documentation [Python Deployment](python/README.md) +* C++ deployment supports `CPU`, `GPU` and `XPU` environments, Windows and Linux systems, and NV Jetson embedded devices. Reference documentation [C++ deployment](cpp/README.md) +* PaddleDetection supports TensorRT acceleration. Please refer to the documentation for [TensorRT Predictive Deployment Tutorial](TENSOR_RT.md) + +**Attention:** Paddle prediction library version requires >=2.1, and batch_size>1 only supports YOLOv3 and PP-YOLO. + +## 2.PaddleServing Deployment +### 2.1 Export model + +If you want to export the model in `PaddleServing` format, set `export_serving_model=True`: +```buildoutcfg +python tools/export_model.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml -o weights=output/yolov3_mobilenet_v1_roadsign/best_model.pdparams --export_serving_model=True +``` +The prediction model will be exported to the `output_inference/yolov3_darknet53_270e_coco` directory `infer_cfg.yml`, `model.pdiparams`, `model.pdiparams.info`, `model.pdmodel`, `serving_client/` and `serving_server/` folder. + +For details on model export, please refer to the documentation [Tutorial on Paddle Detection MODEL EXPORT](./EXPORT_MODEL_en.md). + +### 2.2 Predictions are made using Paddle Serving +* [Install PaddleServing](https://github.com/PaddlePaddle/Serving/blob/develop/README.md#installation) +* [Use PaddleServing](./serving/README.md) + + +## 3. PaddleLite Deployment +- [Deploy the PaddleDetection model using PaddleLite](./lite/README.md) +- For details, please refer to [Paddle-Lite-Demo](https://github.com/PaddlePaddle/Paddle-Lite-Demo) deployment. For more information, please refer to [Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite) + + +## 4.Third-Engine deploy(MNN、NCNN、Openvino) +- The Third-Engine deploy take example of PicoDet、TinyPose，the others model is the same +- Suggestion for TinyPose: For Intel CPU Openvino is recommended，for Nvidia GPU PaddleInference is recommended，and for ARM/ANDROID PaddleLite or MNN is recommended. + +| Third_Engine | MNN | NCNN | OPENVINO | +| ------------ | ------------------------------------------------------ | -------------------------------------------------- | ------------------------------------------------------------ | +| PicoDet | [PicoDet_MNN](./third_engine/demo_mnn/README.md) | [PicoDet_NCNN](./third_engine/demo_ncnn/README.md) | [PicoDet_OPENVINO](./third_engine/demo_openvino/README.md) | +| TinyPose | [TinyPose_MNN](./third_engine/demo_mnn_kpts/README.md) | - | [TinyPose_OPENVINO](./third_engine/demo_openvino_kpts/README.md) | + + +## 5. Benchmark Test +- Using the exported model, run the Benchmark batch test script: +```shell +sh deploy/benchmark/benchmark.sh {model_dir} {model_name} +``` +**Attention** If it is a quantitative model, please use the `deploy/benchmark/benchmark_quant.sh` script. +- Export the test result log to Excel： +``` +python deploy/benchmark/log_parser_excel.py --log_path=./output_pipeline --output_name=benchmark_excel.xlsx +``` + +## 6. FAQ +- 1、Can `Paddle 1.8.4` trained models be deployed with `Paddle2.0`? + Paddle 2.0 is compatible with Paddle 1.8.4, so it is ok. However, some models (such as SOLOv2) use the new OP in Paddle 2.0, which is not allowed. + +- 2、When compiling for Windows, the prediction library is compiled with VS2015, will it be a problem to choose VS2017 or VS2019? + For compatibility issues with VS, please refer to: [C++ Visual Studio 2015, 2017 and 2019 binary compatibility](https://docs.microsoft.com/zh-cn/cpp/porting/binary-compat-2015-2017?view=msvc-160) + +- 3、Does cuDNN 8.0.4 continuously predict memory leaks? + QA tests show that cuDNN 8 series have memory leakage problems in continuous prediction, and cuDNN 8 performance is worse than cuDNN7. CUDA + cuDNN7.6.4 is recommended for deployment. diff --git a/deploy/TENSOR_RT.md b/deploy/TENSOR_RT.md new file mode 100644 index 0000000000000000000000000000000000000000..b1dd29789540746cce5f7ea3ce0a783e2178438d --- /dev/null +++ b/deploy/TENSOR_RT.md @@ -0,0 +1,98 @@ +# TensorRT预测部署教程 +TensorRT是NVIDIA提出的用于统一模型部署的加速库，可以应用于V100、JETSON Xavier等硬件，它可以极大提高预测速度。Paddle TensorRT教程请参考文档[使用Paddle-TensorRT库预测](https://www.paddlepaddle.org.cn/inference/optimize/paddle_trt.html) + +## 1. 安装PaddleInference预测库 +- Python安装包，请从[这里](https://paddleinference.paddlepaddle.org.cn/user_guides/download_lib.html#python) 下载带有tensorrt的安装包进行安装 + +- CPP预测库，请从[这里](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/05_inference_deployment/inference/build_and_install_lib_cn.html) 下载带有TensorRT编译的预测库 + +- 如果Python和CPP官网没有提供已编译好的安装包或预测库，请参考[源码安装](https://www.paddlepaddle.org.cn/documentation/docs/zh/install/compile/linux-compile.html) 自行编译 + +**注意：** +- 您的机器上TensorRT的版本需要跟您使用的预测库中TensorRT版本保持一致。 +- PaddleDetection中部署预测要求TensorRT版本 > 6.0。 + +## 2. 导出模型 +模型导出具体请参考文档[PaddleDetection模型导出教程](./EXPORT_MODEL.md)。 + +## 3. 开启TensorRT加速 +### 3.1 配置TensorRT +在使用Paddle预测库构建预测器配置config时，打开TensorRT引擎就可以了： + +``` +config->EnableUseGpu(100, 0); // 初始化100M显存，使用GPU ID为0 +config->GpuDeviceId(); // 返回正在使用的GPU ID +// 开启TensorRT预测，可提升GPU预测性能，需要使用带TensorRT的预测库 +config->EnableTensorRtEngine(1 << 20 /*workspace_size*/, + batch_size /*max_batch_size*/, + 3 /*min_subgraph_size*/, + AnalysisConfig::Precision::kFloat32 /*precision*/, + false /*use_static*/, + false /*use_calib_mode*/); + +``` +**注意:** + --run_benchmark如果设置为True，则需要安装依赖`pip install pynvml psutil GPUtil`。 + +### 3.2 TensorRT固定尺寸预测 + +例如在模型Reader配置文件中设置： +```yaml +TestReader: + inputs_def: + image_shape: [3,608,608] + ... +``` +或者在导出模型时设置`-o TestReader.inputs_def.image_shape=[3,608,608]`，模型将会进行固定尺寸预测，具体请参考[PaddleDetection模型导出教程](./EXPORT_MODEL.md) 。 + +可以通过[visualdl](https://www.paddlepaddle.org.cn/paddle/visualdl/demo/graph) 打开`model.pdmodel`文件，查看输入的第一个Tensor尺寸是否是固定的，如果不指定，尺寸会用`？`表示，如下图所示： + + + +注意：由于TesnorRT不支持在batch维度进行slice操作，Faster RCNN 和 Mask RCNN不能使用固定尺寸输入预测，所以不能设置`TestReader.inputs_def.image_shape`字段。 + +以`YOLOv3`为例，使用固定尺寸输入预测： +``` +python python/infer.py --model_dir=./output_inference/yolov3_darknet53_270e_coco/ --image_file=./demo/000000014439.jpg --device=GPU --run_mode=trt_fp32 --run_benchmark=True +``` + +### 3.3 TensorRT动态尺寸预测 + +TensorRT版本>=6时，使用TensorRT预测时，可以支持动态尺寸输入。如果模型Reader配置文件中没有设置例如`TestReader.inputs_def.image_shape=[3,608,608]`的字段，或者`image_shape=[3.-1,-1]`，导出模型将以动态尺寸进行预测。一般RCNN系列模型使用动态图尺寸预测。 +Paddle预测库关于动态尺寸输入请查看[Paddle CPP预测](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/05_inference_deployment/inference/native_infer.html) 的`SetTRTDynamicShapeInfo`函数说明。 + +`python/infer.py`设置动态尺寸输入参数说明： + +- trt_min_shape 用于设定TensorRT的输入图像height、width中的最小尺寸，默认值：1 + +- trt_max_shape 用于设定TensorRT的输入图像height、width中的最大尺寸，默认值：1280 + +- trt_opt_shape 用于设定TensorRT的输入图像height、width中的最优尺寸，默认值：640 + +**注意：`TensorRT`中动态尺寸设置是4维的，这里只设置输入图像的尺寸。** + +以`Faster RCNN`为例，使用动态尺寸输入预测： +``` +python python/infer.py --model_dir=./output_inference/faster_rcnn_r50_fpn_1x_coco/ --image_file=./demo/000000014439.jpg --device=GPU --run_mode=trt_fp16 --run_benchmark=True --trt_max_shape=1280 --trt_min_shape=800 --trt_opt_shape=960 +``` + +## 4、常见问题QA +**Q:** 提示没有`tensorrt_op` +**A:** 请检查是否使用带有TensorRT的Paddle Python包或预测库。 + +**Q:** 提示`op out of memory` +**A:** 检查GPU是否是别人也在使用，请尝试使用空闲GPU + +**Q:** 提示`some trt inputs dynamic shape info not set` +**A:** 这是由于`TensorRT`会把网络结果划分成多个子图，我们只设置了输入数据的动态尺寸，划分的其他子图的输入并未设置动态尺寸。有两个解决方法： + +- 方法一：通过增大`min_subgraph_size`，跳过对这些子图的优化。根据提示，设置min_subgraph_size大于并未设置动态尺寸输入的子图中OP个数即可。 +`min_subgraph_size`的意思是，在加载TensorRT引擎的时候，大于`min_subgraph_size`的OP才会被优化，并且这些OP是连续的且是TensorRT可以优化的。 + +- 方法二：找到子图的这些输入，按照上面方式也设置子图的输入动态尺寸。 + +**Q:** 如何打开日志 +**A:** 预测库默认是打开日志的，只要注释掉`config.disable_glog_info()`就可以打开日志 + +**Q:** 开启TensorRT，预测时提示Slice on batch axis is not supported in TensorRT +**A:** 请尝试使用动态尺寸输入 diff --git a/deploy/auto_compression/README.md b/deploy/auto_compression/README.md new file mode 100644 index 0000000000000000000000000000000000000000..11e50138663e7e41788301638e479912029d7b6a --- /dev/null +++ b/deploy/auto_compression/README.md @@ -0,0 +1,186 @@ +# 自动化压缩 + +目录： +- [1.简介](#1简介) +- [2.Benchmark](#2Benchmark) +- [3.开始自动压缩](#自动压缩流程) + - [3.1 环境准备](#31-准备环境) + - [3.2 准备数据集](#32-准备数据集) + - [3.3 准备预测模型](#33-准备预测模型) + - [3.4 测试模型精度](#34-测试模型精度) + - [3.5 自动压缩并产出模型](#35-自动压缩并产出模型) +- [4.预测部署](#4预测部署) + +## 1. 简介 +本示例使用PaddleDetection中Inference部署模型进行自动化压缩，使用的自动化压缩策略为量化蒸馏。 + + +## 2.Benchmark + +### PP-YOLOE+ + +| 模型 | Base mAP | 离线量化mAP | ACT量化mAP | TRT-FP32 | TRT-FP16 | TRT-INT8 | 配置文件 | 量化模型 | +| :-------- |:-------- |:--------: | :---------------------: | :----------------: | :----------------: | :---------------: | :----------------------: | :---------------------: | +| PP-YOLOE+_s | 43.7 | - | 42.9 | - | - | - | [config](./configs/ppyoloe_plus_s_qat_dis.yaml) | [Quant Model](https://bj.bcebos.com/v1/paddledet/deploy/Inference/ppyoloe_plus_s_qat_dis.tar) | +| PP-YOLOE+_m | 49.8 | - | 49.3 | - | - | - | [config](./configs/ppyoloe_plus_m_qat_dis.yaml) | [Quant Model](https://bj.bcebos.com/v1/paddledet/deploy/Inference/ppyoloe_plus_m_qat_dis.tar) | +| PP-YOLOE+_l | 52.9 | - | 52.6 | - | - | - | [config](./configs/ppyoloe_plus_l_qat_dis.yaml) | [Quant Model](https://bj.bcebos.com/v1/paddledet/deploy/Inference/ppyoloe_plus_l_qat_dis.tar) | +| PP-YOLOE+_x | 54.7 | - | 54.4 | - | - | - | [config](./configs/ppyoloe_plus_x_qat_dis.yaml) | [Quant Model](https://bj.bcebos.com/v1/paddledet/deploy/Inference/ppyoloe_plus_x_qat_dis.tar) | + +- mAP的指标均在COCO val2017数据集中评测得到，IoU=0.5:0.95。 + +### YOLOv8 + +| 模型 | Base mAP | 离线量化mAP | ACT量化mAP | TRT-FP32 | TRT-FP16 | TRT-INT8 | 配置文件 | 量化模型 | +| :-------- |:-------- |:--------: | :---------------------: | :----------------: | :----------------: | :---------------: | :----------------------: | :---------------------: | +| YOLOv8-s | 44.9 | 43.9 | 44.3 | 9.27ms | 4.65ms | **3.78ms** | [config](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/example/auto_compression/detection/configs/yolov8_s_qat_dis.yaml) | [Model](https://bj.bcebos.com/v1/paddle-slim-models/act/yolov8_s_500e_coco_trt_nms_quant.tar) | + +**注意：** +- 表格中YOLOv8模型均为带NMS的模型，可直接在TRT中部署，如果需要对齐测试标准，需要测试不带NMS的模型。 +- mAP的指标均在COCO val2017数据集中评测得到，IoU=0.5:0.95。 +- 表格中的性能在Tesla T4的GPU环境下测试，并且开启TensorRT，batch_size=1。 + +### PP-YOLOE + +| 模型 | Base mAP | 离线量化mAP | ACT量化mAP | TRT-FP32 | TRT-FP16 | TRT-INT8 | 配置文件 | 量化模型 | +| :-------- |:-------- |:--------: | :---------------------: | :----------------: | :----------------: | :---------------: | :----------------------: | :---------------------: | +| PP-YOLOE-l | 50.9 | - | 50.6 | 11.2ms | 7.7ms | **6.7ms** | [config](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/deploy/auto_compression/configs/ppyoloe_l_qat_dis.yaml) | [Quant Model](https://bj.bcebos.com/v1/paddle-slim-models/act/ppyoloe_crn_l_300e_coco_quant.tar) | +| PP-YOLOE-SOD | 38.5 | - | 37.6 | - | - | - | [config](./configs/ppyoloe_crn_l_80e_sliced_visdrone_640_025_qat.yml) | [Quant Model](https://bj.bcebos.com/v1/paddle-slim-models/act/ppyoloe_sod_visdrone.tar) | + +git +- PP-YOLOE-l mAP的指标在COCO val2017数据集中评测得到，IoU=0.5:0.95。 +- PP-YOLOE-l模型在Tesla V100的GPU环境下测试，并且开启TensorRT，batch_size=1，包含NMS，测试脚本是[benchmark demo](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.4/deploy/python)。 +- PP-YOLOE-SOD 的指标在VisDrone-DET数据集切图后的COCO格式[数据集](https://bj.bcebos.com/v1/paddledet/data/smalldet/visdrone_sliced.zip)中评测得到，IoU=0.5:0.95。定义文件[ppyoloe_crn_l_80e_sliced_visdrone_640_025.yml](../../configs/smalldet/ppyoloe_crn_l_80e_sliced_visdrone_640_025.yml) + +### PP-PicoDet + +| 模型 | 策略 | mAP | FP32 | FP16 | INT8 | 配置文件 | 模型 | +| :-------- |:-------- |:--------: | :----------------: | :----------------: | :---------------: | :----------------------: | :---------------------: | +| PicoDet-S-NPU | Baseline | 30.1 | - | - | - | [config](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet/picodet_s_416_coco_npu.yml) | [Model](https://bj.bcebos.com/v1/paddle-slim-models/act/picodet_s_416_coco_npu.tar) | +| PicoDet-S-NPU | 量化训练 | 29.7 | - | - | - | [config](https://github.com/PaddlePaddle/PaddleSlim/tree/develop/demo/full_quantization/detection/configs/picodet_s_qat_dis.yaml) | [Model](https://bj.bcebos.com/v1/paddle-slim-models/act/picodet_s_npu_quant.tar) | + +- mAP的指标均在COCO val2017数据集中评测得到，IoU=0.5:0.95。 + +### RT-DETR + +| 模型 | Base mAP | ACT量化mAP | TRT-FP32 | TRT-FP16 | TRT-INT8 | 配置文件 | 量化模型 | +| :---------------- | :------- | :--------: | :------: | :------: | :--------: | :----------------------------------------------------------: | :----------------------------------------------------------: | +| RT-DETR-R50 | 53.1 | 53.0 | 32.05ms | 9.12ms | **6.96ms** | [config](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/example/auto_compression/detection/configs/rtdetr_r50vd_qat_dis.yaml) | [Model](https://bj.bcebos.com/v1/paddle-slim-models/act/rtdetr_r50vd_6x_coco_quant.tar) | +| RT-DETR-R101 | 54.3 | 54.1 | 54.13ms | 12.68ms | **9.20ms** | [config](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/example/auto_compression/detection/configs/rtdetr_r101vd_qat_dis.yaml) | [Model](https://bj.bcebos.com/v1/paddle-slim-models/act/rtdetr_r101vd_6x_coco_quant.tar) | +| RT-DETR-HGNetv2-L | 53.0 | 52.9 | 26.16ms | 8.54ms | **6.65ms** | [config](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/example/auto_compression/detection/configs/rtdetr_hgnetv2_l_qat_dis.yaml) | [Model](https://bj.bcebos.com/v1/paddle-slim-models/act/rtdetr_hgnetv2_l_6x_coco_quant.tar) | +| RT-DETR-HGNetv2-X | 54.8 | 54.6 | 49.22ms | 12.50ms | **9.24ms** | [config](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/example/auto_compression/detection/configs/rtdetr_hgnetv2_x_qat_dis.yaml) | [Model](https://bj.bcebos.com/v1/paddle-slim-models/act/rtdetr_hgnetv2_x_6x_coco_quant.tar) | + +- 上表测试环境：Tesla T4，TensorRT 8.6.0，CUDA 11.7，batch_size=1。 + +| 模型 | Base mAP | ACT量化mAP | TRT-FP32 | TRT-FP16 | TRT-INT8 | 配置文件 | 量化模型 | +| :---------------- | :------- | :--------: | :------: | :------: | :--------: | :----------------------------------------------------------: | :----------------------------------------------------------: | +| RT-DETR-R50 | 53.1 | 53.0 | 9.64ms | 5.00ms | **3.99ms** | [config](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/example/auto_compression/detection/configs/rtdetr_r50vd_qat_dis.yaml) | [Model](https://bj.bcebos.com/v1/paddle-slim-models/act/rtdetr_r50vd_6x_coco_quant.tar) | +| RT-DETR-R101 | 54.3 | 54.1 | 14.93ms | 7.15ms | **5.12ms** | [config](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/example/auto_compression/detection/configs/rtdetr_r101vd_qat_dis.yaml) | [Model](https://bj.bcebos.com/v1/paddle-slim-models/act/rtdetr_r101vd_6x_coco_quant.tar) | +| RT-DETR-HGNetv2-L | 53.0 | 52.9 | 8.17ms | 4.77ms | **4.00ms** | [config](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/example/auto_compression/detection/configs/rtdetr_hgnetv2_l_qat_dis.yaml) | [Model](https://bj.bcebos.com/v1/paddle-slim-models/act/rtdetr_hgnetv2_l_6x_coco_quant.tar) | +| RT-DETR-HGNetv2-X | 54.8 | 54.6 | 12.81ms | 6.97ms | **5.32ms** | [config](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/example/auto_compression/detection/configs/rtdetr_hgnetv2_x_qat_dis.yaml) | [Model](https://bj.bcebos.com/v1/paddle-slim-models/act/rtdetr_hgnetv2_x_6x_coco_quant.tar) | + +- 上表测试环境：A10，TensorRT 8.6.0，CUDA 11.6，batch_size=1。 +- mAP的指标均在COCO val2017数据集中评测得到，IoU=0.5:0.95。 + +## 3. 自动压缩流程 + +#### 3.1 准备环境 +- PaddlePaddle >= 2.4 （可从[Paddle官网](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/install/pip/linux-pip.html)下载安装） +- PaddleSlim >= 2.4.1 +- PaddleDet >= 2.5 +- opencv-python + +安装paddlepaddle： +```shell +# CPU +pip install paddlepaddle +# GPU +pip install paddlepaddle-gpu +``` + +安装paddleslim： +```shell +pip install paddleslim +``` + +安装paddledet： +```shell +pip install paddledet +``` + +**注意：** YOLOv8模型的自动化压缩需要依赖安装最新[Develop Paddle](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html)和[Develop PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim#%E5%AE%89%E8%A3%85)版本。 + +#### 3.2 准备数据集 + +本案例默认以COCO数据进行自动压缩实验，如果自定义COCO数据，或者其他格式数据，请参考[数据准备文档](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/docs/tutorials/data/PrepareDataSet.md) 来准备数据。 + +如果数据集为非COCO格式数据，请修改[configs](./configs)中reader配置文件中的Dataset字段。 + +以PP-YOLOE模型为例，如果已经准备好数据集，请直接修改[./configs/yolo_reader.yml]中`EvalDataset`的`dataset_dir`字段为自己数据集路径即可。 + +#### 3.3 准备预测模型 + +预测模型的格式为：`model.pdmodel` 和 `model.pdiparams`两个，带`pdmodel`的是模型文件，带`pdiparams`后缀的是权重文件。 + + +根据[PaddleDetection文档](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md#8-%E6%A8%A1%E5%9E%8B%E5%AF%BC%E5%87%BA) 导出Inference模型，具体可参考下方PP-YOLOE模型的导出示例： +- 下载代码 +``` +git clone https://github.com/PaddlePaddle/PaddleDetection.git +``` +- 导出预测模型 + +PPYOLOE-l模型，包含NMS：如快速体验，可直接下载[PP-YOLOE-l导出模型](https://bj.bcebos.com/v1/paddle-slim-models/act/ppyoloe_crn_l_300e_coco.tar) +```shell +python tools/export_model.py \ + -c configs/ppyoloe/ppyoloe_crn_l_300e_coco.yml \ + -o weights=https://paddledet.bj.bcebos.com/models/ppyoloe_crn_l_300e_coco.pdparams \ + trt=True \ +``` + +YOLOv8-s模型，包含NMS，具体可参考[YOLOv8模型文档](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/yolov8), 然后执行： +```shell +python tools/export_model.py \ + -c configs/yolov8/yolov8_s_500e_coco.yml \ + -o weights=https://paddledet.bj.bcebos.com/models/yolov8_s_500e_coco.pdparams \ + trt=True +``` + +如快速体验，可直接下载[YOLOv8-s导出模型](https://bj.bcebos.com/v1/paddle-slim-models/act/yolov8_s_500e_coco_trt_nms.tar) + +#### 3.4 自动压缩并产出模型 + +蒸馏量化自动压缩示例通过run.py脚本启动，会使用接口```paddleslim.auto_compression.AutoCompression```对模型进行自动压缩。配置config文件中模型路径、蒸馏、量化、和训练等部分的参数，配置完成后便可对模型进行量化和蒸馏。具体运行命令为： + +- 单卡训练： +``` +export CUDA_VISIBLE_DEVICES=0 +python run.py --config_path=./configs/ppyoloe_l_qat_dis.yaml --save_dir='./output/' +``` + +- 多卡训练： +``` +CUDA_VISIBLE_DEVICES=0,1,2,3 python -m paddle.distributed.launch --log_dir=log --gpus 0,1,2,3 run.py \ + --config_path=./configs/ppyoloe_l_qat_dis.yaml --save_dir='./output/' +``` + +#### 3.5 测试模型精度 + +使用eval.py脚本得到模型的mAP： +``` +export CUDA_VISIBLE_DEVICES=0 +python eval.py --config_path=./configs/ppyoloe_l_qat_dis.yaml +``` + +使用paddle inference并使用trt int8得到模型的mAP: +``` +export CUDA_VISIBLE_DEVICES=0 +python paddle_inference_eval.py --model_path ./output/ --reader_config configs/ppyoloe_reader.yml --precision int8 --use_trt=True +``` + +**注意**： +- 要测试的模型路径可以在配置文件中`model_dir`字段下进行修改。 +- --precision 默认为paddle，如果使用trt，需要设置--use_trt=True，同时--precision 可设置为fp32/fp16/int8 + +## 4.预测部署 + +- 可以参考[PaddleDetection部署教程](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.4/deploy)，GPU上量化模型开启TensorRT并设置trt_int8模式进行部署。 diff --git a/deploy/auto_compression/configs/picodet_reader.yml b/deploy/auto_compression/configs/picodet_reader.yml new file mode 100644 index 0000000000000000000000000000000000000000..952a978ae32723e5a98bc63989e473d04e480c7c --- /dev/null +++ b/deploy/auto_compression/configs/picodet_reader.yml @@ -0,0 +1,32 @@ +metric: COCO +num_classes: 80 + + +# Datset configuration +TrainDataset: + !COCODataSet + image_dir: train2017 + anno_path: annotations/instances_train2017.json + dataset_dir: dataset/coco/ + +EvalDataset: + !COCODataSet + image_dir: val2017 + anno_path: annotations/instances_val2017.json + dataset_dir: dataset/coco/ + +worker_num: 6 +eval_height: &eval_height 416 +eval_width: &eval_width 416 +eval_size: &eval_size [*eval_height, *eval_width] + +EvalReader: + sample_transforms: + - Decode: {} + - Resize: {interp: 2, target_size: *eval_size, keep_ratio: False} + - NormalizeImage: {mean: [0, 0, 0], std: [1, 1, 1], is_scale: True} + - Permute: {} + batch_transforms: + - PadBatch: {pad_to_stride: 32} + batch_size: 8 + shuffle: false diff --git a/deploy/auto_compression/configs/picodet_s_qat_dis.yaml b/deploy/auto_compression/configs/picodet_s_qat_dis.yaml new file mode 100644 index 0000000000000000000000000000000000000000..a5012be15a1e6791b27a9053417709ed96830bb0 --- /dev/null +++ b/deploy/auto_compression/configs/picodet_s_qat_dis.yaml @@ -0,0 +1,34 @@ +Global: + reader_config: ./configs/picodet_reader.yml + include_nms: True + Evaluation: True + model_dir: ./picodet_s_416_coco_npu/ + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: l2 + +QuantAware: + use_pact: true + activation_quantize_type: 'moving_average_abs_max' + weight_bits: 8 + activation_bits: 8 + quantize_op_types: + - conv2d + - depthwise_conv2d + +TrainConfig: + train_iter: 8000 + eval_iter: 1000 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00001 + T_max: 8000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 4.0e-05 + + diff --git a/deploy/auto_compression/configs/ppyoloe_crn_l_80e_sliced_visdrone_640_025_qat.yml b/deploy/auto_compression/configs/ppyoloe_crn_l_80e_sliced_visdrone_640_025_qat.yml new file mode 100644 index 0000000000000000000000000000000000000000..84132455cad5a69b6e37d74343c6c7bae66c2c1a --- /dev/null +++ b/deploy/auto_compression/configs/ppyoloe_crn_l_80e_sliced_visdrone_640_025_qat.yml @@ -0,0 +1,34 @@ + +Global: + reader_config: configs/ppyoloe_crn_l_80e_sliced_visdrone_640_025_reader.yml + input_list: ['image', 'scale_factor'] + arch: YOLO + include_nms: True + Evaluation: True + model_dir: ../../output_inference/ppyoloe_crn_l_80e_sliced_visdrone_640_025 + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + onnx_format: True + use_pact: False + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + +TrainConfig: + train_iter: 8000 + eval_iter: 500 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00003 + T_max: 6000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 4.0e-05 diff --git a/deploy/auto_compression/configs/ppyoloe_crn_l_80e_sliced_visdrone_640_025_reader.yml b/deploy/auto_compression/configs/ppyoloe_crn_l_80e_sliced_visdrone_640_025_reader.yml new file mode 100644 index 0000000000000000000000000000000000000000..0869a4a10ac8355cd20a80ba0d71cddd5a6f0008 --- /dev/null +++ b/deploy/auto_compression/configs/ppyoloe_crn_l_80e_sliced_visdrone_640_025_reader.yml @@ -0,0 +1,25 @@ +metric: COCO +num_classes: 10 + +# Datset configuration +TrainDataset: + !COCODataSet + image_dir: train_images_640_025 + anno_path: train_640_025.json + dataset_dir: dataset/visdrone_sliced + +EvalDataset: + !COCODataSet + image_dir: val_images_640_025 + anno_path: val_640_025.json + dataset_dir: dataset/visdrone_sliced +worker_num: 0 + +# preprocess reader in test +EvalReader: + sample_transforms: + - Decode: {} + - Resize: {target_size: [640, 640], keep_ratio: False, interp: 2} + #- NormalizeImage: {mean: [0., 0., 0.], std: [1., 1., 1.], norm_type: none} + - Permute: {} + batch_size: 16 diff --git a/deploy/auto_compression/configs/ppyoloe_l_qat_dis.yaml b/deploy/auto_compression/configs/ppyoloe_l_qat_dis.yaml new file mode 100644 index 0000000000000000000000000000000000000000..df346d2b00ec24b351f2d62974a13e33293f431b --- /dev/null +++ b/deploy/auto_compression/configs/ppyoloe_l_qat_dis.yaml @@ -0,0 +1,32 @@ + +Global: + reader_config: configs/ppyoloe_reader.yml + include_nms: True + Evaluation: True + model_dir: ./ppyoloe_crn_l_300e_coco + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + use_pact: true + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + +TrainConfig: + train_iter: 5000 + eval_iter: 1000 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00003 + T_max: 6000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 4.0e-05 + diff --git a/deploy/auto_compression/configs/ppyoloe_plus_crn_t_auxhead_300e_coco_qat.yml b/deploy/auto_compression/configs/ppyoloe_plus_crn_t_auxhead_300e_coco_qat.yml new file mode 100644 index 0000000000000000000000000000000000000000..7f8c48ea482630bf2111de274e0316176406fcb8 --- /dev/null +++ b/deploy/auto_compression/configs/ppyoloe_plus_crn_t_auxhead_300e_coco_qat.yml @@ -0,0 +1,32 @@ + +Global: + reader_config: configs/ppyoloe_plus_reader.yml + include_nms: True + Evaluation: True + model_dir: ../../output_inference/ppyoloe_plus_crn_t_auxhead_300e_coco/ + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + onnx_format: True + use_pact: False + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + +TrainConfig: + train_iter: 8000 + eval_iter: 1000 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00003 + T_max: 6000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 4.0e-05 diff --git a/deploy/auto_compression/configs/ppyoloe_plus_l_qat_dis.yaml b/deploy/auto_compression/configs/ppyoloe_plus_l_qat_dis.yaml new file mode 100644 index 0000000000000000000000000000000000000000..fd03aed09d9a1ed3a67eec3283ef227224e941fb --- /dev/null +++ b/deploy/auto_compression/configs/ppyoloe_plus_l_qat_dis.yaml @@ -0,0 +1,32 @@ + +Global: + reader_config: configs/ppyoloe_plus_reader.yml + include_nms: True + Evaluation: True + model_dir: ./ppyoloe_plus_crn_l_80e_coco + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + use_pact: true + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + +TrainConfig: + train_iter: 5000 + eval_iter: 1000 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00003 + T_max: 6000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 4.0e-05 + diff --git a/deploy/auto_compression/configs/ppyoloe_plus_m_qat_dis.yaml b/deploy/auto_compression/configs/ppyoloe_plus_m_qat_dis.yaml new file mode 100644 index 0000000000000000000000000000000000000000..4d31332f5e3745604e03c50ad2f9db62376c1373 --- /dev/null +++ b/deploy/auto_compression/configs/ppyoloe_plus_m_qat_dis.yaml @@ -0,0 +1,32 @@ + +Global: + reader_config: configs/ppyoloe_plus_reader.yml + include_nms: True + Evaluation: True + model_dir: ./ppyoloe_plus_crn_m_80e_coco + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + use_pact: true + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + +TrainConfig: + train_iter: 5000 + eval_iter: 1000 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00003 + T_max: 6000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 4.0e-05 + diff --git a/deploy/auto_compression/configs/ppyoloe_plus_reader.yml b/deploy/auto_compression/configs/ppyoloe_plus_reader.yml new file mode 100644 index 0000000000000000000000000000000000000000..5f3795f29be025e6836a7c88b51dd79ecb04a9f4 --- /dev/null +++ b/deploy/auto_compression/configs/ppyoloe_plus_reader.yml @@ -0,0 +1,26 @@ +metric: COCO +num_classes: 80 + +# Datset configuration +TrainDataset: + !COCODataSet + image_dir: train2017 + anno_path: annotations/instances_train2017.json + dataset_dir: dataset/coco/ + +EvalDataset: + !COCODataSet + image_dir: val2017 + anno_path: annotations/instances_val2017.json + dataset_dir: dataset/coco/ + +worker_num: 0 + +# preprocess reader in test +EvalReader: + sample_transforms: + - Decode: {} + - Resize: {target_size: [640, 640], keep_ratio: False, interp: 2} + - NormalizeImage: {mean: [0., 0., 0.], std: [1., 1., 1.], norm_type: none} + - Permute: {} + batch_size: 4 diff --git a/deploy/auto_compression/configs/ppyoloe_plus_s_qat_dis.yaml b/deploy/auto_compression/configs/ppyoloe_plus_s_qat_dis.yaml new file mode 100644 index 0000000000000000000000000000000000000000..41bfde1e47855cdd1c543d13292d387781b8c0d6 --- /dev/null +++ b/deploy/auto_compression/configs/ppyoloe_plus_s_qat_dis.yaml @@ -0,0 +1,32 @@ + +Global: + reader_config: configs/ppyoloe_plus_reader.yml + include_nms: True + Evaluation: True + model_dir: ./ppyoloe_plus_crn_s_80e_coco + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + use_pact: true + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + +TrainConfig: + train_iter: 5000 + eval_iter: 1000 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00003 + T_max: 6000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 4.0e-05 + diff --git a/deploy/auto_compression/configs/ppyoloe_plus_sod_crn_l_qat_dis.yaml b/deploy/auto_compression/configs/ppyoloe_plus_sod_crn_l_qat_dis.yaml new file mode 100644 index 0000000000000000000000000000000000000000..793afa1a0eba1c45a16042d4d2bf96d7c2777353 --- /dev/null +++ b/deploy/auto_compression/configs/ppyoloe_plus_sod_crn_l_qat_dis.yaml @@ -0,0 +1,33 @@ + +Global: + reader_config: configs/ppyoloe_plus_reader.yml + include_nms: True + Evaluation: True + model_dir: ../../output_inference/ppyoloe_plus_sod_crn_l_80e_coco + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + onnx_format: True + use_pact: true + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + +TrainConfig: + train_iter: 1 + eval_iter: 1 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00003 + T_max: 6000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 4.0e-05 + diff --git a/deploy/auto_compression/configs/ppyoloe_plus_x_qat_dis.yaml b/deploy/auto_compression/configs/ppyoloe_plus_x_qat_dis.yaml new file mode 100644 index 0000000000000000000000000000000000000000..ac62e7ca2d22bae19ffcf99f8265a05ea7e1331c --- /dev/null +++ b/deploy/auto_compression/configs/ppyoloe_plus_x_qat_dis.yaml @@ -0,0 +1,32 @@ + +Global: + reader_config: configs/ppyoloe_plus_reader.yml + include_nms: True + Evaluation: True + model_dir: ./ppyoloe_plus_crn_x_80e_coco + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + use_pact: true + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + +TrainConfig: + train_iter: 5000 + eval_iter: 1000 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00003 + T_max: 6000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 4.0e-05 + diff --git a/deploy/auto_compression/configs/ppyoloe_reader.yml b/deploy/auto_compression/configs/ppyoloe_reader.yml new file mode 100644 index 0000000000000000000000000000000000000000..d1061453051e8f7408f4e605078956a8b634f13c --- /dev/null +++ b/deploy/auto_compression/configs/ppyoloe_reader.yml @@ -0,0 +1,26 @@ +metric: COCO +num_classes: 80 + +# Datset configuration +TrainDataset: + !COCODataSet + image_dir: train2017 + anno_path: annotations/instances_train2017.json + dataset_dir: dataset/coco/ + +EvalDataset: + !COCODataSet + image_dir: val2017 + anno_path: annotations/instances_val2017.json + dataset_dir: dataset/coco/ + +worker_num: 0 + +# preprocess reader in test +EvalReader: + sample_transforms: + - Decode: {} + - Resize: {target_size: [640, 640], keep_ratio: False, interp: 2} + - NormalizeImage: {mean: [0.485, 0.456, 0.406], std: [0.229, 0.224, 0.225], is_scale: True} + - Permute: {} + batch_size: 4 diff --git a/deploy/auto_compression/configs/rtdetr_hgnetv2_l_qat_dis.yaml b/deploy/auto_compression/configs/rtdetr_hgnetv2_l_qat_dis.yaml new file mode 100644 index 0000000000000000000000000000000000000000..83ebd78b1a343c3401830076ab90157a83947026 --- /dev/null +++ b/deploy/auto_compression/configs/rtdetr_hgnetv2_l_qat_dis.yaml @@ -0,0 +1,32 @@ + +Global: + reader_config: configs/rtdetr_reader.yml + include_nms: True + Evaluation: True + model_dir: ./rtdetr_hgnetv2_l_6x_coco/ + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + onnx_format: true + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + - matmul_v2 + +TrainConfig: + train_iter: 200 + eval_iter: 50 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00003 + T_max: 10000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 4.0e-05 diff --git a/deploy/auto_compression/configs/rtdetr_hgnetv2_x_qat_dis.yaml b/deploy/auto_compression/configs/rtdetr_hgnetv2_x_qat_dis.yaml new file mode 100644 index 0000000000000000000000000000000000000000..c4e2889bbd3c69e758199f99b6fd517d227799d0 --- /dev/null +++ b/deploy/auto_compression/configs/rtdetr_hgnetv2_x_qat_dis.yaml @@ -0,0 +1,32 @@ + +Global: + reader_config: configs/rtdetr_reader.yml + include_nms: True + Evaluation: True + model_dir: ./rtdetr_r50vd_6x_coco/ + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + onnx_format: true + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + - matmul_v2 + +TrainConfig: + train_iter: 500 + eval_iter: 100 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00003 + T_max: 10000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 4.0e-05 diff --git a/deploy/auto_compression/configs/rtdetr_r101vd_qat_dis.yaml b/deploy/auto_compression/configs/rtdetr_r101vd_qat_dis.yaml new file mode 100644 index 0000000000000000000000000000000000000000..bd96d085e8a01cabba890f2602e9619855bf9d5a --- /dev/null +++ b/deploy/auto_compression/configs/rtdetr_r101vd_qat_dis.yaml @@ -0,0 +1,32 @@ + +Global: + reader_config: configs/rtdetr_reader.yml + include_nms: True + Evaluation: True + model_dir: ./rtdetr_hgnetv2_x_6x_coco/ + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + onnx_format: true + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + - matmul_v2 + +TrainConfig: + train_iter: 200 + eval_iter: 50 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00003 + T_max: 10000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 4.0e-05 diff --git a/deploy/auto_compression/configs/rtdetr_r50vd_qat_dis.yaml b/deploy/auto_compression/configs/rtdetr_r50vd_qat_dis.yaml new file mode 100644 index 0000000000000000000000000000000000000000..c4e2889bbd3c69e758199f99b6fd517d227799d0 --- /dev/null +++ b/deploy/auto_compression/configs/rtdetr_r50vd_qat_dis.yaml @@ -0,0 +1,32 @@ + +Global: + reader_config: configs/rtdetr_reader.yml + include_nms: True + Evaluation: True + model_dir: ./rtdetr_r50vd_6x_coco/ + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + onnx_format: true + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + - matmul_v2 + +TrainConfig: + train_iter: 500 + eval_iter: 100 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00003 + T_max: 10000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 4.0e-05 diff --git a/deploy/auto_compression/configs/rtdetr_reader.yml b/deploy/auto_compression/configs/rtdetr_reader.yml new file mode 100644 index 0000000000000000000000000000000000000000..7b213ffa202f8812f337f223c721a829fd8a55df --- /dev/null +++ b/deploy/auto_compression/configs/rtdetr_reader.yml @@ -0,0 +1,26 @@ +metric: COCO +num_classes: 80 + +# Datset configuration +TrainDataset: + !COCODataSet + image_dir: train2017 + anno_path: annotations/instances_train2017.json + dataset_dir: dataset/coco/ + !COCODataSet + image_dir: val2017 + anno_path: annotations/instances_val2017.json + dataset_dir: dataset/coco/ + +worker_num: 0 + +# preprocess reader in test +EvalReader: + sample_transforms: + - Decode: {} + - Resize: {target_size: [640, 640], keep_ratio: False, interp: 2} + - NormalizeImage: {mean: [0., 0., 0.], std: [1., 1., 1.], norm_type: none} + - Permute: {} + batch_size: 1 + shuffle: false + drop_last: false diff --git a/deploy/auto_compression/configs/yolov5_reader.yml b/deploy/auto_compression/configs/yolov5_reader.yml new file mode 100644 index 0000000000000000000000000000000000000000..6ad321a04d12f822e98facd179d9d72b0d8aa741 --- /dev/null +++ b/deploy/auto_compression/configs/yolov5_reader.yml @@ -0,0 +1,26 @@ +metric: COCO +num_classes: 80 + +# Datset configuration +TrainDataset: + !COCODataSet + image_dir: train2017 + anno_path: annotations/instances_train2017.json + dataset_dir: dataset/coco/ + +EvalDataset: + !COCODataSet + image_dir: val2017 + anno_path: annotations/instances_val2017.json + dataset_dir: dataset/coco/ + +worker_num: 0 + +# preprocess reader in test +TestReader: + sample_transforms: + - Decode: {} + - Resize: {target_size: [640, 640], keep_ratio: True, interp: 1} + - Pad: {size: [640, 640], fill_value: [114., 114., 114.]} + - Permute: {} + batch_size: 1 diff --git a/deploy/auto_compression/configs/yolov5_s_qat_dis.yml b/deploy/auto_compression/configs/yolov5_s_qat_dis.yml new file mode 100644 index 0000000000000000000000000000000000000000..309977ef696ab23cc859fa224486e2ed7e91900e --- /dev/null +++ b/deploy/auto_compression/configs/yolov5_s_qat_dis.yml @@ -0,0 +1,29 @@ + +Global: + reader_config: configs/yolov5_reader.yml + include_nms: True + Evaluation: True + model_dir: ./yolov5_s_300e_coco + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + use_pact: true + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + +TrainConfig: + train_iter: 3000 + eval_iter: 1000 + learning_rate: 0.00001 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 4.0e-05 + target_metric: 0.365 diff --git a/deploy/auto_compression/configs/yolov6mt_s_qat_dis.yaml b/deploy/auto_compression/configs/yolov6mt_s_qat_dis.yaml new file mode 100644 index 0000000000000000000000000000000000000000..e134494fe2833333f3b2bcf87edb71e0b870a56f --- /dev/null +++ b/deploy/auto_compression/configs/yolov6mt_s_qat_dis.yaml @@ -0,0 +1,30 @@ + +Global: + reader_config: configs/yolov5_reader.yml + include_nms: True + Evaluation: True + model_dir: ./yolov6mt_s_400e_coco + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + +TrainConfig: + train_iter: 8000 + eval_iter: 1000 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00003 + T_max: 8000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 0.00004 diff --git a/deploy/auto_compression/configs/yolov7_l_qat_dis.yaml b/deploy/auto_compression/configs/yolov7_l_qat_dis.yaml new file mode 100644 index 0000000000000000000000000000000000000000..801ccb4057c4f36fe379c281a21965ddc63a2e8b --- /dev/null +++ b/deploy/auto_compression/configs/yolov7_l_qat_dis.yaml @@ -0,0 +1,30 @@ + +Global: + reader_config: configs/yolov5_reader.yml + include_nms: True + Evaluation: True + model_dir: ./yolov7_l_300e_coco + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + +TrainConfig: + train_iter: 8000 + eval_iter: 1000 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00003 + T_max: 8000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 0.00004 \ No newline at end of file diff --git a/deploy/auto_compression/configs/yolov8_reader.yml b/deploy/auto_compression/configs/yolov8_reader.yml new file mode 100644 index 0000000000000000000000000000000000000000..202a49415572201811ed53fe806c2b31c9051fde --- /dev/null +++ b/deploy/auto_compression/configs/yolov8_reader.yml @@ -0,0 +1,27 @@ +metric: COCO +num_classes: 80 + +# Dataset configuration +TrainDataset: + !COCODataSet + image_dir: train2017 + anno_path: annotations/instances_train2017.json + dataset_dir: dataset/coco/ + +EvalDataset: + !COCODataSet + image_dir: val2017 + anno_path: annotations/instances_val2017.json + dataset_dir: dataset/coco/ + +worker_num: 0 + +# preprocess reader in test +EvalReader: + sample_transforms: + - Decode: {} + - Resize: {target_size: [640, 640], keep_ratio: True, interp: 1} + - Pad: {size: [640, 640], fill_value: [114., 114., 114.]} + - NormalizeImage: {mean: [0., 0., 0.], std: [1., 1., 1.], norm_type: none} + - Permute: {} + batch_size: 4 diff --git a/deploy/auto_compression/configs/yolov8_s_qat_dis.yaml b/deploy/auto_compression/configs/yolov8_s_qat_dis.yaml new file mode 100644 index 0000000000000000000000000000000000000000..8c93e203e918d798e055e260d73f747a6ef9d5cb --- /dev/null +++ b/deploy/auto_compression/configs/yolov8_s_qat_dis.yaml @@ -0,0 +1,32 @@ + +Global: + reader_config: configs/yolov8_reader.yml + include_nms: True + Evaluation: True + model_dir: ./yolov8_s_500e_coco_trt_nms/ + model_filename: model.pdmodel + params_filename: model.pdiparams + +Distillation: + alpha: 1.0 + loss: soft_label + +QuantAware: + onnx_format: true + activation_quantize_type: 'moving_average_abs_max' + quantize_op_types: + - conv2d + - depthwise_conv2d + +TrainConfig: + train_iter: 8000 + eval_iter: 1000 + learning_rate: + type: CosineAnnealingDecay + learning_rate: 0.00003 + T_max: 10000 + optimizer_builder: + optimizer: + type: SGD + weight_decay: 4.0e-05 + diff --git a/deploy/auto_compression/eval.py b/deploy/auto_compression/eval.py new file mode 100644 index 0000000000000000000000000000000000000000..6de8aff85ce5f3cffa4119a1a3c26e318101db74 --- /dev/null +++ b/deploy/auto_compression/eval.py @@ -0,0 +1,163 @@ +# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os +import sys +import numpy as np +import argparse +import paddle +from ppdet.core.workspace import load_config, merge_config +from ppdet.core.workspace import create +from ppdet.metrics import COCOMetric, VOCMetric, KeyPointTopDownCOCOEval +from paddleslim.auto_compression.config_helpers import load_config as load_slim_config +from post_process import PPYOLOEPostProcess + + +def argsparser(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + '--config_path', + type=str, + default=None, + help="path of compression strategy config.", + required=True) + parser.add_argument( + '--devices', + type=str, + default='gpu', + help="which device used to compress.") + + return parser + + +def reader_wrapper(reader, input_list): + def gen(): + for data in reader: + in_dict = {} + if isinstance(input_list, list): + for input_name in input_list: + in_dict[input_name] = data[input_name] + elif isinstance(input_list, dict): + for input_name in input_list.keys(): + in_dict[input_list[input_name]] = data[input_name] + yield in_dict + + return gen + + +def convert_numpy_data(data, metric): + data_all = {} + data_all = {k: np.array(v) for k, v in data.items()} + if isinstance(metric, VOCMetric): + for k, v in data_all.items(): + if not isinstance(v[0], np.ndarray): + tmp_list = [] + for t in v: + tmp_list.append(np.array(t)) + data_all[k] = np.array(tmp_list) + else: + data_all = {k: np.array(v) for k, v in data.items()} + return data_all + + +def eval(): + + place = paddle.CUDAPlace(0) if FLAGS.devices == 'gpu' else paddle.CPUPlace() + exe = paddle.static.Executor(place) + + val_program, feed_target_names, fetch_targets = paddle.static.load_inference_model( + global_config["model_dir"].rstrip('/'), + exe, + model_filename=global_config["model_filename"], + params_filename=global_config["params_filename"]) + print('Loaded model from: {}'.format(global_config["model_dir"])) + + metric = global_config['metric'] + for batch_id, data in enumerate(val_loader): + data_all = convert_numpy_data(data, metric) + data_input = {} + for k, v in data.items(): + if isinstance(global_config['input_list'], list): + if k in global_config['input_list']: + data_input[k] = np.array(v) + elif isinstance(global_config['input_list'], dict): + if k in global_config['input_list'].keys(): + data_input[global_config['input_list'][k]] = np.array(v) + + outs = exe.run(val_program, + feed=data_input, + fetch_list=fetch_targets, + return_numpy=False) + res = {} + if 'arch' in global_config and global_config['arch'] == 'PPYOLOE': + postprocess = PPYOLOEPostProcess( + score_threshold=0.01, nms_threshold=0.6) + res = postprocess(np.array(outs[0]), data_all['scale_factor']) + else: + for out in outs: + v = np.array(out) + if len(v.shape) > 1: + res['bbox'] = v + else: + res['bbox_num'] = v + metric.update(data_all, res) + if batch_id % 100 == 0: + print('Eval iter:', batch_id) + metric.accumulate() + metric.log() + metric.reset() + + +def main(): + global global_config + all_config = load_slim_config(FLAGS.config_path) + assert "Global" in all_config, "Key 'Global' not found in config file." + global_config = all_config["Global"] + reader_cfg = load_config(global_config['reader_config']) + + dataset = reader_cfg['EvalDataset'] + global val_loader + val_loader = create('EvalReader')(reader_cfg['EvalDataset'], + reader_cfg['worker_num'], + return_list=True) + metric = None + if reader_cfg['metric'] == 'COCO': + clsid2catid = {v: k for k, v in dataset.catid2clsid.items()} + anno_file = dataset.get_anno() + metric = COCOMetric( + anno_file=anno_file, clsid2catid=clsid2catid, IouType='bbox') + elif reader_cfg['metric'] == 'VOC': + metric = VOCMetric( + label_list=dataset.get_label_list(), + class_num=reader_cfg['num_classes'], + map_type=reader_cfg['map_type']) + elif reader_cfg['metric'] == 'KeyPointTopDownCOCOEval': + anno_file = dataset.get_anno() + metric = KeyPointTopDownCOCOEval(anno_file, + len(dataset), 17, 'output_eval') + else: + raise ValueError("metric currently only supports COCO and VOC.") + global_config['metric'] = metric + + eval() + + +if __name__ == '__main__': + paddle.enable_static() + parser = argsparser() + FLAGS = parser.parse_args() + assert FLAGS.devices in ['cpu', 'gpu', 'xpu', 'npu'] + paddle.set_device(FLAGS.devices) + + main() diff --git a/deploy/auto_compression/paddle_inference_eval.py b/deploy/auto_compression/paddle_inference_eval.py new file mode 100644 index 0000000000000000000000000000000000000000..053ee35e752c17fdb0069c6953b5e8f43f6aded1 --- /dev/null +++ b/deploy/auto_compression/paddle_inference_eval.py @@ -0,0 +1,499 @@ +#opyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os +import argparse +import time +import sys +import cv2 +import numpy as np + +import paddle +from paddle.inference import Config +from paddle.inference import create_predictor +from ppdet.core.workspace import load_config, create +from ppdet.metrics import COCOMetric + +from post_process import PPYOLOEPostProcess + + +def argsparser(): + """ + argsparser func + """ + parser = argparse.ArgumentParser() + parser.add_argument( + "--model_path", type=str, help="inference model filepath") + parser.add_argument( + "--image_file", + type=str, + default=None, + help="image path, if set image_file, it will not eval coco.") + parser.add_argument( + "--reader_config", + type=str, + default=None, + help="path of datset and reader config.") + parser.add_argument( + "--benchmark", + type=bool, + default=False, + help="Whether run benchmark or not.") + parser.add_argument( + "--use_trt", + type=bool, + default=False, + help="Whether use TensorRT or not.") + parser.add_argument( + "--precision", + type=str, + default="paddle", + help="mode of running(fp32/fp16/int8)") + parser.add_argument( + "--device", + type=str, + default="GPU", + help="Choose the device you want to run, it can be: CPU/GPU/XPU, default is GPU", + ) + parser.add_argument( + "--use_dynamic_shape", + type=bool, + default=True, + help="Whether use dynamic shape or not.") + parser.add_argument( + "--use_mkldnn", + type=bool, + default=False, + help="Whether use mkldnn or not.") + parser.add_argument( + "--cpu_threads", type=int, default=10, help="Num of cpu threads.") + parser.add_argument("--img_shape", type=int, default=640, help="input_size") + parser.add_argument( + '--include_nms', + type=bool, + default=True, + help="Whether include nms or not.") + + return parser + + +CLASS_LABEL = [ + 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', + 'truck', 'boat', 'traffic light', 'fire hydrant', 'stop sign', + 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', + 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', + 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball', 'kite', + 'baseball bat', 'baseball glove', 'skateboard', 'surfboard', + 'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', + 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', + 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'potted plant', + 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', + 'keyboard', 'cell phone', 'microwave', 'oven', 'toaster', 'sink', + 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear', + 'hair drier', 'toothbrush' +] + + +def generate_scale(im, target_shape, keep_ratio=True): + """ + Args: + im (np.ndarray): image (np.ndarray) + Returns: + im_scale_x: the resize ratio of X + im_scale_y: the resize ratio of Y + """ + origin_shape = im.shape[:2] + if keep_ratio: + im_size_min = np.min(origin_shape) + im_size_max = np.max(origin_shape) + target_size_min = np.min(target_shape) + target_size_max = np.max(target_shape) + im_scale = float(target_size_min) / float(im_size_min) + if np.round(im_scale * im_size_max) > target_size_max: + im_scale = float(target_size_max) / float(im_size_max) + im_scale_x = im_scale + im_scale_y = im_scale + else: + resize_h, resize_w = target_shape + im_scale_y = resize_h / float(origin_shape[0]) + im_scale_x = resize_w / float(origin_shape[1]) + return im_scale_y, im_scale_x + + +def image_preprocess(img_path, target_shape): + """ + image_preprocess func + """ + img = cv2.imread(img_path) + im_scale_y, im_scale_x = generate_scale(img, target_shape, keep_ratio=False) + img = cv2.resize( + img, (target_shape[0], target_shape[0]), + interpolation=cv2.INTER_LANCZOS4) + img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) + img = np.transpose(img, [2, 0, 1]) / 255 + img = np.expand_dims(img, 0) + img_mean = np.array([0.485, 0.456, 0.406]).reshape((3, 1, 1)) + img_std = np.array([0.229, 0.224, 0.225]).reshape((3, 1, 1)) + img -= img_mean + img /= img_std + scale_factor = np.array([[im_scale_y, im_scale_x]]) + return img.astype(np.float32), scale_factor.astype(np.float32) + + +def get_color_map_list(num_classes): + """ + get_color_map_list func + """ + color_map = num_classes * [0, 0, 0] + for i in range(0, num_classes): + j = 0 + lab = i + while lab: + color_map[i * 3] |= ((lab >> 0) & 1) << (7 - j) + color_map[i * 3 + 1] |= ((lab >> 1) & 1) << (7 - j) + color_map[i * 3 + 2] |= ((lab >> 2) & 1) << (7 - j) + j += 1 + lab >>= 3 + color_map = [color_map[i:i + 3] for i in range(0, len(color_map), 3)] + return color_map + + +def draw_box(image_file, results, class_label, threshold=0.5): + """ + draw_box func + """ + srcimg = cv2.imread(image_file, 1) + for i in range(len(results)): + color_list = get_color_map_list(len(class_label)) + clsid2color = {} + classid, conf = int(results[i, 0]), results[i, 1] + if conf < threshold: + continue + xmin, ymin, xmax, ymax = int(results[i, 2]), int(results[i, 3]), int( + results[i, 4]), int(results[i, 5]) + + if classid not in clsid2color: + clsid2color[classid] = color_list[classid] + color = tuple(clsid2color[classid]) + + cv2.rectangle(srcimg, (xmin, ymin), (xmax, ymax), color, thickness=2) + print(class_label[classid] + ": " + str(round(conf, 3))) + cv2.putText( + srcimg, + class_label[classid] + ":" + str(round(conf, 3)), + (xmin, ymin - 10), + cv2.FONT_HERSHEY_SIMPLEX, + 0.8, + (0, 255, 0), + thickness=2, ) + return srcimg + + +def load_predictor( + model_dir, + precision="fp32", + use_trt=False, + use_mkldnn=False, + batch_size=1, + device="CPU", + min_subgraph_size=3, + use_dynamic_shape=False, + trt_min_shape=1, + trt_max_shape=1280, + trt_opt_shape=640, + cpu_threads=1, ): + """set AnalysisConfig, generate AnalysisPredictor + Args: + model_dir (str): root path of __model__ and __params__ + precision (str): mode of running(fp32/fp16/int8) + use_trt (bool): whether use TensorRT or not. + use_mkldnn (bool): whether use MKLDNN or not in CPU. + device (str): Choose the device you want to run, it can be: CPU/GPU, default is CPU + use_dynamic_shape (bool): use dynamic shape or not + trt_min_shape (int): min shape for dynamic shape in trt + trt_max_shape (int): max shape for dynamic shape in trt + trt_opt_shape (int): opt shape for dynamic shape in trt + Returns: + predictor (PaddlePredictor): AnalysisPredictor + Raises: + ValueError: predict by TensorRT need device == 'GPU'. + """ + rerun_flag = False + if device != "GPU" and use_trt: + raise ValueError( + "Predict by TensorRT mode: {}, expect device=='GPU', but device == {}". + format(precision, device)) + config = Config( + os.path.join(model_dir, "model.pdmodel"), + os.path.join(model_dir, "model.pdiparams")) + if device == "GPU": + # initial GPU memory(M), device ID + config.enable_use_gpu(200, 0) + # optimize graph and fuse op + config.switch_ir_optim(True) + else: + config.disable_gpu() + config.set_cpu_math_library_num_threads(cpu_threads) + config.switch_ir_optim() + if use_mkldnn: + config.enable_mkldnn() + if precision == "int8": + config.enable_mkldnn_int8( + {"conv2d", "depthwise_conv2d", "transpose2", "pool2d"}) + + precision_map = { + "int8": Config.Precision.Int8, + "fp32": Config.Precision.Float32, + "fp16": Config.Precision.Half, + } + if precision in precision_map.keys() and use_trt: + config.enable_tensorrt_engine( + workspace_size=(1 << 25) * batch_size, + max_batch_size=batch_size, + min_subgraph_size=min_subgraph_size, + precision_mode=precision_map[precision], + use_static=True, + use_calib_mode=False, ) + + if use_dynamic_shape: + dynamic_shape_file = os.path.join(FLAGS.model_path, + "dynamic_shape.txt") + if os.path.exists(dynamic_shape_file): + config.enable_tuned_tensorrt_dynamic_shape(dynamic_shape_file, + True) + print("trt set dynamic shape done!") + else: + config.collect_shape_range_info(dynamic_shape_file) + print("Start collect dynamic shape...") + rerun_flag = True + + # enable shared memory + config.enable_memory_optim() + predictor = create_predictor(config) + return predictor, rerun_flag + + +def get_current_memory_mb(): + """ + It is used to Obtain the memory usage of the CPU and GPU during the running of the program. + And this function Current program is time-consuming. + """ + try: + pkg.require('pynvml') + except: + from pip._internal import main + main(['install', 'pynvml']) + try: + pkg.require('psutil') + except: + from pip._internal import main + main(['install', 'psutil']) + try: + pkg.require('GPUtil') + except: + from pip._internal import main + main(['install', 'GPUtil']) + import pynvml + import psutil + import GPUtil + + gpu_id = int(os.environ.get("CUDA_VISIBLE_DEVICES", 0)) + + pid = os.getpid() + p = psutil.Process(pid) + info = p.memory_full_info() + cpu_mem = info.uss / 1024.0 / 1024.0 + gpu_mem = 0 + gpu_percent = 0 + gpus = GPUtil.getGPUs() + if gpu_id is not None and len(gpus) > 0: + gpu_percent = gpus[gpu_id].load + pynvml.nvmlInit() + handle = pynvml.nvmlDeviceGetHandleByIndex(0) + meminfo = pynvml.nvmlDeviceGetMemoryInfo(handle) + gpu_mem = meminfo.used / 1024.0 / 1024.0 + return round(cpu_mem, 4), round(gpu_mem, 4) + + +def predict_image(predictor, + image_file, + image_shape=[640, 640], + warmup=1, + repeats=1, + threshold=0.5): + """ + predict image main func + """ + img, scale_factor = image_preprocess(image_file, image_shape) + inputs = {} + inputs["image"] = img + if FLAGS.include_nms: + inputs['scale_factor'] = scale_factor + input_names = predictor.get_input_names() + for i, _ in enumerate(input_names): + input_tensor = predictor.get_input_handle(input_names[i]) + input_tensor.copy_from_cpu(inputs[input_names[i]]) + + for i in range(warmup): + predictor.run() + + np_boxes, np_boxes_num = None, None + cpu_mems, gpu_mems = 0, 0 + predict_time = 0.0 + time_min = float("inf") + time_max = float("-inf") + for i in range(repeats): + start_time = time.time() + predictor.run() + output_names = predictor.get_output_names() + boxes_tensor = predictor.get_output_handle(output_names[0]) + np_boxes = boxes_tensor.copy_to_cpu() + if FLAGS.include_nms: + boxes_num = predictor.get_output_handle(output_names[1]) + np_boxes_num = boxes_num.copy_to_cpu() + end_time = time.time() + timed = end_time - start_time + time_min = min(time_min, timed) + time_max = max(time_max, timed) + predict_time += timed + cpu_mem, gpu_mem = get_current_memory_mb() + cpu_mems += cpu_mem + gpu_mems += gpu_mem + + time_avg = predict_time / repeats + print("[Benchmark]Avg cpu_mem:{} MB, avg gpu_mem: {} MB".format( + cpu_mems / repeats, gpu_mems / repeats)) + print("[Benchmark]Inference time(ms): min={}, max={}, avg={}".format( + round(time_min * 1000, 2), + round(time_max * 1000, 1), round(time_avg * 1000, 1))) + if not FLAGS.include_nms: + postprocess = PPYOLOEPostProcess(score_threshold=0.3, nms_threshold=0.6) + res = postprocess(np_boxes, scale_factor) + else: + res = {'bbox': np_boxes, 'bbox_num': np_boxes_num} + res_img = draw_box( + image_file, res["bbox"], CLASS_LABEL, threshold=threshold) + cv2.imwrite("result.jpg", res_img) + + +def eval(predictor, val_loader, metric, rerun_flag=False): + """ + eval main func + """ + cpu_mems, gpu_mems = 0, 0 + predict_time = 0.0 + time_min = float("inf") + time_max = float("-inf") + sample_nums = len(val_loader) + input_names = predictor.get_input_names() + output_names = predictor.get_output_names() + boxes_tensor = predictor.get_output_handle(output_names[0]) + if FLAGS.include_nms: + boxes_num = predictor.get_output_handle(output_names[1]) + for batch_id, data in enumerate(val_loader): + data_all = {k: np.array(v) for k, v in data.items()} + for i, _ in enumerate(input_names): + input_tensor = predictor.get_input_handle(input_names[i]) + input_tensor.copy_from_cpu(data_all[input_names[i]]) + start_time = time.time() + predictor.run() + np_boxes = boxes_tensor.copy_to_cpu() + if FLAGS.include_nms: + np_boxes_num = boxes_num.copy_to_cpu() + if rerun_flag: + return + end_time = time.time() + timed = end_time - start_time + time_min = min(time_min, timed) + time_max = max(time_max, timed) + predict_time += timed + cpu_mem, gpu_mem = get_current_memory_mb() + cpu_mems += cpu_mem + gpu_mems += gpu_mem + if not FLAGS.include_nms: + postprocess = PPYOLOEPostProcess( + score_threshold=0.3, nms_threshold=0.6) + res = postprocess(np_boxes, data_all['scale_factor']) + else: + res = {'bbox': np_boxes, 'bbox_num': np_boxes_num} + metric.update(data_all, res) + if batch_id % 100 == 0: + print("Eval iter:", batch_id) + sys.stdout.flush() + metric.accumulate() + metric.log() + map_res = metric.get_results() + metric.reset() + time_avg = predict_time / sample_nums + print("[Benchmark]Avg cpu_mem:{} MB, avg gpu_mem: {} MB".format( + cpu_mems / sample_nums, gpu_mems / sample_nums)) + print("[Benchmark]Inference time(ms): min={}, max={}, avg={}".format( + round(time_min * 1000, 2), + round(time_max * 1000, 1), round(time_avg * 1000, 1))) + print("[Benchmark] COCO mAP: {}".format(map_res["bbox"][0])) + sys.stdout.flush() + + +def main(): + """ + main func + """ + predictor, rerun_flag = load_predictor( + FLAGS.model_path, + device=FLAGS.device, + use_trt=FLAGS.use_trt, + use_mkldnn=FLAGS.use_mkldnn, + precision=FLAGS.precision, + use_dynamic_shape=FLAGS.use_dynamic_shape, + cpu_threads=FLAGS.cpu_threads) + + if FLAGS.image_file: + warmup, repeats = 1, 1 + if FLAGS.benchmark: + warmup, repeats = 50, 100 + predict_image( + predictor, + FLAGS.image_file, + image_shape=[FLAGS.img_shape, FLAGS.img_shape], + warmup=warmup, + repeats=repeats) + else: + reader_cfg = load_config(FLAGS.reader_config) + + dataset = reader_cfg["EvalDataset"] + global val_loader + val_loader = create("EvalReader")(reader_cfg["EvalDataset"], + reader_cfg["worker_num"], + return_list=True) + clsid2catid = {v: k for k, v in dataset.catid2clsid.items()} + anno_file = dataset.get_anno() + metric = COCOMetric( + anno_file=anno_file, clsid2catid=clsid2catid, IouType="bbox") + eval(predictor, val_loader, metric, rerun_flag=rerun_flag) + + if rerun_flag: + print( + "***** Collect dynamic shape done, Please rerun the program to get correct results. *****" + ) + + +if __name__ == "__main__": + paddle.enable_static() + parser = argsparser() + FLAGS = parser.parse_args() + + # DataLoader need run on cpu + paddle.set_device("cpu") + + main() diff --git a/deploy/auto_compression/post_process.py b/deploy/auto_compression/post_process.py new file mode 100644 index 0000000000000000000000000000000000000000..eea2f019548ec288a23e37b3bd2faf24f9a98935 --- /dev/null +++ b/deploy/auto_compression/post_process.py @@ -0,0 +1,157 @@ +# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import numpy as np +import cv2 + + +def hard_nms(box_scores, iou_threshold, top_k=-1, candidate_size=200): + """ + Args: + box_scores (N, 5): boxes in corner-form and probabilities. + iou_threshold: intersection over union threshold. + top_k: keep top_k results. If k <= 0, keep all the results. + candidate_size: only consider the candidates with the highest scores. + Returns: + picked: a list of indexes of the kept boxes + """ + scores = box_scores[:, -1] + boxes = box_scores[:, :-1] + picked = [] + indexes = np.argsort(scores) + indexes = indexes[-candidate_size:] + while len(indexes) > 0: + current = indexes[-1] + picked.append(current) + if 0 < top_k == len(picked) or len(indexes) == 1: + break + current_box = boxes[current, :] + indexes = indexes[:-1] + rest_boxes = boxes[indexes, :] + iou = iou_of( + rest_boxes, + np.expand_dims( + current_box, axis=0), ) + indexes = indexes[iou <= iou_threshold] + + return box_scores[picked, :] + + +def iou_of(boxes0, boxes1, eps=1e-5): + """Return intersection-over-union (Jaccard index) of boxes. + Args: + boxes0 (N, 4): ground truth boxes. + boxes1 (N or 1, 4): predicted boxes. + eps: a small number to avoid 0 as denominator. + Returns: + iou (N): IoU values. + """ + overlap_left_top = np.maximum(boxes0[..., :2], boxes1[..., :2]) + overlap_right_bottom = np.minimum(boxes0[..., 2:], boxes1[..., 2:]) + + overlap_area = area_of(overlap_left_top, overlap_right_bottom) + area0 = area_of(boxes0[..., :2], boxes0[..., 2:]) + area1 = area_of(boxes1[..., :2], boxes1[..., 2:]) + return overlap_area / (area0 + area1 - overlap_area + eps) + + +def area_of(left_top, right_bottom): + """Compute the areas of rectangles given two corners. + Args: + left_top (N, 2): left top corner. + right_bottom (N, 2): right bottom corner. + Returns: + area (N): return the area. + """ + hw = np.clip(right_bottom - left_top, 0.0, None) + return hw[..., 0] * hw[..., 1] + + +class PPYOLOEPostProcess(object): + """ + Args: + input_shape (int): network input image size + scale_factor (float): scale factor of ori image + """ + + def __init__(self, + score_threshold=0.4, + nms_threshold=0.5, + nms_top_k=10000, + keep_top_k=300): + self.score_threshold = score_threshold + self.nms_threshold = nms_threshold + self.nms_top_k = nms_top_k + self.keep_top_k = keep_top_k + + def _non_max_suppression(self, prediction, scale_factor): + batch_size = prediction.shape[0] + out_boxes_list = [] + box_num_list = [] + for batch_id in range(batch_size): + bboxes, confidences = prediction[batch_id][..., :4], prediction[ + batch_id][..., 4:] + # nms + picked_box_probs = [] + picked_labels = [] + for class_index in range(0, confidences.shape[1]): + probs = confidences[:, class_index] + mask = probs > self.score_threshold + probs = probs[mask] + if probs.shape[0] == 0: + continue + subset_boxes = bboxes[mask, :] + box_probs = np.concatenate( + [subset_boxes, probs.reshape(-1, 1)], axis=1) + box_probs = hard_nms( + box_probs, + iou_threshold=self.nms_threshold, + top_k=self.nms_top_k) + picked_box_probs.append(box_probs) + picked_labels.extend([class_index] * box_probs.shape[0]) + + if len(picked_box_probs) == 0: + out_boxes_list.append(np.empty((0, 4))) + + else: + picked_box_probs = np.concatenate(picked_box_probs) + # resize output boxes + picked_box_probs[:, 0] /= scale_factor[batch_id][1] + picked_box_probs[:, 2] /= scale_factor[batch_id][1] + picked_box_probs[:, 1] /= scale_factor[batch_id][0] + picked_box_probs[:, 3] /= scale_factor[batch_id][0] + + # clas score box + out_box = np.concatenate( + [ + np.expand_dims( + np.array(picked_labels), axis=-1), np.expand_dims( + picked_box_probs[:, 4], axis=-1), + picked_box_probs[:, :4] + ], + axis=1) + if out_box.shape[0] > self.keep_top_k: + out_box = out_box[out_box[:, 1].argsort()[::-1] + [:self.keep_top_k]] + out_boxes_list.append(out_box) + box_num_list.append(out_box.shape[0]) + + out_boxes_list = np.concatenate(out_boxes_list, axis=0) + box_num_list = np.array(box_num_list) + return out_boxes_list, box_num_list + + def __call__(self, outs, scale_factor): + out_boxes_list, box_num_list = self._non_max_suppression(outs, + scale_factor) + return {'bbox': out_boxes_list, 'bbox_num': box_num_list} diff --git a/deploy/auto_compression/run.py b/deploy/auto_compression/run.py new file mode 100644 index 0000000000000000000000000000000000000000..d940307db618c80f015b32637e7610784d1affb9 --- /dev/null +++ b/deploy/auto_compression/run.py @@ -0,0 +1,191 @@ +# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os +import sys +import numpy as np +import argparse +import paddle +from ppdet.core.workspace import load_config, merge_config +from ppdet.core.workspace import create +from ppdet.metrics import COCOMetric, VOCMetric, KeyPointTopDownCOCOEval +from paddleslim.auto_compression.config_helpers import load_config as load_slim_config +from paddleslim.auto_compression import AutoCompression +from post_process import PPYOLOEPostProcess +from paddleslim.common.dataloader import get_feed_vars + + +def argsparser(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + '--config_path', + type=str, + default=None, + help="path of compression strategy config.", + required=True) + parser.add_argument( + '--save_dir', + type=str, + default='output', + help="directory to save compressed model.") + parser.add_argument( + '--devices', + type=str, + default='gpu', + help="which device used to compress.") + + return parser + + +def reader_wrapper(reader, input_list): + def gen(): + for data in reader: + in_dict = {} + if isinstance(input_list, list): + for input_name in input_list: + in_dict[input_name] = data[input_name] + elif isinstance(input_list, dict): + for input_name in input_list.keys(): + in_dict[input_list[input_name]] = data[input_name] + yield in_dict + + return gen + + +def convert_numpy_data(data, metric): + data_all = {} + data_all = {k: np.array(v) for k, v in data.items()} + if isinstance(metric, VOCMetric): + for k, v in data_all.items(): + if not isinstance(v[0], np.ndarray): + tmp_list = [] + for t in v: + tmp_list.append(np.array(t)) + data_all[k] = np.array(tmp_list) + else: + data_all = {k: np.array(v) for k, v in data.items()} + return data_all + + +def eval_function(exe, compiled_test_program, test_feed_names, test_fetch_list): + metric = global_config['metric'] + for batch_id, data in enumerate(val_loader): + data_all = convert_numpy_data(data, metric) + data_input = {} + for k, v in data.items(): + if isinstance(global_config['input_list'], list): + if k in test_feed_names: + data_input[k] = np.array(v) + elif isinstance(global_config['input_list'], dict): + if k in global_config['input_list'].keys(): + data_input[global_config['input_list'][k]] = np.array(v) + outs = exe.run(compiled_test_program, + feed=data_input, + fetch_list=test_fetch_list, + return_numpy=False) + res = {} + if 'include_nms' in global_config and not global_config['include_nms']: + if 'arch' in global_config and global_config['arch'] == 'PPYOLOE': + postprocess = PPYOLOEPostProcess( + score_threshold=0.01, nms_threshold=0.6) + else: + assert "Not support arch={} now.".format(global_config['arch']) + res = postprocess(np.array(outs[0]), data_all['scale_factor']) + else: + for out in outs: + v = np.array(out) + if len(v.shape) > 1: + res['bbox'] = v + else: + res['bbox_num'] = v + + metric.update(data_all, res) + if batch_id % 100 == 0: + print('Eval iter:', batch_id) + metric.accumulate() + metric.log() + map_res = metric.get_results() + metric.reset() + map_key = 'keypoint' if 'arch' in global_config and global_config[ + 'arch'] == 'keypoint' else 'bbox' + return map_res[map_key][0] + + +def main(): + global global_config + all_config = load_slim_config(FLAGS.config_path) + assert "Global" in all_config, "Key 'Global' not found in config file." + global_config = all_config["Global"] + reader_cfg = load_config(global_config['reader_config']) + + train_loader = create('EvalReader')(reader_cfg['TrainDataset'], + reader_cfg['worker_num'], + return_list=True) + if global_config.get('input_list') is None: + global_config['input_list'] = get_feed_vars( + global_config['model_dir'], global_config['model_filename'], + global_config['params_filename']) + train_loader = reader_wrapper(train_loader, global_config['input_list']) + + if 'Evaluation' in global_config.keys() and global_config[ + 'Evaluation'] and paddle.distributed.get_rank() == 0: + eval_func = eval_function + dataset = reader_cfg['EvalDataset'] + global val_loader + _eval_batch_sampler = paddle.io.BatchSampler( + dataset, batch_size=reader_cfg['EvalReader']['batch_size']) + val_loader = create('EvalReader')(dataset, + reader_cfg['worker_num'], + batch_sampler=_eval_batch_sampler, + return_list=True) + metric = None + if reader_cfg['metric'] == 'COCO': + clsid2catid = {v: k for k, v in dataset.catid2clsid.items()} + anno_file = dataset.get_anno() + metric = COCOMetric( + anno_file=anno_file, clsid2catid=clsid2catid, IouType='bbox') + elif reader_cfg['metric'] == 'VOC': + metric = VOCMetric( + label_list=dataset.get_label_list(), + class_num=reader_cfg['num_classes'], + map_type=reader_cfg['map_type']) + elif reader_cfg['metric'] == 'KeyPointTopDownCOCOEval': + anno_file = dataset.get_anno() + metric = KeyPointTopDownCOCOEval(anno_file, + len(dataset), 17, 'output_eval') + else: + raise ValueError("metric currently only supports COCO and VOC.") + global_config['metric'] = metric + else: + eval_func = None + + ac = AutoCompression( + model_dir=global_config["model_dir"], + model_filename=global_config["model_filename"], + params_filename=global_config["params_filename"], + save_dir=FLAGS.save_dir, + config=all_config, + train_dataloader=train_loader, + eval_callback=eval_func) + ac.compress() + + +if __name__ == '__main__': + paddle.enable_static() + parser = argsparser() + FLAGS = parser.parse_args() + assert FLAGS.devices in ['cpu', 'gpu', 'xpu', 'npu'] + paddle.set_device(FLAGS.devices) + + main() diff --git a/deploy/benchmark/benchmark.sh b/deploy/benchmark/benchmark.sh new file mode 100644 index 0000000000000000000000000000000000000000..e29aaa884d30316237aede0c18b38e2cc520ee4b --- /dev/null +++ b/deploy/benchmark/benchmark.sh @@ -0,0 +1,36 @@ +# All rights `PaddleDetection` reserved +#!/bin/bash +model_dir=$1 +model_name=$2 + +export img_dir="demo" +export log_path="output_pipeline" + + +echo "model_dir : ${model_dir}" +echo "img_dir: ${img_dir}" + +# TODO: support batch size>1 +for use_mkldnn in "True" "False"; do + for threads in "1" "6"; do + echo "${model_name} ${model_dir}, use_mkldnn: ${use_mkldnn} threads: ${threads}" + python deploy/python/infer.py \ + --model_dir=${model_dir} \ + --run_benchmark True \ + --enable_mkldnn=${use_mkldnn} \ + --device=CPU \ + --cpu_threads=${threads} \ + --image_dir=${img_dir} 2>&1 | tee ${log_path}/${model_name}_cpu_usemkldnn_${use_mkldnn}_cputhreads_${threads}_bs1_infer.log + done +done + +for run_mode in "fluid" "trt_fp32" "trt_fp16"; do + echo "${model_name} ${model_dir}, run_mode: ${run_mode}" + python deploy/python/infer.py \ + --model_dir=${model_dir} \ + --run_benchmark=True \ + --device=GPU \ + --run_mode=${run_mode} \ + --image_dir=${img_dir} 2>&1 | tee ${log_path}/${model_name}_gpu_runmode_${run_mode}_bs1_infer.log +done + diff --git a/deploy/benchmark/benchmark_quant.sh b/deploy/benchmark/benchmark_quant.sh new file mode 100644 index 0000000000000000000000000000000000000000..a21541dd044bf9bd4a33bb4eb2116b47743e5a8a --- /dev/null +++ b/deploy/benchmark/benchmark_quant.sh @@ -0,0 +1,23 @@ +# All rights `PaddleDetection` reserved +#!/bin/bash +model_dir=$1 +model_name=$2 + +export img_dir="demo" +export log_path="output_pipeline" + + +echo "model_dir : ${model_dir}" +echo "img_dir: ${img_dir}" + +# TODO: support batch size>1 +for run_mode in "trt_int8"; do + echo "${model_name} ${model_dir}, run_mode: ${run_mode}" + python deploy/python/infer.py \ + --model_dir=${model_dir} \ + --run_benchmark=True \ + --device=GPU \ + --run_mode=${run_mode} \ + --image_dir=${img_dir} 2>&1 | tee ${log_path}/${model_name}_gpu_runmode_${run_mode}_bs1_infer.log +done + diff --git a/deploy/benchmark/log_parser_excel.py b/deploy/benchmark/log_parser_excel.py new file mode 100644 index 0000000000000000000000000000000000000000..317b3759572c6acef3438fbc654bc5918e8bdd38 --- /dev/null +++ b/deploy/benchmark/log_parser_excel.py @@ -0,0 +1,300 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os +import re +import argparse +import pandas as pd + + +def parse_args(): + """ + parse input args + """ + parser = argparse.ArgumentParser() + parser.add_argument( + "--log_path", + type=str, + default="./output_pipeline", + help="benchmark log path") + parser.add_argument( + "--output_name", + type=str, + default="benchmark_excel.xlsx", + help="output excel file name") + parser.add_argument( + "--analysis_trt", dest="analysis_trt", action='store_true') + parser.add_argument( + "--analysis_mkl", dest="analysis_mkl", action='store_true') + return parser.parse_args() + + +def find_all_logs(path_walk): + """ + find all .log files from target dir + """ + for root, ds, files in os.walk(path_walk): + for file_name in files: + if re.match(r'.*.log', file_name): + full_path = os.path.join(root, file_name) + yield file_name, full_path + + +def process_log(file_name): + """ + process log to dict + """ + output_dict = {} + with open(file_name, 'r') as f: + for i, data in enumerate(f.readlines()): + if i == 0: + continue + line_lists = data.split(" ") + + # conf info + if "runtime_device:" in line_lists: + pos_buf = line_lists.index("runtime_device:") + output_dict["runtime_device"] = line_lists[pos_buf + 1].strip() + if "ir_optim:" in line_lists: + pos_buf = line_lists.index("ir_optim:") + output_dict["ir_optim"] = line_lists[pos_buf + 1].strip() + if "enable_memory_optim:" in line_lists: + pos_buf = line_lists.index("enable_memory_optim:") + output_dict["enable_memory_optim"] = line_lists[pos_buf + + 1].strip() + if "enable_tensorrt:" in line_lists: + pos_buf = line_lists.index("enable_tensorrt:") + output_dict["enable_tensorrt"] = line_lists[pos_buf + 1].strip() + if "precision:" in line_lists: + pos_buf = line_lists.index("precision:") + output_dict["precision"] = line_lists[pos_buf + 1].strip() + if "enable_mkldnn:" in line_lists: + pos_buf = line_lists.index("enable_mkldnn:") + output_dict["enable_mkldnn"] = line_lists[pos_buf + 1].strip() + if "cpu_math_library_num_threads:" in line_lists: + pos_buf = line_lists.index("cpu_math_library_num_threads:") + output_dict["cpu_math_library_num_threads"] = line_lists[ + pos_buf + 1].strip() + + # model info + if "model_name:" in line_lists: + pos_buf = line_lists.index("model_name:") + output_dict["model_name"] = list( + filter(None, line_lists[pos_buf + 1].strip().split('/')))[ + -1] + + # data info + if "batch_size:" in line_lists: + pos_buf = line_lists.index("batch_size:") + output_dict["batch_size"] = line_lists[pos_buf + 1].strip() + if "input_shape:" in line_lists: + pos_buf = line_lists.index("input_shape:") + output_dict["input_shape"] = line_lists[pos_buf + 1].strip() + + # perf info + if "cpu_rss(MB):" in line_lists: + pos_buf = line_lists.index("cpu_rss(MB):") + output_dict["cpu_rss(MB)"] = line_lists[pos_buf + 1].strip( + ).split(',')[0] + if "gpu_rss(MB):" in line_lists: + pos_buf = line_lists.index("gpu_rss(MB):") + output_dict["gpu_rss(MB)"] = line_lists[pos_buf + 1].strip( + ).split(',')[0] + if "gpu_util:" in line_lists: + pos_buf = line_lists.index("gpu_util:") + output_dict["gpu_util"] = line_lists[pos_buf + 1].strip().split( + ',')[0] + if "preproce_time(ms):" in line_lists: + pos_buf = line_lists.index("preproce_time(ms):") + output_dict["preproce_time(ms)"] = line_lists[ + pos_buf + 1].strip().split(',')[0] + if "inference_time(ms):" in line_lists: + pos_buf = line_lists.index("inference_time(ms):") + output_dict["inference_time(ms)"] = line_lists[ + pos_buf + 1].strip().split(',')[0] + if "postprocess_time(ms):" in line_lists: + pos_buf = line_lists.index("postprocess_time(ms):") + output_dict["postprocess_time(ms)"] = line_lists[ + pos_buf + 1].strip().split(',')[0] + return output_dict + + +def filter_df_merge(cpu_df, filter_column=None): + """ + process cpu data frame, merge by 'model_name', 'batch_size' + Args: + cpu_df ([type]): [description] + """ + if not filter_column: + raise Exception( + "please assign filter_column for filter_df_merge function") + + df_lists = [] + filter_column_lists = [] + for k, v in cpu_df.groupby(filter_column, dropna=True): + filter_column_lists.append(k) + df_lists.append(v) + final_output_df = df_lists[-1] + + # merge same model + for i in range(len(df_lists) - 1): + left_suffix = cpu_df[filter_column].unique()[0] + right_suffix = df_lists[i][filter_column].unique()[0] + print(left_suffix, right_suffix) + if not pd.isnull(right_suffix): + final_output_df = pd.merge( + final_output_df, + df_lists[i], + how='left', + left_on=['model_name', 'batch_size'], + right_on=['model_name', 'batch_size'], + suffixes=('', '_{0}_{1}'.format(filter_column, right_suffix))) + + # rename default df columns + origin_column_names = list(cpu_df.columns.values) + origin_column_names.remove(filter_column) + suffix = final_output_df[filter_column].unique()[0] + for name in origin_column_names: + final_output_df.rename( + columns={name: "{0}_{1}_{2}".format(name, filter_column, suffix)}, + inplace=True) + final_output_df.rename( + columns={ + filter_column: "{0}_{1}_{2}".format(filter_column, filter_column, + suffix) + }, + inplace=True) + + final_output_df.sort_values( + by=[ + "model_name_{0}_{1}".format(filter_column, suffix), + "batch_size_{0}_{1}".format(filter_column, suffix) + ], + inplace=True) + return final_output_df + + +def trt_perf_analysis(raw_df): + """ + sperate raw dataframe to a list of dataframe + compare tensorrt percision performance + """ + # filter df by gpu, compare tensorrt and gpu + # define default dataframe for gpu performance analysis + gpu_df = raw_df.loc[raw_df['runtime_device'] == 'gpu'] + new_df = filter_df_merge(gpu_df, "precision") + + # calculate qps diff percentile + infer_fp32 = "inference_time(ms)_precision_fp32" + infer_fp16 = "inference_time(ms)_precision_fp16" + infer_int8 = "inference_time(ms)_precision_int8" + new_df["fp32_fp16_diff"] = new_df[[infer_fp32, infer_fp16]].apply( + lambda x: (float(x[infer_fp16]) - float(x[infer_fp32])) / float(x[infer_fp32]), + axis=1) + new_df["fp32_gpu_diff"] = new_df[["inference_time(ms)", infer_fp32]].apply( + lambda x: (float(x[infer_fp32]) - float(x[infer_fp32])) / float(x["inference_time(ms)"]), + axis=1) + new_df["fp16_int8_diff"] = new_df[[infer_fp16, infer_int8]].apply( + lambda x: (float(x[infer_int8]) - float(x[infer_fp16])) / float(x[infer_fp16]), + axis=1) + + return new_df + + +def mkl_perf_analysis(raw_df): + """ + sperate raw dataframe to a list of dataframe + compare mkldnn performance with not enable mkldnn + """ + # filter df by cpu, compare mkl and cpu + # define default dataframe for cpu mkldnn analysis + cpu_df = raw_df.loc[raw_df['runtime_device'] == 'cpu'] + mkl_compare_df = cpu_df.loc[cpu_df['cpu_math_library_num_threads'] == '1'] + thread_compare_df = cpu_df.loc[cpu_df['enable_mkldnn'] == 'True'] + + # define dataframe need to be analyzed + output_mkl_df = filter_df_merge(mkl_compare_df, 'enable_mkldnn') + output_thread_df = filter_df_merge(thread_compare_df, + 'cpu_math_library_num_threads') + + # calculate performance diff percentile + # compare mkl performance with cpu + enable_mkldnn = "inference_time(ms)_enable_mkldnn_True" + disable_mkldnn = "inference_time(ms)_enable_mkldnn_False" + output_mkl_df["mkl_infer_diff"] = output_mkl_df[[ + enable_mkldnn, disable_mkldnn + ]].apply( + lambda x: (float(x[enable_mkldnn]) - float(x[disable_mkldnn])) / float(x[disable_mkldnn]), + axis=1) + cpu_enable_mkldnn = "cpu_rss(MB)_enable_mkldnn_True" + cpu_disable_mkldnn = "cpu_rss(MB)_enable_mkldnn_False" + output_mkl_df["mkl_cpu_rss_diff"] = output_mkl_df[[ + cpu_enable_mkldnn, cpu_disable_mkldnn + ]].apply( + lambda x: (float(x[cpu_enable_mkldnn]) - float(x[cpu_disable_mkldnn])) / float(x[cpu_disable_mkldnn]), + axis=1) + + # compare cpu_multi_thread performance with cpu + num_threads_1 = "inference_time(ms)_cpu_math_library_num_threads_1" + num_threads_6 = "inference_time(ms)_cpu_math_library_num_threads_6" + output_thread_df["mkl_infer_diff"] = output_thread_df[[ + num_threads_6, num_threads_1 + ]].apply( + lambda x: (float(x[num_threads_6]) - float(x[num_threads_1])) / float(x[num_threads_1]), + axis=1) + cpu_num_threads_1 = "cpu_rss(MB)_cpu_math_library_num_threads_1" + cpu_num_threads_6 = "cpu_rss(MB)_cpu_math_library_num_threads_6" + output_thread_df["mkl_cpu_rss_diff"] = output_thread_df[[ + cpu_num_threads_6, cpu_num_threads_1 + ]].apply( + lambda x: (float(x[cpu_num_threads_6]) - float(x[cpu_num_threads_1])) / float(x[cpu_num_threads_1]), + axis=1) + + return output_mkl_df, output_thread_df + + +def main(): + """ + main + """ + args = parse_args() + # create empty DataFrame + origin_df = pd.DataFrame(columns=[ + "model_name", "batch_size", "input_shape", "runtime_device", "ir_optim", + "enable_memory_optim", "enable_tensorrt", "precision", "enable_mkldnn", + "cpu_math_library_num_threads", "preproce_time(ms)", + "inference_time(ms)", "postprocess_time(ms)", "cpu_rss(MB)", + "gpu_rss(MB)", "gpu_util" + ]) + + for file_name, full_path in find_all_logs(args.log_path): + dict_log = process_log(full_path) + origin_df = origin_df.append(dict_log, ignore_index=True) + + raw_df = origin_df.sort_values(by='model_name') + raw_df.sort_values(by=["model_name", "batch_size"], inplace=True) + raw_df.to_excel(args.output_name) + + if args.analysis_trt: + trt_df = trt_perf_analysis(raw_df) + trt_df.to_excel("trt_analysis_{}".format(args.output_name)) + + if args.analysis_mkl: + mkl_df, thread_df = mkl_perf_analysis(raw_df) + mkl_df.to_excel("mkl_enable_analysis_{}".format(args.output_name)) + thread_df.to_excel("mkl_threads_analysis_{}".format(args.output_name)) + + +if __name__ == "__main__": + main() diff --git a/deploy/cpp/CMakeLists.txt b/deploy/cpp/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..34f8808d53e085c43048c4955a5715d663e4291e --- /dev/null +++ b/deploy/cpp/CMakeLists.txt @@ -0,0 +1,264 @@ +cmake_minimum_required(VERSION 3.0) +project(PaddleObjectDetector CXX C) + +option(WITH_MKL "Compile demo with MKL/OpenBlas support,defaultuseMKL." ON) +option(WITH_GPU "Compile demo with GPU/CPU, default use CPU." ON) +option(WITH_TENSORRT "Compile demo with TensorRT." OFF) + +option(WITH_KEYPOINT "Whether to Compile KeyPoint detector" OFF) +option(WITH_MOT "Whether to Compile MOT detector" OFF) + +SET(PADDLE_DIR "" CACHE PATH "Location of libraries") +SET(PADDLE_LIB_NAME "" CACHE STRING "libpaddle_inference") +SET(OPENCV_DIR "" CACHE PATH "Location of libraries") +SET(CUDA_LIB "" CACHE PATH "Location of libraries") +SET(CUDNN_LIB "" CACHE PATH "Location of libraries") +SET(TENSORRT_INC_DIR "" CACHE PATH "Compile demo with TensorRT") +SET(TENSORRT_LIB_DIR "" CACHE PATH "Compile demo with TensorRT") + +include(cmake/yaml-cpp.cmake) + +include_directories("${CMAKE_SOURCE_DIR}/") +include_directories("${CMAKE_CURRENT_BINARY_DIR}/ext/yaml-cpp/src/ext-yaml-cpp/include") +link_directories("${CMAKE_CURRENT_BINARY_DIR}/ext/yaml-cpp/lib") + +if (WITH_KEYPOINT) + set(SRCS src/main_keypoint.cc src/preprocess_op.cc src/object_detector.cc src/picodet_postprocess.cc src/utils.cc src/keypoint_detector.cc src/keypoint_postprocess.cc) +elseif (WITH_MOT) + set(SRCS src/main_jde.cc src/preprocess_op.cc src/object_detector.cc src/jde_detector.cc src/tracker.cc src/trajectory.cc src/lapjv.cpp src/picodet_postprocess.cc src/utils.cc) +else () + set(SRCS src/main.cc src/preprocess_op.cc src/object_detector.cc src/picodet_postprocess.cc src/utils.cc) +endif() + +macro(safe_set_static_flag) + foreach(flag_var + CMAKE_CXX_FLAGS CMAKE_CXX_FLAGS_DEBUG CMAKE_CXX_FLAGS_RELEASE + CMAKE_CXX_FLAGS_MINSIZEREL CMAKE_CXX_FLAGS_RELWITHDEBINFO) + if(${flag_var} MATCHES "/MD") + string(REGEX REPLACE "/MD" "/MT" ${flag_var} "${${flag_var}}") + endif(${flag_var} MATCHES "/MD") + endforeach(flag_var) +endmacro() + +if (WITH_MKL) + ADD_DEFINITIONS(-DUSE_MKL) +endif() + +if (NOT DEFINED PADDLE_DIR OR ${PADDLE_DIR} STREQUAL "") + message(FATAL_ERROR "please set PADDLE_DIR with -DPADDLE_DIR=/path/paddle_influence_dir") +endif() +message("PADDLE_DIR IS:" ${PADDLE_DIR}) + +if (NOT DEFINED OPENCV_DIR OR ${OPENCV_DIR} STREQUAL "") + message(FATAL_ERROR "please set OPENCV_DIR with -DOPENCV_DIR=/path/opencv") +endif() + +include_directories("${CMAKE_SOURCE_DIR}/") +include_directories("${PADDLE_DIR}/") +include_directories("${PADDLE_DIR}/third_party/install/protobuf/include") +include_directories("${PADDLE_DIR}/third_party/install/glog/include") +include_directories("${PADDLE_DIR}/third_party/install/gflags/include") +include_directories("${PADDLE_DIR}/third_party/install/xxhash/include") +if (EXISTS "${PADDLE_DIR}/third_party/install/snappy/include") + include_directories("${PADDLE_DIR}/third_party/install/snappy/include") +endif() +if(EXISTS "${PADDLE_DIR}/third_party/install/snappystream/include") + include_directories("${PADDLE_DIR}/third_party/install/snappystream/include") +endif() +include_directories("${PADDLE_DIR}/third_party/boost") +include_directories("${PADDLE_DIR}/third_party/eigen3") + +if (EXISTS "${PADDLE_DIR}/third_party/install/snappy/lib") + link_directories("${PADDLE_DIR}/third_party/install/snappy/lib") +endif() +if(EXISTS "${PADDLE_DIR}/third_party/install/snappystream/lib") + link_directories("${PADDLE_DIR}/third_party/install/snappystream/lib") +endif() + +link_directories("${PADDLE_DIR}/third_party/install/protobuf/lib") +link_directories("${PADDLE_DIR}/third_party/install/glog/lib") +link_directories("${PADDLE_DIR}/third_party/install/gflags/lib") +link_directories("${PADDLE_DIR}/third_party/install/xxhash/lib") +link_directories("${PADDLE_DIR}/third_party/install/paddle2onnx/lib") +link_directories("${PADDLE_DIR}/third_party/install/onnxruntime/lib") +link_directories("${PADDLE_DIR}/paddle/lib/") +link_directories("${CMAKE_CURRENT_BINARY_DIR}") + + + +if (WIN32) + include_directories("${PADDLE_DIR}/paddle/fluid/inference") + include_directories("${PADDLE_DIR}/paddle/include") + link_directories("${PADDLE_DIR}/paddle/fluid/inference") + find_package(OpenCV REQUIRED PATHS ${OPENCV_DIR}/build/ NO_DEFAULT_PATH) + +else () + find_package(OpenCV REQUIRED PATHS ${OPENCV_DIR}/share/OpenCV NO_DEFAULT_PATH) + include_directories("${PADDLE_DIR}/paddle/include") + link_directories("${PADDLE_DIR}/paddle/lib") +endif () +include_directories(${OpenCV_INCLUDE_DIRS}) + +if (WIN32) + add_definitions("/DGOOGLE_GLOG_DLL_DECL=") + set(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /bigobj /MTd") + set(CMAKE_C_FLAGS_RELEASE "${CMAKE_C_FLAGS_RELEASE} /bigobj /MT") + set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /bigobj /MTd") + set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /bigobj /MT") +else() + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g -o2 -fopenmp -std=c++11") + set(CMAKE_STATIC_LIBRARY_PREFIX "") +endif() + +# TODO let users define cuda lib path +if (WITH_GPU) + if (NOT DEFINED CUDA_LIB OR ${CUDA_LIB} STREQUAL "") + message(FATAL_ERROR "please set CUDA_LIB with -DCUDA_LIB=/path/cuda-8.0/lib64") + endif() + if (NOT WIN32) + if (NOT DEFINED CUDNN_LIB) + message(FATAL_ERROR "please set CUDNN_LIB with -DCUDNN_LIB=/path/cudnn_v7.4/cuda/lib64") + endif() + endif(NOT WIN32) +endif() + + +if (NOT WIN32) + if (WITH_TENSORRT AND WITH_GPU) + include_directories("${TENSORRT_INC_DIR}/") + link_directories("${TENSORRT_LIB_DIR}/") + endif() +endif(NOT WIN32) + +if (NOT WIN32) + set(NGRAPH_PATH "${PADDLE_DIR}/third_party/install/ngraph") + if(EXISTS ${NGRAPH_PATH}) + include(GNUInstallDirs) + include_directories("${NGRAPH_PATH}/include") + link_directories("${NGRAPH_PATH}/${CMAKE_INSTALL_LIBDIR}") + set(NGRAPH_LIB ${NGRAPH_PATH}/${CMAKE_INSTALL_LIBDIR}/libngraph${CMAKE_SHARED_LIBRARY_SUFFIX}) + endif() +endif() + +if(WITH_MKL) + include_directories("${PADDLE_DIR}/third_party/install/mklml/include") + if (WIN32) + set(MATH_LIB ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.lib + ${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5md.lib) + else () + set(MATH_LIB ${PADDLE_DIR}/third_party/install/mklml/lib/libmklml_intel${CMAKE_SHARED_LIBRARY_SUFFIX} + ${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5${CMAKE_SHARED_LIBRARY_SUFFIX}) + execute_process(COMMAND cp -r ${PADDLE_DIR}/third_party/install/mklml/lib/libmklml_intel${CMAKE_SHARED_LIBRARY_SUFFIX} /usr/lib) + endif () + set(MKLDNN_PATH "${PADDLE_DIR}/third_party/install/mkldnn") + if(EXISTS ${MKLDNN_PATH}) + include_directories("${MKLDNN_PATH}/include") + if (WIN32) + set(MKLDNN_LIB ${MKLDNN_PATH}/lib/mkldnn.lib) + else () + set(MKLDNN_LIB ${MKLDNN_PATH}/lib/libmkldnn.so.0) + endif () + endif() +else() + set(MATH_LIB ${PADDLE_DIR}/third_party/install/openblas/lib/libopenblas${CMAKE_STATIC_LIBRARY_SUFFIX}) +endif() + + +if (WIN32) + if(EXISTS "${PADDLE_DIR}/paddle/fluid/inference/${PADDLE_LIB_NAME}${CMAKE_STATIC_LIBRARY_SUFFIX}") + set(DEPS + ${PADDLE_DIR}/paddle/fluid/inference/${PADDLE_LIB_NAME}${CMAKE_STATIC_LIBRARY_SUFFIX}) + else() + set(DEPS + ${PADDLE_DIR}/paddle/lib/${PADDLE_LIB_NAME}${CMAKE_STATIC_LIBRARY_SUFFIX}) + endif() +endif() + + +if (WIN32) + set(DEPS ${PADDLE_DIR}/paddle/lib/${PADDLE_LIB_NAME}${CMAKE_STATIC_LIBRARY_SUFFIX}) +else() + set(DEPS ${PADDLE_DIR}/paddle/lib/${PADDLE_LIB_NAME}${CMAKE_SHARED_LIBRARY_SUFFIX}) +endif() + +message("PADDLE_LIB_NAME:" ${PADDLE_LIB_NAME}) +message("DEPS:" $DEPS) + +if (NOT WIN32) + set(DEPS ${DEPS} + ${MATH_LIB} ${MKLDNN_LIB} + glog gflags protobuf z xxhash yaml-cpp + ) + if(EXISTS "${PADDLE_DIR}/third_party/install/snappystream/lib") + set(DEPS ${DEPS} snappystream) + endif() + if (EXISTS "${PADDLE_DIR}/third_party/install/snappy/lib") + set(DEPS ${DEPS} snappy) + endif() +else() + set(DEPS ${DEPS} + ${MATH_LIB} ${MKLDNN_LIB} + glog gflags_static libprotobuf xxhash libyaml-cppmt) + set(DEPS ${DEPS} libcmt shlwapi) + if (EXISTS "${PADDLE_DIR}/third_party/install/snappy/lib") + set(DEPS ${DEPS} snappy) + endif() + if(EXISTS "${PADDLE_DIR}/third_party/install/snappystream/lib") + set(DEPS ${DEPS} snappystream) + endif() +endif(NOT WIN32) + +if(WITH_GPU) + if(NOT WIN32) + if (WITH_TENSORRT) + set(DEPS ${DEPS} ${TENSORRT_LIB_DIR}/libnvinfer${CMAKE_SHARED_LIBRARY_SUFFIX}) + set(DEPS ${DEPS} ${TENSORRT_LIB_DIR}/libnvinfer_plugin${CMAKE_SHARED_LIBRARY_SUFFIX}) + endif() + set(DEPS ${DEPS} ${CUDA_LIB}/libcudart${CMAKE_SHARED_LIBRARY_SUFFIX}) + set(DEPS ${DEPS} ${CUDNN_LIB}/libcudnn${CMAKE_SHARED_LIBRARY_SUFFIX}) + else() + set(DEPS ${DEPS} ${CUDA_LIB}/cudart${CMAKE_STATIC_LIBRARY_SUFFIX} ) + set(DEPS ${DEPS} ${CUDA_LIB}/cublas${CMAKE_STATIC_LIBRARY_SUFFIX} ) + set(DEPS ${DEPS} ${CUDNN_LIB}/cudnn${CMAKE_STATIC_LIBRARY_SUFFIX}) + endif() +endif() + +if (NOT WIN32) + set(EXTERNAL_LIB "-ldl -lrt -lgomp -lz -lm -lpthread") + set(DEPS ${DEPS} ${EXTERNAL_LIB}) +endif() + +set(DEPS ${DEPS} ${OpenCV_LIBS}) +add_executable(main ${SRCS}) +ADD_DEPENDENCIES(main ext-yaml-cpp) +message("DEPS:" $DEPS) +target_link_libraries(main ${DEPS}) + +if (WIN32 AND WITH_MKL) + add_custom_command(TARGET main POST_BUILD + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.dll ./mklml.dll + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5md.dll ./libiomp5md.dll + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mkldnn/lib/mkldnn.dll ./mkldnn.dll + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.dll ./release/mklml.dll + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5md.dll ./release/libiomp5md.dll + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mkldnn/lib/mkldnn.dll ./release/mkldnn.dll + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/paddle/lib/${PADDLE_LIB_NAME}.dll ./release/${PADDLE_LIB_NAME}.dll + ) +endif() + +if (WIN32 AND NOT WITH_MKL) + add_custom_command(TARGET main POST_BUILD + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/openblas/lib/openblas.dll ./openblas.dll + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/openblas/lib/openblas.dll ./release/openblas.dll + ) +endif() + +if (WIN32) + add_custom_command(TARGET main POST_BUILD + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/onnxruntime/lib/onnxruntime.dll ./onnxruntime.dll + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/paddle2onnx/lib/paddle2onnx.dll ./paddle2onnx.dll + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/onnxruntime/lib/onnxruntime.dll ./release/onnxruntime.dll + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/paddle2onnx/lib/paddle2onnx.dll ./release/paddle2onnx.dll + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/paddle/lib/${PADDLE_LIB_NAME}.dll ./release/${PADDLE_LIB_NAME}.dll + ) +endif() diff --git a/deploy/cpp/README.md b/deploy/cpp/README.md new file mode 100644 index 0000000000000000000000000000000000000000..ffa5e251e7913b4af30fa6abe9912c9434af996f --- /dev/null +++ b/deploy/cpp/README.md @@ -0,0 +1,54 @@ +# C++端预测部署 + + + +## 各环境编译部署教程 +- [Linux 编译部署](docs/linux_build.md) +- [Windows编译部署(使用Visual Studio 2019)](docs/windows_vs2019_build.md) +- [NV Jetson编译部署](docs/Jetson_build.md) + + +## C++部署总览 +[1.说明](#1说明) + +[2.主要目录和文件](#2主要目录和文件) + + +### 1.说明 + +本目录为用户提供一个跨平台的`C++`部署方案，让用户通过`PaddleDetection`训练的模型导出后，即可基于本项目快速运行，也可以快速集成代码结合到自己的项目实际应用中去。 + +主要设计的目标包括以下四点： +- 跨平台，支持在 `Windows` 和 `Linux` 完成编译、二次开发集成和部署运行 +- 可扩展性，支持用户针对新模型开发自己特殊的数据预处理等逻辑 +- 高性能，除了`PaddlePaddle`自身带来的性能优势，我们还针对图像检测的特点对关键步骤进行了性能优化 +- 支持各种不同检测模型结构，包括`Yolov3`/`Faster_RCNN`/`SSD`等 + +### 2.主要目录和文件 + +```bash +deploy/cpp +| +├── src +│ ├── main.cc # 集成代码示例, 程序入口 +│ ├── object_detector.cc # 模型加载和预测主要逻辑封装类实现 +│ └── preprocess_op.cc # 预处理相关主要逻辑封装实现 +| +├── include +│ ├── config_parser.h # 导出模型配置yaml文件解析 +│ ├── object_detector.h # 模型加载和预测主要逻辑封装类 +│ └── preprocess_op.h # 预处理相关主要逻辑类封装 +| +├── docs +│ ├── linux_build.md # Linux 编译指南 +│ └── windows_vs2019_build.md # Windows VS2019编译指南 +│ +├── build.sh # 编译命令脚本 +│ +├── CMakeList.txt # cmake编译入口文件 +| +├── CMakeSettings.json # Visual Studio 2019 CMake项目编译设置 +│ +└── cmake # 依赖的外部项目cmake（目前仅有yaml-cpp） + +``` diff --git a/deploy/cpp/cmake/yaml-cpp.cmake b/deploy/cpp/cmake/yaml-cpp.cmake new file mode 100644 index 0000000000000000000000000000000000000000..7bc7f34d476d69d57336940bcf6c8c55311b8112 --- /dev/null +++ b/deploy/cpp/cmake/yaml-cpp.cmake @@ -0,0 +1,30 @@ + +find_package(Git REQUIRED) + +include(ExternalProject) + +message("${CMAKE_BUILD_TYPE}") + +ExternalProject_Add( + ext-yaml-cpp + URL https://bj.bcebos.com/paddlex/deploy/deps/yaml-cpp.zip + URL_MD5 9542d6de397d1fbd649ed468cb5850e6 + CMAKE_ARGS + -DYAML_CPP_BUILD_TESTS=OFF + -DYAML_CPP_BUILD_TOOLS=OFF + -DYAML_CPP_INSTALL=OFF + -DYAML_CPP_BUILD_CONTRIB=OFF + -DMSVC_SHARED_RT=OFF + -DBUILD_SHARED_LIBS=OFF + -DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE} + -DCMAKE_CXX_FLAGS=${CMAKE_CXX_FLAGS} + -DCMAKE_CXX_FLAGS_DEBUG=${CMAKE_CXX_FLAGS_DEBUG} + -DCMAKE_CXX_FLAGS_RELEASE=${CMAKE_CXX_FLAGS_RELEASE} + -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=${CMAKE_BINARY_DIR}/ext/yaml-cpp/lib + -DCMAKE_ARCHIVE_OUTPUT_DIRECTORY=${CMAKE_BINARY_DIR}/ext/yaml-cpp/lib + PREFIX "${CMAKE_BINARY_DIR}/ext/yaml-cpp" + # Disable install step + INSTALL_COMMAND "" + LOG_DOWNLOAD ON + LOG_BUILD 1 +) diff --git a/deploy/cpp/docs/Jetson_build.md b/deploy/cpp/docs/Jetson_build.md new file mode 100644 index 0000000000000000000000000000000000000000..ea9699a438ed3977e118b155a01b533d83bb12f4 --- /dev/null +++ b/deploy/cpp/docs/Jetson_build.md @@ -0,0 +1,210 @@ +# Jetson平台编译指南 + +## 说明 +`NVIDIA Jetson`设备是具有`NVIDIA GPU`的嵌入式设备，可以将目标检测算法部署到该设备上。本文档是在`Jetson`硬件上部署`PaddleDetection`模型的教程。 + +本文档以`Jetson TX2`硬件、`JetPack 4.3`版本为例进行说明。 + +`Jetson`平台的开发指南请参考[NVIDIA Jetson Linux Developer Guide](https://docs.nvidia.com/jetson/l4t/index.html). + +## Jetson环境搭建 +`Jetson`系统软件安装，请参考[NVIDIA Jetson Linux Developer Guide](https://docs.nvidia.com/jetson/l4t/index.html). + +* (1) 查看硬件系统的l4t的版本号 +``` +cat /etc/nv_tegra_release +``` +* (2) 根据硬件，选择硬件可安装的`JetPack`版本，硬件和`JetPack`版本对应关系请参考[jetpack-archive](https://developer.nvidia.com/embedded/jetpack-archive). + +* (3) 下载`JetPack`，请参考[NVIDIA Jetson Linux Developer Guide](https://docs.nvidia.com/jetson/l4t/index.html) 中的`Preparing a Jetson Developer Kit for Use`章节内容进行刷写系统镜像。 + +**注意**: 请在[jetpack-archive](https://developer.nvidia.com/embedded/jetpack-archive) 根据硬件选择适配的`JetPack`版本进行刷机。 + +## 下载或编译`Paddle`预测库 +本文档使用`Paddle`在`JetPack4.3`上预先编译好的预测库，请根据硬件在[安装与编译 Linux 预测库](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/05_inference_deployment/inference/build_and_install_lib_cn.html) 中选择对应版本的`Paddle`预测库。 + +这里选择[nv_jetson_cuda10_cudnn7.6_trt6(jetpack4.3)](https://paddle-inference-lib.bj.bcebos.com/2.0.0-nv-jetson-jetpack4.3-all/paddle_inference.tgz), `Paddle`版本`2.0.0-rc0`,`CUDA`版本`10.0`,`CUDNN`版本`7.6`，`TensorRT`版本`6`。 + +若需要自己在`Jetson`平台上自定义编译`Paddle`库，请参考文档[安装与编译 Linux 预测库](https://www.paddlepaddle.org.cn/documentation/docs/zh/advanced_guide/inference_deployment/inference/build_and_install_lib_cn.html) 的`NVIDIA Jetson嵌入式硬件预测库源码编译`部分内容。 + +### Step1: 下载代码 + + `git clone https://github.com/PaddlePaddle/PaddleDetection.git` + +**说明**：其中`C++`预测代码在`/root/projects/PaddleDetection/deploy/cpp` 目录，该目录不依赖任何`PaddleDetection`下其他目录。 + + +### Step2: 下载PaddlePaddle C++ 预测库 paddle_inference + +解压下载的[nv_jetson_cuda10_cudnn7.6_trt6(jetpack4.3)](https://paddle-inference-lib.bj.bcebos.com/2.0.1-nv-jetson-jetpack4.3-all/paddle_inference.tgz) 。 + +下载并解压后`/root/projects/paddle_inference`目录包含内容为： +``` +paddle_inference +├── paddle # paddle核心库和头文件 +| +├── third_party # 第三方依赖库和头文件 +| +└── version.txt # 版本和编译信息 +``` + +**注意:** 预编译库`nv-jetson-cuda10-cudnn7.6-trt6`使用的`GCC`版本是`7.5.0`，其他都是使用`GCC 4.8.5`编译的。使用高版本的GCC可能存在`ABI`兼容性问题，建议降级或[自行编译预测库](https://www.paddlepaddle.org.cn/documentation/docs/zh/advanced_guide/inference_deployment/inference/build_and_install_lib_cn.html)。 + + +### Step4: 编译 + +编译`cmake`的命令在`scripts/build.sh`中，请根据实际情况修改主要参数，其主要内容说明如下： + +注意，`TX2`平台的`CUDA`、`CUDNN`需要通过`JetPack`安装。 + +``` +# 是否使用GPU(即是否使用 CUDA) +WITH_GPU=ON + +# 是否使用MKL or openblas，TX2需要设置为OFF +WITH_MKL=OFF + +# 是否集成 TensorRT(仅WITH_GPU=ON 有效) +WITH_TENSORRT=ON + +# TensorRT 的include路径 +TENSORRT_INC_DIR=/usr/include/aarch64-linux-gnu + +# TensorRT 的lib路径 +TENSORRT_LIB_DIR=/usr/lib/aarch64-linux-gnu + +# Paddle 预测库路径 +PADDLE_DIR=/path/to/paddle_inference/ + +# Paddle 预测库名称 +PADDLE_LIB_NAME=paddle_inference + +# Paddle 的预测库是否使用静态库来编译 +# 使用TensorRT时，Paddle的预测库通常为动态库 +WITH_STATIC_LIB=OFF + +# CUDA 的 lib 路径 +CUDA_LIB=/usr/local/cuda-10.0/lib64 + +# CUDNN 的 lib 路径 +CUDNN_LIB=/usr/lib/aarch64-linux-gnu + +# 是否开启关键点模型预测功能 +WITH_KEYPOINT=ON + +# OPENCV_DIR 的路径 +# linux平台请下载：https://bj.bcebos.com/paddleseg/deploy/opencv3.4.6gcc4.8ffmpeg.tar.gz2，并解压到deps文件夹下 +# TX2平台请下载：https://paddlemodels.bj.bcebos.com/TX2_JetPack4.3_opencv_3.4.10_gcc7.5.0.zip，并解压到deps文件夹下 +OPENCV_DIR=/path/to/opencv + +# 请检查以上各个路径是否正确 + +# 以下无需改动 +cmake .. \ + -DWITH_GPU=${WITH_GPU} \ + -DWITH_MKL=OFF \ + -DWITH_TENSORRT=${WITH_TENSORRT} \ + -DTENSORRT_DIR=${TENSORRT_DIR} \ + -DPADDLE_DIR=${PADDLE_DIR} \ + -DWITH_STATIC_LIB=${WITH_STATIC_LIB} \ + -DCUDA_LIB=${CUDA_LIB} \ + -DCUDNN_LIB=${CUDNN_LIB} \ + -DOPENCV_DIR=${OPENCV_DIR} \ + -DPADDLE_LIB_NAME={PADDLE_LIB_NAME} \ + -DWITH_KEYPOINT=${WITH_KEYPOINT} +make +``` + +例如设置如下： +``` +# 是否使用GPU(即是否使用 CUDA) +WITH_GPU=ON + +# 是否使用MKL or openblas +WITH_MKL=OFF + +# 是否集成 TensorRT(仅WITH_GPU=ON 有效) +WITH_TENSORRT=OFF + +# TensorRT 的include路径 +TENSORRT_INC_DIR=/usr/include/aarch64-linux-gnu + +# TensorRT 的lib路径 +TENSORRT_LIB_DIR=/usr/lib/aarch64-linux-gnu + +# Paddle 预测库路径 +PADDLE_DIR=/home/nvidia/PaddleDetection_infer/paddle_inference/ + +# Paddle 预测库名称 +PADDLE_LIB_NAME=paddle_inference + +# Paddle 的预测库是否使用静态库来编译 +# 使用TensorRT时，Paddle的预测库通常为动态库 +WITH_STATIC_LIB=OFF + +# CUDA 的 lib 路径 +CUDA_LIB=/usr/local/cuda-10.0/lib64 + +# CUDNN 的 lib 路径 +CUDNN_LIB=/usr/lib/aarch64-linux-gnu/ + +# 是否开启关键点模型预测功能 +WITH_KEYPOINT=ON +``` + +修改脚本设置好主要参数后，执行`build`脚本： + ```shell + sh ./scripts/build.sh + ``` + +### Step5: 预测及可视化 +编译成功后，预测入口程序为`build/main`其主要命令参数说明如下： +| 参数 | 说明 | +| ---- | ---- | +| --model_dir | 导出的检测预测模型所在路径 | +| --model_dir_keypoint | Option | 导出的关键点预测模型所在路径 | +| --image_file | 要预测的图片文件路径 | +| --image_dir | 要预测的图片文件夹路径 | +| --video_file | 要预测的视频文件路径 | +| --camera_id | Option | 用来预测的摄像头ID，默认为-1（表示不使用摄像头预测）| +| --device | 运行时的设备，可选择`CPU/GPU/XPU`，默认为`CPU`| +| --gpu_id | 指定进行推理的GPU device id(默认值为0)| +| --run_mode | 使用GPU时，默认为paddle, 可选（paddle/trt_fp32/trt_fp16/trt_int8）| +| --batch_size | 检测模型预测时的batch size，在指定`image_dir`时有效 | +| --batch_size_keypoint | 关键点模型预测时的batch size，默认为8 | +| --run_benchmark | 是否重复预测来进行benchmark测速 ｜ +| --output_dir | 输出图片所在的文件夹, 默认为output ｜ +| --use_mkldnn | CPU预测中是否开启MKLDNN加速 | +| --cpu_threads | 设置cpu线程数，默认为1 | +| --use_dark | 关键点模型输出预测是否使用DarkPose后处理，默认为true | + +**注意**: +- 优先级顺序：`camera_id` > `video_file` > `image_dir` > `image_file`。 +- --run_benchmark如果设置为True，则需要安装依赖`pip install pynvml psutil GPUtil`。 + + +`样例一`： +```shell +#不使用`GPU`测试图片 `/root/projects/images/test.jpeg` +./main --model_dir=/root/projects/models/yolov3_darknet --image_file=/root/projects/images/test.jpeg +``` + +图片文件`可视化预测结果`会保存在当前目录下`output.jpg`文件中。 + + +`样例二`: +```shell +#使用 `GPU`预测视频`/root/projects/videos/test.mp4` +./main --model_dir=/root/projects/models/yolov3_darknet --video_path=/root/projects/images/test.mp4 --device=GPU +``` +视频文件目前支持`.mp4`格式的预测，`可视化预测结果`会保存在当前目录下`output.mp4`文件中。 + +`样例三`： +```shell +#使用关键点模型与检测模型联合预测，使用 `GPU`预测 +#检测模型检测到的人送入关键点模型进行关键点预测 +./main --model_dir=/root/projects/models/yolov3_darknet --model_dir_keypoint=/root/projects/models/hrnet_w32_256x192 --image_file=/root/projects/images/test.jpeg --device=GPU +``` + +## 性能测试 +benchmark请查看[BENCHMARK_INFER](../../BENCHMARK_INFER.md) diff --git a/deploy/cpp/docs/linux_build.md b/deploy/cpp/docs/linux_build.md new file mode 100644 index 0000000000000000000000000000000000000000..ee28e73ee56db3ec46a1674a6af0cb3af1012b3e --- /dev/null +++ b/deploy/cpp/docs/linux_build.md @@ -0,0 +1,149 @@ +# Linux平台编译指南 + +## 说明 +本文档在 `Linux`平台使用`GCC 8.2`测试过，如果需要使用其他G++版本编译使用，则需要重新编译Paddle预测库，请参考: [从源码编译Paddle预测库](https://paddleinference.paddlepaddle.org.cn/user_guides/source_compile.html)。本文档使用的预置的opencv库是在ubuntu 16.04上用gcc8.2编译的，如果需要在gcc8.2以外的环境编译，那么需自行编译opencv库。 + +## 前置条件 +* G++ 8.2 +* CUDA 9.0 / CUDA 10.1, cudnn 7+ （仅在使用GPU版本的预测库时需要） +* CMake 3.0+ + +请确保系统已经安装好上述基本软件，**下面所有示例以工作目录为 `/root/projects/`演示**。 + +### Step1: 下载代码 + + `git clone https://github.com/PaddlePaddle/PaddleDetection.git` + +**说明**：其中`C++`预测代码在`/root/projects/PaddleDetection/deploy/cpp` 目录，该目录不依赖任何`PaddleDetection`下其他目录。 + + +### Step2: 下载PaddlePaddle C++ 预测库 paddle_inference + +PaddlePaddle C++ 预测库针对不同的`CPU`和`CUDA`版本提供了不同的预编译版本，请根据实际情况下载: [C++预测库下载列表](https://paddleinference.paddlepaddle.org.cn/user_guides/download_lib.html) + + +下载并解压后`/root/projects/paddle_inference`目录包含内容为： +``` +paddle_inference +├── paddle # paddle核心库和头文件 +| +├── third_party # 第三方依赖库和头文件 +| +└── version.txt # 版本和编译信息 +``` + +**注意:** 预编译版本除`nv-jetson-cuda10-cudnn7.5-trt5` 以外其它包都是基于`GCC 4.8.5`编译，使用高版本`GCC`可能存在 `ABI`兼容性问题，建议降级或[自行编译预测库](https://www.paddlepaddle.org.cn/documentation/docs/zh/advanced_guide/inference_deployment/inference/build_and_install_lib_cn.html)。 + + +### Step3: 编译 + +编译`cmake`的命令在`scripts/build.sh`中，请根据实际情况修改主要参数，其主要内容说明如下： + +``` +# 是否使用GPU(即是否使用 CUDA) +WITH_GPU=OFF + +# 使用MKL or openblas +WITH_MKL=ON + +# 是否集成 TensorRT(仅WITH_GPU=ON 有效) +WITH_TENSORRT=OFF + +# TensorRT 的include路径 +TENSORRT_LIB_DIR=/path/to/TensorRT/include + +# TensorRT 的lib路径 +TENSORRT_LIB_DIR=/path/to/TensorRT/lib + +# Paddle 预测库路径 +PADDLE_DIR=/path/to/paddle_inference + +# Paddle 预测库名称 +PADDLE_LIB_NAME=paddle_inference + +# CUDA 的 lib 路径 +CUDA_LIB=/path/to/cuda/lib + +# CUDNN 的 lib 路径 +CUDNN_LIB=/path/to/cudnn/lib + +# 是否开启关键点模型预测功能 +WITH_KEYPOINT=ON + +# 请检查以上各个路径是否正确 + +# 以下无需改动 +cmake .. \ + -DWITH_GPU=${WITH_GPU} \ + -DWITH_MKL=${WITH_MKL} \ + -DWITH_TENSORRT=${WITH_TENSORRT} \ + -DTENSORRT_LIB_DIR=${TENSORRT_LIB_DIR} \ + -DTENSORRT_INC_DIR=${TENSORRT_INC_DIR} \ + -DPADDLE_DIR=${PADDLE_DIR} \ + -DCUDA_LIB=${CUDA_LIB} \ + -DCUDNN_LIB=${CUDNN_LIB} \ + -DOPENCV_DIR=${OPENCV_DIR} \ + -DPADDLE_LIB_NAME=${PADDLE_LIB_NAME} \ + -DWITH_KEYPOINT=${WITH_KEYPOINT} +make + +``` + +修改脚本设置好主要参数后，执行`build`脚本： + ```shell + sh ./scripts/build.sh + ``` + +**注意**: OPENCV依赖OPENBLAS，Ubuntu用户需确认系统是否已存在`libopenblas.so`。如未安装，可执行apt-get install libopenblas-dev进行安装。 + +### Step4: 预测及可视化 +编译成功后，预测入口程序为`build/main`其主要命令参数说明如下： +| 参数 | 说明 | +| ---- | ---- | +| --model_dir | 导出的检测预测模型所在路径 | +| --model_dir_keypoint | Option | 导出的关键点预测模型所在路径 | +| --image_file | 要预测的图片文件路径 | +| --image_dir | 要预测的图片文件夹路径 | +| --video_file | 要预测的视频文件路径 | +| --camera_id | Option | 用来预测的摄像头ID，默认为-1（表示不使用摄像头预测）| +| --device | 运行时的设备，可选择`CPU/GPU/XPU`，默认为`CPU`| +| --gpu_id | 指定进行推理的GPU device id(默认值为0)| +| --run_mode | 使用GPU时，默认为paddle, 可选（paddle/trt_fp32/trt_fp16/trt_int8）| +| --batch_size | 检测模型预测时的batch size，在指定`image_dir`时有效 | +| --batch_size_keypoint | 关键点模型预测时的batch size，默认为8 | +| --run_benchmark | 是否重复预测来进行benchmark测速 ｜ +| --output_dir | 输出图片所在的文件夹, 默认为output ｜ +| --use_mkldnn | CPU预测中是否开启MKLDNN加速 | +| --cpu_threads | 设置cpu线程数，默认为1 | +| --use_dark | 关键点模型输出预测是否使用DarkPose后处理，默认为true | + +**注意**: +- 优先级顺序：`camera_id` > `video_file` > `image_dir` > `image_file`。 +- --run_benchmark如果设置为True，则需要安装依赖`pip install pynvml psutil GPUtil`。 + +`样例一`： +```shell +#不使用`GPU`测试图片 `/root/projects/images/test.jpeg` +./build/main --model_dir=/root/projects/models/yolov3_darknet --image_file=/root/projects/images/test.jpeg +``` + +图片文件`可视化预测结果`会保存在当前目录下`output.jpg`文件中。 + + +`样例二`: +```shell +#使用 `GPU`预测视频`/root/projects/videos/test.mp4` +./build/main --model_dir=/root/projects/models/yolov3_darknet --video_file=/root/projects/images/test.mp4 --device=GPU +``` +视频文件目前支持`.mp4`格式的预测，`可视化预测结果`会保存在当前目录下`output.mp4`文件中。 + + +`样例三`： +```shell +#使用关键点模型与检测模型联合预测，使用 `GPU`预测 +#检测模型检测到的人送入关键点模型进行关键点预测 +./build/main --model_dir=/root/projects/models/yolov3_darknet --model_dir_keypoint=/root/projects/models/hrnet_w32_256x192 --image_file=/root/projects/images/test.jpeg --device=GPU +``` + +## 性能测试 +benchmark请查看[BENCHMARK_INFER](../../BENCHMARK_INFER.md) diff --git a/deploy/cpp/docs/windows_vs2019_build.md b/deploy/cpp/docs/windows_vs2019_build.md new file mode 100644 index 0000000000000000000000000000000000000000..1a23cabc7bf640ed548942012354013f500d6be2 --- /dev/null +++ b/deploy/cpp/docs/windows_vs2019_build.md @@ -0,0 +1,158 @@ +# Visual Studio 2019 Community CMake 编译指南 + +Windows 平台下，我们使用`Visual Studio 2019 Community` 进行了测试。微软从`Visual Studio 2017`开始即支持直接管理`CMake`跨平台编译项目，但是直到`2019`才提供了稳定和完全的支持，所以如果你想使用CMake管理项目编译构建，我们推荐你使用`Visual Studio 2019`环境下构建。 + + +## 前置条件 +* Visual Studio 2019 (根据Paddle预测库所使用的VS版本选择，请参考 [Visual Studio 不同版本二进制兼容性](https://docs.microsoft.com/zh-cn/cpp/porting/binary-compat-2015-2017?view=vs-2019) ) +* CUDA 9.0 / CUDA 10.0，cudnn 7+ / TensorRT（仅在使用GPU版本的预测库时需要） +* CMake 3.0+ [CMake下载](https://cmake.org/download/) + +**特别注意：windows下预测库需要的TensorRT版本为：**。 + +| 预测库版本 | TensorRT版本 | +| ---- | ---- | +| cuda10.1_cudnn7.6_avx_mkl_trt6 | TensorRT-6.0.1.5 | +| cuda10.2_cudnn7.6_avx_mkl_trt7 | TensorRT-7.0.0.11 | +| cuda11.0_cudnn8.0_avx_mkl_trt7 | TensorRT-7.2.1.6 | + +请确保系统已经安装好上述基本软件，我们使用的是`VS2019`的社区版。 + +**下面所有示例以工作目录为 `D:\projects`演示**。 + +### Step1: 下载代码 + +下载源代码 +```shell +git clone https://github.com/PaddlePaddle/PaddleDetection.git +``` + +**说明**：其中`C++`预测代码在`PaddleDetection/deploy/cpp` 目录，该目录不依赖任何`PaddleDetection`下其他目录。 + + +### Step2: 下载PaddlePaddle C++ 预测库 paddle_inference + +PaddlePaddle C++ 预测库针对不同的`CPU`和`CUDA`版本提供了不同的预编译版本，请根据实际情况下载: [C++预测库下载列表](https://paddleinference.paddlepaddle.org.cn/user_guides/download_lib.html#windows) + +解压后`D:\projects\paddle_inference`目录包含内容为： +``` +paddle_inference +├── paddle # paddle核心库和头文件 +| +├── third_party # 第三方依赖库和头文件 +| +└── version.txt # 版本和编译信息 +``` + +### Step3: 安装配置OpenCV + +1. 在OpenCV官网下载适用于Windows平台的3.4.6版本， [下载地址](https://sourceforge.net/projects/opencvlibrary/files/3.4.6/opencv-3.4.6-vc14_vc15.exe/download) +2. 运行下载的可执行文件，将OpenCV解压至指定目录，如`D:\projects\opencv` +3. 配置环境变量，如下流程所示（如果使用全局绝对路径，可以不用设置环境变量） + - 我的电脑->属性->高级系统设置->环境变量 + - 在系统变量中找到Path（如没有，自行创建），并双击编辑 + - 新建，将opencv路径填入并保存，如`D:\projects\opencv\build\x64\vc14\bin` + +### Step4: 编译 + +1. 进入到`cpp`文件夹 +``` +cd D:\projects\PaddleDetection\deploy\cpp +``` + +2. 使用CMake生成项目文件 + +编译参数的含义说明如下（带`*`表示仅在使用**GPU版本**预测库时指定, 其中CUDA库版本尽量对齐，**使用9.0、10.0版本，不使用9.2、10.1等版本CUDA库**）： + +| 参数名 | 含义 | +| ---- | ---- | +| *CUDA_LIB | CUDA的库路径 | +| *CUDNN_LIB | CUDNN的库路径 | +| OPENCV_DIR | OpenCV的安装路径， | +| PADDLE_DIR | Paddle预测库的路径 | +| PADDLE_LIB_NAME | Paddle 预测库名称 | + +**注意：** + +1. 如果编译环境为CPU，需要下载`CPU`版预测库，请把`WITH_GPU`的勾去掉 +2. 如果使用的是`openblas`版本，请把`WITH_MKL`勾去掉 +3. 如无需使用关键点模型可以把`WITH_KEYPOINT`勾去掉 +4. Windows环境下，`PADDLE_LIB_NAME`需要设置为`paddle_inference` + +执行如下命令项目文件： +``` +cmake . -G "Visual Studio 16 2019" -A x64 -T host=x64 -DWITH_GPU=ON -DWITH_MKL=ON -DCMAKE_BUILD_TYPE=Release -DCUDA_LIB=path_to_cuda_lib -DCUDNN_LIB=path_to_cudnn_lib -DPADDLE_DIR=path_to_paddle_lib -DPADDLE_LIB_NAME=paddle_inference -DOPENCV_DIR=path_to_opencv -DWITH_KEYPOINT=ON +``` + +例如： +``` +cmake . -G "Visual Studio 16 2019" -A x64 -T host=x64 -DWITH_GPU=ON -DWITH_MKL=ON -DCMAKE_BUILD_TYPE=Release -DCUDA_LIB=D:\projects\packages\cuda10_0\lib\x64 -DCUDNN_LIB=D:\projects\packages\cuda10_0\lib\x64 -DPADDLE_DIR=D:\projects\packages\paddle_inference -DPADDLE_LIB_NAME=paddle_inference -DOPENCV_DIR=D:\projects\packages\opencv3_4_6 -DWITH_KEYPOINT=ON +``` + + + +3. 编译 +用`Visual Studio 16 2019`打开`cpp`文件夹下的`PaddleObjectDetector.sln`，将编译模式设置为`Release`，点击`生成`->`全部生成 + + +### Step5: 预测及可视化 + +上述`Visual Studio 2019`编译产出的可执行文件在`out\build\x64-Release`目录下，打开`cmd`，并切换到该目录： + +``` +cd D:\projects\PaddleDetection\deploy\cpp\out\build\x64-Release +``` +可执行文件`main`即为样例的预测程序，其主要的命令行参数如下： + +| 参数 | 说明 | +| ---- | ---- | +| --model_dir | 导出的检测预测模型所在路径 | +| --model_dir_keypoint | Option | 导出的关键点预测模型所在路径 | +| --image_file | 要预测的图片文件路径 | +| --image_dir | 要预测的图片文件夹路径 | +| --video_file | 要预测的视频文件路径 | +| --camera_id | Option | 用来预测的摄像头ID，默认为-1（表示不使用摄像头预测）| +| --device | 运行时的设备，可选择`CPU/GPU/XPU`，默认为`CPU`| +| --gpu_id | 指定进行推理的GPU device id(默认值为0)| +| --run_mode | 使用GPU时，默认为paddle, 可选（paddle/trt_fp32/trt_fp16/trt_int8）| +| --batch_size | 检测模型预测时的batch size，在指定`image_dir`时有效 | +| --batch_size_keypoint | 关键点模型预测时的batch size，默认为8 | +| --run_benchmark | 是否重复预测来进行benchmark测速 ｜ +| --output_dir | 输出图片所在的文件夹, 默认为output ｜ +| --use_mkldnn | CPU预测中是否开启MKLDNN加速 | +| --cpu_threads | 设置cpu线程数，默认为1 | +| --use_dark | 关键点模型输出预测是否使用DarkPose后处理，默认为true | + +**注意**： +（1）优先级顺序：`camera_id` > `video_file` > `image_dir` > `image_file`。 +（2）如果提示找不到`opencv_world346.dll`，把`D:\projects\packages\opencv3_4_6\build\x64\vc14\bin`文件夹下的`opencv_world346.dll`拷贝到`main.exe`文件夹下即可。 +（3）--run_benchmark如果设置为True，则需要安装依赖`pip install pynvml psutil GPUtil`。 + + +`样例一`： +```shell +#不使用`GPU`测试图片 `D:\\images\\test.jpeg` +.\main --model_dir=D:\\models\\yolov3_darknet --image_file=D:\\images\\test.jpeg +``` + +图片文件`可视化预测结果`会保存在当前目录下`output.jpg`文件中。 + + +`样例二`: +```shell +#使用`GPU`测试视频 `D:\\videos\\test.mp4` +.\main --model_dir=D:\\models\\yolov3_darknet --video_path=D:\\videos\\test.mp4 --device=GPU +``` + +视频文件目前支持`.mp4`格式的预测，`可视化预测结果`会保存在当前目录下`output.mp4`文件中。 + + +`样例三`： +```shell +#使用关键点模型与检测模型联合预测，使用 `GPU`预测 +#检测模型检测到的人送入关键点模型进行关键点预测 +.\main --model_dir=D:\\models\\yolov3_darknet --model_dir_keypoint=D:\\models\\hrnet_w32_256x192 --image_file=D:\\images\\test.jpeg --device=GPU +``` + +## 性能测试 +Benchmark请查看[BENCHMARK_INFER](../../BENCHMARK_INFER.md) diff --git a/deploy/cpp/include/config_parser.h b/deploy/cpp/include/config_parser.h new file mode 100644 index 0000000000000000000000000000000000000000..1f2e381c5284bb7ce16a6b06f858a32e83290f98 --- /dev/null +++ b/deploy/cpp/include/config_parser.h @@ -0,0 +1,142 @@ +// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#pragma once + +#include