snpe.md

# Build for SNPE

It is quite simple to support snpe backend: Client/Server mode.

this mode

1. Can split `model convert` and `inference` environments;

- Inference irrelevant matters are done on host
- We can get the real running results of gpu/npu instead of cpu simulation values

2. Can cover cost-sensitive device, armv7/risc-v/mips chips meet product requirements, but often have limited support for Python;

3. Can simplify mmdeploy installation steps. If you only want to convert snpe model and test, you don't need to compile the .whl package.

## 1. Run inference server

Download the prebuilt snpe inference server package, `adb push` it to the phone and execute.
Note that **the phone must have a qcom chip**.

```bash
$ wget https://media.githubusercontent.com/media/tpoisonooo/mmdeploy_snpe_testdata/main/snpe-inference-server-1.59.tar.gz
...
$ sudo apt install adb
$ adb push snpe-inference-server-1.59.tar.gz  /data/local/tmp/

# decompress and execute
$ adb shell
venus:/ $ cd /data/local/tmp
130|venus:/data/local/tmp $ tar xvf snpe-inference-server-1.59.tar.gz
...
130|venus:/data/local/tmp $ source export1.59.sh
130|venus:/data/local/tmp $ ./inference_server
...
  Server listening on [::]:60000
```

At this point the inference service should print all the ipv6 and ipv4 addresses of the device and listen on the port.

tips:

- If `adb devices` cannot find the device, may be:
  - Some cheap cables can only charge and cannot transmit data
  - or the "developer mode" of the phone is not turned on
- If you need to compile the binary by self, please refer to [NDK Cross Compiling snpe Inference Service](../appendix/cross_build_snpe_service.md)
- If a `segmentation fault` occurs when listening on a port, it may be because:
  - The port number is already occupied, use another port

## 2. Build mmdeploy

### 1) Environment

| Matters | Version            | Remarks                |
| ------- | ------------------ | ---------------------- |
| host OS | ubuntu18.04 x86_64 | snpe specified version |
| Python  | **3.6.0**          | snpe specified version |

### 2) Installation

Download [snpe-1.59 from the official website](https://developer.qualcomm.com/qfile/69652/snpe-1.59.0.zip)

```bash
$ unzip snpe-1.59.0.zip
$ export SNPE_ROOT=${PWD}/snpe-1.59.0.3230
$ cd /path/to/mmdeploy
$ export PYTHONPATH=${PWD}/service/snpe/client:${SNPE_ROOT}/lib/python:${PYTHONPATH}
$ export LD_LIBRARY_PATH=${SNPE_ROOT}/lib/x86_64-linux-clang:${LD_LIBRARY_PATH}
$ export PATH=${SNPE_ROOT}/bin/x86_64-linux-clang:${PATH}
$ python3 -m pip install -e .
```

## 3. Test the model

Take Resnet-18 as an example. First refer to [documentation to install mmpretrain](https://github.com/open-mmlab/mmpretrain/tree/main)  and  use `tools/deploy.py` to convert the model.

```bash
$ export MODEL_CONFIG=/path/to/mmpretrain/configs/resnet/resnet18_8xb16_cifar10.py
$ export MODEL_PATH=https://download.openmmlab.com/mmclassification/v0/resnet/resnet18_b16x8_cifar10_20210528-bd6371c8.pth

# Convert the model
$ cd /path/to/mmdeploy
$ python3 tools/deploy.py  configs/mmpretrain/classification_snpe_static.py $MODEL_CONFIG  $MODEL_PATH   /path/to/test.png   --work-dir resnet18   --device cpu  --uri 10.0.0.1\:60000  --dump-info

# Test
$ python3 tools/test.py configs/mmpretrain/classification_snpe_static.py   $MODEL_CONFIG    --model reset18/end2end.dlc   --metrics accuracy precision f1_score recall  --uri 10.0.0.1\:60000
```

Note that `--uri` is required to specify the ip and port of the snpe inference service, ipv4 and ipv6 addresses can be used.

## 4. Build SDK with Android SDK

If you also need to compile mmdeploy SDK with Android NDK, please continue reading.

### 1) Download NDK and OpenCV package and setup environment

```bash
# Download android OCV
$ export OPENCV_VERSION=4.5.4
$ wget https://github.com/opencv/opencv/releases/download/${OPENCV_VERSION}/opencv-${OPENCV_VERSION}-android-sdk.zip
$ unzip opencv-${OPENCV_VERSION}-android-sdk.zip

$ export ANDROID_OCV_ROOT=`realpath opencv-${OPENCV_VERSION}-android-sdk`

# Download ndk r23b
$ wget https://dl.google.com/android/repository/android-ndk-r23b-linux.zip
$ unzip android-ndk-r23b-linux.zip

$ export ANDROID_NDK_ROOT=`realpath android-ndk-r23b`
```

### 2) Compile mmdeploy SDK and demo

```bash
$ cd /path/to/mmdeploy
$ mkdir build && cd build
$ cmake .. \
  -DMMDEPLOY_BUILD_SDK=ON \
  -DCMAKE_TOOLCHAIN_FILE=${ANDROID_NDK_ROOT}/build/cmake/android.toolchain.cmake \
  -DMMDEPLOY_TARGET_BACKENDS=snpe \
  -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-30  \
  -DANDROID_STL=c++_static  \
  -DOpenCV_DIR=${ANDROID_OCV_ROOT}/sdk/native/jni/abi-arm64-v8a \
  -DMMDEPLOY_BUILD_EXAMPLES=ON

  $ make && make install
  $ tree ./bin
./bin
├── image_classification
├── image_restorer
├── image_segmentation
├── mmdeploy_onnx2ncnn
├── object_detection
├── ocr
├── pose_detection
└── rotated_object_detection
```

| Options                       | Description                                                  |
| ----------------------------- | ------------------------------------------------------------ |
| CMAKE_TOOLCHAIN_FILE          | Load NDK parameters, mainly used to select compiler          |
| MMDEPLOY_TARGET_BACKENDS=snpe | Inference backend                                            |
| ANDROID_STL=c++\_static       | In case of NDK environment can not find suitable c++ library |
| MMDEPLOY_SHARED_LIBS=ON       | snpe does not provide static library                         |

[Here](../01-how-to-build/cmake_option.md) is all cmake build option description.

### 3) Run the demo

First make sure that`--dump-info`is used during convert model, so that the `resnet18` directory has the files required by the SDK such as `pipeline.json`.

`adb push` the model directory, executable file and .so to the device.

```bash
$ cd /path/to/mmdeploy
$ adb push resnet18  /data/local/tmp
$ adb push tests/data/tiger.jpeg /data/local/tmp/resnet18/

$ cd /path/to/install/
$ adb push lib /data/local/tmp
$ adb push bin/image_classification /data/local/tmp/resnet18/
```

Set up environment variable and execute the sample.

```bash
$ adb push /path/to/mmpretrain/demo/demo.JPEG /data/local/tmp
$ adb shell
venus:/ $ cd /data/local/tmp/resnet18
venus:/data/local/tmp/resnet18 $ export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/data/local/tmp/lib

venus:/data/local/tmp/resnet18 $ ./image_classification cpu ./  tiger.jpeg
..
label: 3, score: 0.3214
```