Skip to content

Commit c0bbed7

Browse files
authored
[FastDeploy] support FastDeploy for PaddleSeg (#2976)
1 parent 3586adf commit c0bbed7

File tree

163 files changed

+15035
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

163 files changed

+15035
-0
lines changed

deploy/fastdeploy/README.md

+139
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
# PaddleSeg高性能全场景模型部署方案—FastDeploy
2+
3+
## 目录
4+
- [FastDeploy介绍](#FastDeploy介绍)
5+
- [语义分割模型部署](#语义分割模型部署)
6+
- [Matting模型部署](#Matting模型部署)
7+
- [常见问题](#常见问题)
8+
9+
## 1. FastDeploy介绍
10+
<div id="FastDeploy介绍"></div>
11+
12+
**[⚡️FastDeploy](https://github.com/PaddlePaddle/FastDeploy)**是一款**全场景****易用灵活****极致高效**的AI推理部署工具,支持**云边端**部署。使用FastDeploy可以简单高效的在X86 CPU、NVIDIA GPU、飞腾CPU、ARM CPU、Intel GPU、昆仑、昇腾、瑞芯微、晶晨、算能等10+款硬件上对PaddleSeg模型进行快速部署,并且支持Paddle Inference、Paddle Lite、TensorRT、OpenVINO、ONNXRuntime、RKNPU2、SOPHGO等多种推理后端。
13+
14+
<div align="center">
15+
16+
<img src="https://user-images.githubusercontent.com/31974251/219546373-c02f24b7-2222-4ad4-9b43-42b8122b898f.png" >
17+
18+
</div>
19+
20+
## 2. 语义分割模型部署
21+
<div id="语义分割模型部署"></div>
22+
23+
### 2.1 硬件支持列表
24+
25+
|硬件类型|该硬件是否支持|使用指南|Python|C++|
26+
|:---:|:---:|:---:|:---:|:---:|
27+
|X86 CPU||[链接](semantic_segmentation/cpu-gpu)|||
28+
|NVIDIA GPU||[链接](semantic_segmentation/cpu-gpu)|||
29+
|飞腾CPU||[链接](semantic_segmentation/cpu-gpu)|||
30+
|ARM CPU||[链接](semantic_segmentation/cpu-gpu)|||
31+
|Intel GPU(集成显卡)||[链接](semantic_segmentation/cpu-gpu)|||
32+
|Intel GPU(独立显卡)||[链接](semantic_segmentation/cpu-gpu)|||
33+
|昆仑||[链接](semantic_segmentation/kunlun)|||
34+
|昇腾||[链接](semantic_segmentation/ascend)|||
35+
|瑞芯微||[链接](semantic_segmentation/rockchip)|||
36+
|晶晨||[链接](semantic_segmentation/amlogic)|--|||
37+
|算能||[链接](semantic_segmentation/sophgo)|||
38+
39+
### 2.2. 详细使用文档
40+
- X86 CPU
41+
- [部署模型准备](semantic_segmentation/cpu-gpu)
42+
- [Python部署示例](semantic_segmentation/cpu-gpu/python/)
43+
- [C++部署示例](semantic_segmentation/cpu-gpu/cpp/)
44+
- NVIDIA GPU
45+
- [部署模型准备](semantic_segmentation/cpu-gpu)
46+
- [Python部署示例](semantic_segmentation/cpu-gpu/python/)
47+
- [C++部署示例](semantic_segmentation/cpu-gpu/cpp/)
48+
- 飞腾CPU
49+
- [部署模型准备](semantic_segmentation/cpu-gpu)
50+
- [Python部署示例](semantic_segmentation/cpu-gpu/python/)
51+
- [C++部署示例](semantic_segmentation/cpu-gpu/cpp/)
52+
- ARM CPU
53+
- [部署模型准备](semantic_segmentation/cpu-gpu)
54+
- [Python部署示例](semantic_segmentation/cpu-gpu/python/)
55+
- [C++部署示例](semantic_segmentation/cpu-gpu/cpp/)
56+
- Intel GPU
57+
- [部署模型准备](semantic_segmentation/cpu-gpu)
58+
- [Python部署示例](semantic_segmentation/cpu-gpu/python/)
59+
- [C++部署示例](semantic_segmentation/cpu-gpu/cpp/)
60+
- 昆仑 XPU
61+
- [部署模型准备](semantic_segmentation/kunlun)
62+
- [Python部署示例](semantic_segmentation/kunlun/python/)
63+
- [C++部署示例](semantic_segmentation/kunlun/cpp/)
64+
- 昇腾 Ascend
65+
- [部署模型准备](semantic_segmentation/ascend)
66+
- [Python部署示例](semantic_segmentation/ascend/python/)
67+
- [C++部署示例](semantic_segmentation/ascend/cpp/)
68+
- 瑞芯微 Rockchip
69+
- [部署模型准备](semantic_segmentation/rockchip/)
70+
- [Python部署示例](semantic_segmentation/rockchip/rknpu2/)
71+
- [C++部署示例](semantic_segmentation/rockchip/rknpu2/)
72+
- 晶晨 Amlogic
73+
- [部署模型准备](semantic_segmentation/amlogic/a311d/)
74+
- [C++部署示例](semantic_segmentation/amlogic/a311d/cpp/)
75+
- 算能 Sophgo
76+
- [部署模型准备](semantic_segmentation/sophgo/)
77+
- [Python部署示例](semantic_segmentation/sophgo/python/)
78+
- [C++部署示例](semantic_segmentation/sophgo/cpp/)
79+
80+
### 2.3 更多部署方式
81+
82+
- [Android ARM CPU部署](semantic_segmentation/android)
83+
- [服务化Serving部署](semantic_segmentation/serving)
84+
- [web部署](semantic_segmentation/web)
85+
- [模型自动化压缩工具](semantic_segmentation/quantize)
86+
87+
## 3. Matting模型部署
88+
<div id="Matting模型部署"></div>
89+
90+
### 3.1 硬件支持列表
91+
92+
|硬件类型|该硬件是否支持|使用指南|Python|C++|
93+
|:---:|:---:|:---:|:---:|:---:|
94+
|X86 CPU||[链接](matting/cpu-gpu)|||
95+
|NVIDIA GPU||[链接](matting/cpu-gpu)|||
96+
|飞腾CPU||[链接](matting/cpu-gpu)|||
97+
|ARM CPU||[链接](matting/cpu-gpu)|||
98+
|Intel GPU(集成显卡)||[链接](matting/cpu-gpu)|||
99+
|Intel GPU(独立显卡)||[链接](matting/cpu-gpu)|||
100+
|昆仑||[链接](matting/kunlun)|||
101+
|昇腾||[链接](matting/ascend)|||
102+
103+
### 3.2 详细使用文档
104+
- X86 CPU
105+
- [部署模型准备](matting/cpu-gpu)
106+
- [Python部署示例](matting/cpu-gpu/python/)
107+
- [C++部署示例](matting/cpu-gpu/cpp/)
108+
- NVIDIA GPU
109+
- [部署模型准备](matting/cpu-gpu)
110+
- [Python部署示例](matting/cpu-gpu/python/)
111+
- [C++部署示例](matting/cpu-gpu/cpp/)
112+
- 飞腾CPU
113+
- [部署模型准备](matting/cpu-gpu)
114+
- [Python部署示例](matting/cpu-gpu/python/)
115+
- [C++部署示例](matting/cpu-gpu/cpp/)
116+
- ARM CPU
117+
- [部署模型准备](matting/cpu-gpu)
118+
- [Python部署示例](matting/cpu-gpu/python/)
119+
- [C++部署示例](matting/cpu-gpu/cpp/)
120+
- Intel GPU
121+
- [部署模型准备](matting/cpu-gpu)
122+
- [Python部署示例](matting/cpu-gpu/python/)
123+
- [C++部署示例](cpu-gpu/cpp/)
124+
- 昆仑 XPU
125+
- [部署模型准备](matting/kunlun)
126+
- [Python部署示例](matting/kunlun/README.md)
127+
- [C++部署示例](matting/kunlun/README.md)
128+
- 昇腾 Ascend
129+
- [部署模型准备](matting/ascend)
130+
- [Python部署示例](matting/ascend/README.md)
131+
- [C++部署示例](matting/ascend/README.md)
132+
133+
## 4. 常见问题
134+
<div id="常见问题"></div>
135+
136+
遇到问题可查看常见问题集合,搜索FastDeploy issue,*或给FastDeploy提交[issue](https://github.com/PaddlePaddle/FastDeploy/issues)*:
137+
138+
[常见问题集合](https://github.com/PaddlePaddle/FastDeploy/tree/develop/docs/cn/faq)
139+
[FastDeploy issues](https://github.com/PaddlePaddle/FastDeploy/issues)

deploy/fastdeploy/matting/README.md

+54
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# PaddleSeg Matting模型高性能全场景部署方案-FastDeploy
2+
3+
## 1. FastDeploy介绍
4+
**[⚡️FastDeploy](https://github.com/PaddlePaddle/FastDeploy)**是一款**全场景****易用灵活****极致高效**的AI推理部署工具,支持**云边端**部署。使用FastDeploy可以简单高效的在X86 CPU、NVIDIA GPU、飞腾CPU、ARM CPU、Intel GPU、昆仑、昇腾、瑞芯微、晶晨、算能等10+款硬件上对PaddleSeg Matting模型进行快速部署,并且支持Paddle Inference、Paddle Lite、TensorRT、OpenVINO、ONNXRuntime、RKNPU2、SOPHGO等多种推理后端。
5+
6+
## 2. 硬件支持列表
7+
8+
|硬件类型|该硬件是否支持|使用指南|Python|C++|
9+
|:---:|:---:|:---:|:---:|:---:|
10+
|X86 CPU||[链接](cpu-gpu)|||
11+
|NVIDIA GPU||[链接](cpu-gpu)|||
12+
|飞腾CPU||[链接](cpu-gpu)|||
13+
|ARM CPU||[链接](cpu-gpu)|||
14+
|Intel GPU(集成显卡)||[链接](cpu-gpu)|||
15+
|Intel GPU(独立显卡)||[链接](cpu-gpu)|||
16+
|昆仑||[链接](kunlun)|||
17+
|昇腾||[链接](ascend)|||
18+
19+
## 3. 详细使用文档
20+
- X86 CPU
21+
- [部署模型准备](cpu-gpu)
22+
- [Python部署示例](cpu-gpu/python/)
23+
- [C++部署示例](cpu-gpu/cpp/)
24+
- NVIDIA GPU
25+
- [部署模型准备](cpu-gpu)
26+
- [Python部署示例](cpu-gpu/python/)
27+
- [C++部署示例](cpu-gpu/cpp/)
28+
- 飞腾CPU
29+
- [部署模型准备](cpu-gpu)
30+
- [Python部署示例](cpu-gpu/python/)
31+
- [C++部署示例](cpu-gpu/cpp/)
32+
- ARM CPU
33+
- [部署模型准备](cpu-gpu)
34+
- [Python部署示例](cpu-gpu/python/)
35+
- [C++部署示例](cpu-gpu/cpp/)
36+
- Intel GPU
37+
- [部署模型准备](cpu-gpu)
38+
- [Python部署示例](cpu-gpu/python/)
39+
- [C++部署示例](cpu-gpu/cpp/)
40+
- 昆仑 XPU
41+
- [部署模型准备](kunlun)
42+
- [Python部署示例](kunlun/README.md)
43+
- [C++部署示例](kunlun/README.md)
44+
- 昇腾 Ascend
45+
- [部署模型准备](ascend)
46+
- [Python部署示例](ascend/README.md)
47+
- [C++部署示例](ascend/README.md)
48+
49+
## 4. 常见问题
50+
51+
遇到问题可查看常见问题集合,搜索FastDeploy issue,*或给FastDeploy提交[issue](https://github.com/PaddlePaddle/FastDeploy/issues)*:
52+
53+
[常见问题集合](https://github.com/PaddlePaddle/FastDeploy/tree/develop/docs/cn/faq)
54+
[FastDeploy issues](https://github.com/PaddlePaddle/FastDeploy/issues)
+31
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# PaddleSeg Matting模型高性能全场景部署方案-FastDeploy
2+
3+
## 1. 说明
4+
PaddleSeg支持利用FastDeploy在NVIDIA GPU、X86 CPU、飞腾CPU、ARM CPU、Intel GPU(独立显卡/集成显卡)硬件上快速部署Matting模型
5+
6+
## 2. 使用预导出的模型列表
7+
为了方便开发者的测试,下面提供了PP-Matting导出的各系列模型,开发者可直接下载使用。其中精度指标来源于PP-Matting中对各模型的介绍(未提供精度数据),详情各参考PP-Matting中的说明。**注意**`deploy.yaml`文件记录导出模型的`input_shape`以及预处理信息,若不满足要求,用户可重新导出相关模型。
8+
9+
| 模型 | 参数大小 | 精度 | 备注 |
10+
|:---------------------------------------------------------------- |:----- |:----- | :------ |
11+
| [PP-Matting-512](https://bj.bcebos.com/paddlehub/fastdeploy/PP-Matting-512.tgz) | 106MB | - |
12+
| [PP-Matting-1024](https://bj.bcebos.com/paddlehub/fastdeploy/PP-Matting-1024.tgz) | 106MB | - |
13+
| [PP-HumanMatting](https://bj.bcebos.com/paddlehub/fastdeploy/PPHumanMatting.tgz) | 247MB | - |
14+
| [Modnet-ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/PPModnet_ResNet50_vd.tgz) | 355MB | - |
15+
| [Modnet-MobileNetV2](https://bj.bcebos.com/paddlehub/fastdeploy/PPModnet_MobileNetV2.tgz) | 28MB | - |
16+
| [Modnet-HRNet_w18](https://bj.bcebos.com/paddlehub/fastdeploy/PPModnet_HRNet_w18.tgz) | 51MB | - |
17+
18+
## 3. 自行导出PaddleSeg部署模型
19+
### 3.1 模型版本
20+
21+
支持[PaddleSeg](https://github.com/PaddlePaddle/PaddleSeg/tree/develop) 高于2.6版本的Matting模型,目前FastDeploy中测试过模型如下:
22+
- [PP-Matting系列模型](https://github.com/PaddlePaddle/PaddleSeg/tree/develop/Matting)
23+
- [PP-HumanMatting系列模型](https://github.com/PaddlePaddle/PaddleSeg/tree/develop/Matting)
24+
- [ModNet系列模型](https://github.com/PaddlePaddle/PaddleSeg/tree/develop/Matting)
25+
26+
### 3.2 模型导出
27+
PaddleSeg模型导出,请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleSeg/tree/develop/Matting)**注意**:PaddleSeg导出的模型包含`model.pdmodel``model.pdiparams``deploy.yaml`三个文件,FastDeploy会从yaml文件中获取模型在推理时需要的预处理信息
28+
29+
## 4. 详细的部署示例
30+
- [Python部署](../cpu-gpu/python)
31+
- [C++部署](../cpu-gpu/cpp)
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# PaddleSeg Matting模型高性能全场景部署方案-FastDeploy
2+
3+
## 1. 说明
4+
PaddleSeg支持利用FastDeploy在NVIDIA GPU、X86 CPU、飞腾CPU、ARM CPU、Intel GPU(独立显卡/集成显卡)硬件上快速部署Matting模型
5+
6+
## 2. 使用预导出的模型列表
7+
为了方便开发者的测试,下面提供了PP-Matting导出的各系列模型,开发者可直接下载使用。其中精度指标来源于PP-Matting中对各模型的介绍(未提供精度数据),详情各参考PP-Matting中的说明。**注意**`deploy.yaml`文件记录导出模型的`input_shape`以及预处理信息,若不满足要求,用户可重新导出相关模型。
8+
9+
| 模型 | 参数大小 | 精度 | 备注 |
10+
|:---------------------------------------------------------------- |:----- |:----- | :------ |
11+
| [PP-Matting-512](https://bj.bcebos.com/paddlehub/fastdeploy/PP-Matting-512.tgz) | 106MB | - |
12+
| [PP-Matting-1024](https://bj.bcebos.com/paddlehub/fastdeploy/PP-Matting-1024.tgz) | 106MB | - |
13+
| [PP-HumanMatting](https://bj.bcebos.com/paddlehub/fastdeploy/PPHumanMatting.tgz) | 247MB | - |
14+
| [Modnet-ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/PPModnet_ResNet50_vd.tgz) | 355MB | - |
15+
| [Modnet-MobileNetV2](https://bj.bcebos.com/paddlehub/fastdeploy/PPModnet_MobileNetV2.tgz) | 28MB | - |
16+
| [Modnet-HRNet_w18](https://bj.bcebos.com/paddlehub/fastdeploy/PPModnet_HRNet_w18.tgz) | 51MB | - |
17+
18+
## 3. 自行导出PaddleSeg部署模型
19+
### 3.1 模型版本
20+
21+
支持[PaddleSeg](https://github.com/PaddlePaddle/PaddleSeg/tree/develop) 高于2.6版本的Matting模型,目前FastDeploy中测试过模型如下:
22+
- [PP-Matting系列模型](https://github.com/PaddlePaddle/PaddleSeg/tree/develop/Matting)
23+
- [PP-HumanMatting系列模型](https://github.com/PaddlePaddle/PaddleSeg/tree/develop/Matting)
24+
- [ModNet系列模型](https://github.com/PaddlePaddle/PaddleSeg/tree/develop/Matting)
25+
26+
### 3.2 模型导出
27+
PaddleSeg模型导出,请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleSeg/tree/develop/Matting)**注意**:PaddleSeg导出的模型包含`model.pdmodel``model.pdiparams``deploy.yaml`三个文件,FastDeploy会从yaml文件中获取模型在推理时需要的预处理信息
28+
29+
## 4. 详细的部署示例
30+
- [Python部署](python)
31+
- [C++部署](cpp)
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
PROJECT(infer_demo C CXX)
2+
CMAKE_MINIMUM_REQUIRED (VERSION 3.10)
3+
4+
option(FASTDEPLOY_INSTALL_DIR "Path of downloaded fastdeploy sdk.")
5+
6+
include(${FASTDEPLOY_INSTALL_DIR}/FastDeploy.cmake)
7+
8+
include_directories(${FASTDEPLOY_INCS})
9+
10+
add_executable(infer_demo ${PROJECT_SOURCE_DIR}/infer.cc)
11+
target_link_libraries(infer_demo ${FASTDEPLOY_LIBS})
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
[English](README.md) | 简体中文
2+
# PP-Matting CPU-GPU C++部署示例
3+
4+
本目录下提供`infer.cc`快速完成PP-Matting在CPU/GPU、昆仑芯、华为昇腾以及GPU上通过Paddle-TensorRT加速部署的示例。
5+
6+
## 1. 说明
7+
PaddleSeg支持利用FastDeploy在NVIDIA GPU、X86 CPU、飞腾CPU、ARM CPU、Intel GPU(独立显卡/集成显卡)硬件上快速部署Matting模型
8+
9+
## 2. 部署环境准备
10+
在部署前,需确认软硬件环境,同时下载预编译部署库,参考文档[FastDeploy预编译库安装](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install)**注意** 只有CPU、GPU提供预编译库,华为昇腾以及昆仑芯需要参考以上文档自行编译部署环境。
11+
12+
## 3. 部署模型准备
13+
在部署前,请准备好您所需要运行的推理模型,你可以选择使用[预导出的推理模型](../README.md)或者[自行导出PaddleSeg部署模型](../README.md)
14+
15+
## 4. 运行部署示例
16+
以Linux上推理为例,在本目录执行如下命令即可完成编译测试,支持此模型需保证FastDeploy版本1.0.0以上(x.x.x>=1.0.0)
17+
18+
```bash
19+
# 下载FastDeploy预编译库,用户可在上文提到的`FastDeploy预编译库`中自行选择合适的版本使用
20+
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz
21+
tar xvf fastdeploy-linux-x64-x.x.x.tgz
22+
23+
# 下载部署示例代码
24+
git clone https://github.com/PaddlePaddle/PaddleSeg.git
25+
cd PaddleSeg/deploy/fastdeploy/matting/cpp-gpu/cpp
26+
27+
# 编译部署示例
28+
mkdir build && cd build
29+
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
30+
make -j
31+
32+
# 下载PP-Matting模型文件和测试图片
33+
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP-Matting-512.tgz
34+
tar -xvf PP-Matting-512.tgz
35+
wget https://bj.bcebos.com/paddlehub/fastdeploy/matting_input.jpg
36+
wget https://bj.bcebos.com/paddlehub/fastdeploy/matting_bgr.jpg
37+
38+
# CPU推理
39+
./infer_demo PP-Matting-512 matting_input.jpg matting_bgr.jpg 0
40+
# GPU推理
41+
./infer_demo PP-Matting-512 matting_input.jpg matting_bgr.jpg 1
42+
# GPU上TensorRT推理
43+
./infer_demo PP-Matting-512 matting_input.jpg matting_bgr.jpg 2
44+
# 昆仑芯XPU推理
45+
./infer_demo PP-Matting-512 matting_input.jpg matting_bgr.jpg 3
46+
```
47+
**注意** 以上示例未提供华为昇腾的示例,在编译好昇腾部署环境后,只需改造一行代码,将示例文件中KunlunXinInfer方法的`option.UseKunlunXin()``option.UseAscend()`就可以完成在华为昇腾上的推理部署
48+
49+
运行完成可视化结果如下图所示
50+
<div width="840">
51+
<img width="200" height="200" float="left" src="https://user-images.githubusercontent.com/67993288/186852040-759da522-fca4-4786-9205-88c622cd4a39.jpg">
52+
<img width="200" height="200" float="left" src="https://user-images.githubusercontent.com/67993288/186852587-48895efc-d24a-43c9-aeec-d7b0362ab2b9.jpg">
53+
<img width="200" height="200" float="left" src="https://user-images.githubusercontent.com/67993288/186852116-cf91445b-3a67-45d9-a675-c69fe77c383a.jpg">
54+
<img width="200" height="200" float="left" src="https://user-images.githubusercontent.com/67993288/186852554-6960659f-4fd7-4506-b33b-54e1a9dd89bf.jpg">
55+
</div>
56+
57+
以上命令只适用于Linux或MacOS, Windows下SDK的使用方式请参考:
58+
- [如何在Windows中使用FastDeploy C++ SDK](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/use_sdk_on_windows.md)
59+
60+
## 5. 更多指南
61+
- [PaddleSeg C++ API文档](https://www.paddlepaddle.org.cn/fastdeploy-api-doc/cpp/html/namespacefastdeploy_1_1vision_1_1segmentation.html)
62+
- [FastDeploy部署PaddleSeg模型概览](../../)
63+
- [Python部署](../python)
64+
65+
## 6. 常见问题
66+
- [如何切换模型推理后端引擎](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/how_to_change_backend.md)
67+
- [Intel GPU(独立显卡/集成显卡)的使用](https://github.com/PaddlePaddle/FastDeploy/blob/develop/tutorials/intel_gpu/README.md)
68+
- [编译CPU部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/cpu.md)
69+
- [编译GPU部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/gpu.md)
70+
- [编译Jetson部署库](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/jetson.md)

0 commit comments

Comments
 (0)