Skip to content

Commit 038cde8

Browse files
LutaoChuLutaoChu
authored
add lovasz loss, mixed loss, road extration dataset (PaddlePaddle#825)
1 parent df178c2 commit 038cde8

18 files changed

+683
-1
lines changed

‎README.md

+1
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@ python train.py --config configs/quick_start/bisenet_optic_disc_512x512_1k.yml
9393
* [API Tutorial](https://aistudio.baidu.com/aistudio/projectdetail/1339458)
9494
* [Data Preparation](./docs/data_prepare.md)
9595
* [Training Configuration](./configs/)
96+
* [Loss Usage](./docs/loss_usage.md)
9697
* [API References](./docs/apis)
9798
* [Add New Components](./docs/add_new_model.md)
9899
* [Model Compression](./slim)

‎README_CN.md

+1
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,7 @@ python train.py --config configs/quick_start/bisenet_optic_disc_512x512_1k.yml
9595
* [API使用教程](https://aistudio.baidu.com/aistudio/projectdetail/1339458)
9696
* [数据集准备](./docs/data_prepare.md)
9797
* [配置项](./configs/)
98+
* [Loss使用](./docs/loss_usage.md)
9899
* [API参考](./docs/apis)
99100
* [添加新组件](./docs/add_new_model.md)
100101
* [模型压缩](./slim)

‎configs/ocrnet/ocrnet_hrnetw18_cityscapes_1024x512_160k_lovasz_softmax.yml

+35
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
_base_: '../_base_/cityscapes.yml'
2+
3+
batch_size: 2
4+
iters: 160000
5+
6+
model:
7+
type: OCRNet
8+
backbone:
9+
type: HRNet_W18
10+
pretrained: https://bj.bcebos.com/paddleseg/dygraph/hrnet_w18_ssld.tar.gz
11+
backbone_indices: [0]
12+
13+
optimizer:
14+
type: sgd
15+
16+
learning_rate:
17+
value: 0.01
18+
decay:
19+
type: poly
20+
power: 0.9
21+
22+
23+
loss:
24+
types:
25+
- type: MixedLoss
26+
losses:
27+
- type: CrossEntropyLoss
28+
- type: LovaszSoftmaxLoss
29+
coef: [0.8, 0.2]
30+
- type: MixedLoss
31+
losses:
32+
- type: CrossEntropyLoss
33+
- type: LovaszSoftmaxLoss
34+
coef: [0.8, 0.2]
35+
coef: [1, 0.4]

‎configs/ocrnet/ocrnet_hrnetw18_road_extraction_768x768_15k.yml

+48
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
2+
3+
batch_size: 4
4+
iters: 15000
5+
6+
train_dataset:
7+
type: MiniDeepGlobeRoadExtraction
8+
dataset_root: data/MiniDeepGlobeRoadExtraction
9+
transforms:
10+
- type: ResizeStepScaling
11+
min_scale_factor: 0.5
12+
max_scale_factor: 2.0
13+
scale_step_size: 0.25
14+
- type: RandomPaddingCrop
15+
crop_size: [768, 768]
16+
- type: RandomHorizontalFlip
17+
- type: Normalize
18+
mode: train
19+
20+
val_dataset:
21+
type: MiniDeepGlobeRoadExtraction
22+
dataset_root: data/MiniDeepGlobeRoadExtraction
23+
transforms:
24+
- type: Normalize
25+
mode: val
26+
27+
model:
28+
type: OCRNet
29+
backbone:
30+
type: HRNet_W18
31+
pretrained: https://bj.bcebos.com/paddleseg/dygraph/hrnet_w18_ssld.tar.gz
32+
backbone_indices: [0]
33+
34+
optimizer:
35+
type: sgd
36+
37+
learning_rate:
38+
value: 0.01
39+
decay:
40+
type: poly
41+
power: 0.9
42+
43+
44+
loss:
45+
types:
46+
- type: CrossEntropyLoss
47+
- type: CrossEntropyLoss
48+
coef: [1, 0.4]

‎configs/ocrnet/ocrnet_hrnetw18_road_extraction_768x768_15k_lovasz_hinge.yml

+55
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
2+
3+
batch_size: 4
4+
iters: 15000
5+
6+
train_dataset:
7+
type: MiniDeepGlobeRoadExtraction
8+
dataset_root: data/MiniDeepGlobeRoadExtraction
9+
transforms:
10+
- type: ResizeStepScaling
11+
min_scale_factor: 0.5
12+
max_scale_factor: 2.0
13+
scale_step_size: 0.25
14+
- type: RandomPaddingCrop
15+
crop_size: [768, 768]
16+
- type: RandomHorizontalFlip
17+
- type: Normalize
18+
mode: train
19+
20+
val_dataset:
21+
type: MiniDeepGlobeRoadExtraction
22+
dataset_root: data/MiniDeepGlobeRoadExtraction
23+
transforms:
24+
- type: Normalize
25+
mode: val
26+
27+
model:
28+
type: OCRNet
29+
backbone:
30+
type: HRNet_W18
31+
pretrained: https://bj.bcebos.com/paddleseg/dygraph/hrnet_w18_ssld.tar.gz
32+
backbone_indices: [0]
33+
34+
optimizer:
35+
type: sgd
36+
37+
learning_rate:
38+
value: 0.01
39+
decay:
40+
type: poly
41+
power: 0.9
42+
43+
loss:
44+
types:
45+
- type: MixedLoss
46+
losses:
47+
- type: CrossEntropyLoss
48+
- type: LovaszHingeLoss
49+
coef: [1, 0.01]
50+
- type: MixedLoss
51+
losses:
52+
- type: CrossEntropyLoss
53+
- type: LovaszHingeLoss
54+
coef: [1, 0.01]
55+
coef: [1, 0.4]

‎docs/apis/README.md

+1
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,5 @@
55
* [paddleseg.datasets](./datasets.md)
66
* [paddleseg.models](./models.md)
77
* [paddleseg.models.backbones](./backbones.md)
8+
* [paddleseg.models.losses](./losses.md)
89
* [paddleseg.transforms](./transforms.md)

‎docs/apis/losses.md

+33
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# [paddleseg.models.losses](../../paddleseg/models/losses)
2+
3+
## LovaszSoftmaxLoss
4+
> CLASS paddleseg.models.losses.LovaszSoftmaxLoss(ignore_index=255, classes='present')
5+
6+
Multi-class Lovasz-Softmax loss.
7+
8+
> > Args
9+
> > > - **ignore_index** (int64): Specifies a target value that is ignored and does not contribute to the input gradient. Default ``255``.
10+
> > > - **classes** (str|list): 'all' for all, 'present' for classes present in labels, or a list of classes to average.
11+
12+
13+
## LovaszHingeLoss
14+
> CLASS paddleseg.models.losses.LovaszHingeLoss(ignore_index=255)
15+
16+
Binary Lovasz hinge loss.
17+
18+
> > Args
19+
> > > - **ignore_index** (int64): Specifies a target value that is ignored and does not contribute to the input gradient. Default ``255``.
20+
21+
22+
## MixedLoss
23+
> CLASS paddleseg.models.losses.MixedLoss(losses, coef)
24+
25+
Weighted computations for multiple Loss.
26+
The advantage is that mixed loss training can be achieved without changing the networking code.
27+
28+
> > Args
29+
> > > - **losses** (list of nn.Layer): A list consisting of multiple loss classes
30+
> > > - **coef** (float|int): Weighting coefficient of multiple loss
31+
32+
> > Returns
33+
> > > - A callable object of MixedLoss.

‎docs/images/Lovasz_Hinge_Evaluate_mIoU.png

48.2 KB
Loading

‎docs/images/Lovasz_Softmax_Evaluate_mIoU.png

58.3 KB
Loading

‎docs/images/deepglobe.png

173 KB
Loading

‎docs/loss_usage.md

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# Loss usage
2+
3+
- [Lovasz loss](lovasz_loss.md)
4+
- To be continued

‎docs/lovasz_loss.md

+125
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# Lovasz loss
2+
对于图像分割任务中，经常出现类别分布不均匀的情况，例如：工业产品的瑕疵检测、道路提取及病变区域提取等。我们可使用lovasz loss解决这个问题。
3+
4+
Lovasz loss基于子模损失(submodular losses)的凸Lovasz扩展，对神经网络的mean IoU损失进行优化。Lovasz loss根据分割目标的类别数量可分为两种：lovasz hinge loss和lovasz softmax loss. 其中lovasz hinge loss适用于二分类问题，lovasz softmax loss适用于多分类问题。该工作发表在CVPR 2018上，可点击[参考文献](#参考文献)查看具体原理。
5+
6+
7+
## Lovasz loss使用指南
8+
接下来介绍如何使用lovasz loss进行训练。需要注意的是，通常的直接训练方式并一定管用，我们推荐另外2种训练方式：
9+
- （1）与cross entropy loss或bce loss(binary cross-entropy loss)加权结合使用。
10+
- （2）先使用cross entropy loss或bce loss进行训练，再使用lovasz softmax loss或lovasz hinge loss进行finetuning.
11+
12+
以方式（1）为例，通过`MixedLoss`类选择训练时的损失函数， 通过`coef`参数对不同loss进行权重配比，从而灵活地进行训练调参。如下所示：
13+
14+
```yaml
15+
loss:
16+
types:
17+
- type: MixedLoss
18+
losses:
19+
- type: CrossEntropyLoss
20+
- type: LovaszSoftmaxLoss
21+
coef: [0.8, 0.2]
22+
```
23+
24+
```yaml
25+
loss:
26+
types:
27+
- type: MixedLoss
28+
losses:
29+
- type: CrossEntropyLoss
30+
- type: LovaszHingeLoss
31+
coef: [1, 0.02]
32+
```
33+
34+
35+
## Lovasz softmax loss实验对比
36+
37+
接下来以经典的[Cityscapes](https://www.cityscapes-dataset.com/)数据集为例应用lovasz softmax loss. Cityscapes数据集共有19类目标，其中的类别并不均衡，例如类别`road`、`building`很常见，`fence`、`motocycle`、`wall`则较为罕见。我们将lovasz softmax loss与softmax loss进行了实验对比。这里使用OCRNet模型，backbone为HRNet w18.
38+
39+
40+
* 数据准备
41+
42+
见[数据集准备教程](data_prepare.md)
43+
44+
* Lovasz loss训练
45+
```shell
46+
CUDA_VISIBLE_DEVICES=0,1,2,3 python -u -m paddle.distributed.launch train.py \
47+
--config configs/ocrnet/ocrnet_hrnetw18_cityscapes_1024x512_160k_lovasz_softmax.yml \
48+
--use_vdl --num_workers 3 --do_eval
49+
```
50+
51+
* Cross entropy loss训练
52+
```shell
53+
CUDA_VISIBLE_DEVICES=0,1,2,3 python -u -m paddle.distributed.launch train.py \
54+
--config configs/ocrnet/ocrnet_hrnetw18_cityscapes_1024x512_160k.yml \
55+
--use_vdl --num_workers 3 --do_eval
56+
```
57+
58+
* 结果比较
59+
60+
实验mIoU曲线如下图所示。
61+
<p align="center">
62+
<img src="./images/Lovasz_Softmax_Evaluate_mIoU.png" hspace='10' /> <br />
63+
</p>
64+
65+
66+
|Loss|best mIoU|
67+
|-|-|
68+
|cross entropy loss|80.46%|
69+
|lovasz softmax loss + cross entropy loss|81.53%|
70+
71+
图中蓝色曲线代表lovasz softmax loss + cross entropy loss，绿色曲线代表cross entropy loss，相比提升1个百分点。
72+
73+
可看出使用lovasz softmax loss后，精度曲线基本都高于原来的精度。
74+
75+
## Lovasz hinge loss实验对比
76+
77+
我们以道路提取任务为例应用lovasz hinge loss.
78+
基于MiniDeepGlobeRoadExtraction数据集与cross entropy loss进行了实验对比。
79+
该数据集来源于[DeepGlobe CVPR2018挑战赛](http://deepglobe.org/)的Road Extraction单项，训练数据道路占比为 4.5%. 道路在整张图片中的比例很小，是典型的类别不均衡场景。图片样例如下：
80+
<p align="center">
81+
<img src="./images/deepglobe.png" hspace='10'/> <br />
82+
</p>
83+
84+
这里使用OCRNet模型，backbone为HRNet w18.
85+
86+
* 数据集
87+
我们从DeepGlobe比赛的Road Extraction的训练集中随机抽取了800张图片作为训练集，200张图片作为验证集，
88+
制作了一个小型的道路提取数据集[MiniDeepGlobeRoadExtraction](https://paddleseg.bj.bcebos.com/dataset/MiniDeepGlobeRoadExtraction.zip)。
89+
运行训练脚本将自动下载该数据集。
90+
91+
* Lovasz loss训练
92+
```shell
93+
CUDA_VISIBLE_DEVICES=0,1,2,3 python -u -m paddle.distributed.launch train.py \
94+
--config configs/ocrnet/ocrnet_hrnetw18_road_extraction_768x768_15k_lovasz_hinge.yml \
95+
--use_vdl --num_workers 3 --do_eval
96+
```
97+
98+
* Cross entropy loss训练
99+
```shell
100+
CUDA_VISIBLE_DEVICES=0,1,2,3 python -u -m paddle.distributed.launch train.py \
101+
--config configs/ocrnet/ocrnet_hrnetw18_road_extraction_768x768_15k.yml \
102+
--use_vdl --num_workers 3 --do_eval
103+
```
104+
105+
* 结果比较
106+
107+
实验mIoU曲线如下图所示。
108+
<p align="center">
109+
<img src="./images/Lovasz_Hinge_Evaluate_mIoU.png" hspace='10'/> <br />
110+
</p>
111+
112+
113+
|Loss|best mIoU|
114+
|-|-|
115+
|cross entropy loss|78.69%|
116+
|lovasz softmax loss + cross entropy loss|79.18%|
117+
118+
图中紫色曲线为lovasz hinge loss + cross entropy loss，蓝色曲线为cross entropy loss，相比提升0.5个百分点。
119+
120+
可看出使用lovasz hinge loss后，精度曲线全面高于原来的精度。
121+
122+
123+
124+
## 参考文献
125+
[Berman M, Rannen Triki A, Blaschko M B. The lovász-softmax loss: a tractable surrogate for the optimization of the intersection-over-union measure in neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 4413-4421.](http://openaccess.thecvf.com/content_cvpr_2018/html/Berman_The_LovaSz-Softmax_Loss_CVPR_2018_paper.html)

‎paddleseg/cvlibs/config.py

+3-1
Original file line numberDiff line numberDiff line change
@@ -211,7 +211,9 @@ def loss(self) -> dict:
211211
if key == 'types':
212212
self._losses['types'] = []
213213
for item in args['types']:
214-
item['ignore_index'] = self.train_dataset.ignore_index
214+
if item['type'] != 'MixedLoss':
215+
item['ignore_index'] = \
216+
self.train_dataset.ignore_index
215217
self._losses['types'].append(self._load_object(item))
216218
else:
217219
self._losses[key] = val

‎paddleseg/datasets/__init__.py

+1
Original file line numberDiff line numberDiff line change
@@ -18,3 +18,4 @@
1818
from .ade import ADE20K
1919
from .optic_disc_seg import OpticDiscSeg
2020
from .pascal_context import PascalContext
21+
from .mini_deep_globe_road_extraction import MiniDeepGlobeRoadExtraction

0 commit comments

Comments
 (0)