problem about low miou result on 11 class

[WangPeng19981204]

Hello!
Thank you for your wonderful code! I have a problem that why the result is so low? "11 class(background and 10 class): mIOU: 34%, epoch50. Is this result consistent with the original ChangNet paper? Or is this a research direction that can be improved? look forward to your reply and guidance！
2021.08.18

[zhijiejia]

Maybe there are some problems with the mask I get when I process the dataset, I guess. If you have time, you can reorganize the dataset and regenerate the mask by yourself. Because I don't have a GPU now, I can't do relevant experiments and welcome to share your discovery if you have, thanks.

[WangPeng19981204]

hello！
I think i find the reason why miou result on 11 class is so low. because your test set of VL-CMU-CD have some promble. in test.txt only have about 52 image pairs in order to test your trained model. while about 1300 image pairs to train your model. the radio of train and test set is not suiltable while the proportion accounts for less than 5% of the whole data set. It leads to that in 11 class training and then test in 52 image pairs, iou of some class is very low like 0, so miou is very low. If divide more image pairs to test, the miou result of 11 class will improve.
I find that in 2 class training, if divide more image pairs to training and the remaining image pairs to test, f1score will improve. I noticed that in ChangeNet paper , VL-CMU-CD dataset was divided into train ,val ,test follow the radio 0.7,0.15,0.15, I follow this set, but get the f1score result about 0. 58, which is extremely inconsistent with the result 0.84 in paper. I also try the data split provided in CDnet paper and get the result f1score 0.60,which is also much lower then result in paper. I am very confused, how do you split your dataset, and why the result of f1sore much lower then result in paper?
Look forward to your reply and guidance！
2021.08.25