[BUG] DL Streamer - Geti Anomaly Detection PADIM Inference Result Is Not Correct

[ekaakurniawan]

Bug Description

I get the following inference results from DL Streamer. The model tested fine on Geti with 96% accuracy.

• Normal image: 70% normal

  {"objects":[{"classification_layer_name:output1":{"confidence":0.6993954181671143,"label":"Normal","model":{"name":"Model6"}},"h":471,"tensors":[{"confidence":0.6993954181671143,"label":"Normal","layer_name":"output1","layout":"ANY","model_name":"Model6","name":"classification_layer_name:output1","precision":"FP32"}],"w":640,"x":0,"y":0}],"resolution":{"height":471,"width":640},"timestamp":0}
  

• Anomaly image: 99% normal

  {"objects":[{"classification_layer_name:output1":{"confidence":0.9920363426208496,"label":"Normal","model":{"name":"Model6"}},"h":506,"tensors":[{"confidence":0.9920363426208496,"label":"Normal","layer_name":"output1","layout":"ANY","model_name":"Model6","name":"classification_layer_name:output1","precision":"FP32"}],"w":640,"x":0,"y":0}],"resolution":{"height":506,"width":640},"timestamp":0}
  

• Totally different image: 100% normal

System Setup

Processor: MTL-H 155H
OS: Ubuntu 24.04 LTS
DL Streamer: main branch 9 June 2025
PADIM model from Geti 2.10.1
Model XML: model-xml.txt

Reproducible Steps

Build Docker Image

$ sudo snap install docker

$ cd /home/eka/Workspace/
$ git clone https://github.com/open-edge-platform/edge-ai-libraries.git

$ cd /home/eka/Workspace/edge-ai-libraries/libraries/dl-streamer/thirdparty
$ rm -r spdlog
$ git clone https://github.com/gabime/spdlog.git
$ cd spdlog
$ git checkout -b 6fa3601

$ cd /home/eka/Workspace/edge-ai-libraries/libraries/dl-streamer
$ sudo docker build \
  -f /home/eka/Workspace/edge-ai-libraries/libraries/dl-streamer/docker/devel/ubuntu24/dlstreamer_dev_ubuntu24.Dockerfile \
  -t dlstreamer:main-dev-ubuntu24 .

$ sudo docker images
REPOSITORY         TAG                       IMAGE ID       CREATED          SIZE
dlstreamer         main-dev-ubuntu24         2ec360ae4177   17 minutes ago   7.83GB

Run Docker Container

$ sudo docker run \
  -v "/home/eka/Workspace/geti/datasets/anomaly/inference/a.jpg":/home/dlstreamer/intel/dl_streamer/datasets/a.jpg \
  -v "/home/eka/Workspace/geti/datasets/anomaly/inference/n.jpg":/home/dlstreamer/intel/dl_streamer/datasets/n.jpg \
  -v "/home/eka/Workspace/geti/models/PADIM_2v2_FP16/PADIM_2v2_FP16/deployment/Anomaly classification/model/model.xml":/home/dlstreamer/intel/dl_streamer/models/intel/geti-anomaly-detection/padim/FP16/geti-anomaly-detection.xml \
  -v "/home/eka/Workspace/geti/models/PADIM_2v2_FP16/PADIM_2v2_FP16/deployment/Anomaly classification/model/model.bin":/home/dlstreamer/intel/dl_streamer/models/intel/geti-anomaly-detection/padim/FP16/geti-anomaly-detection.bin \
  -e DISPLAY=$DISPLAY \
  -v /tmp/.X11-unix:/tmp/.X11-unix \
  --net=host \
  --privileged \
  --user root \
  --device /dev/dri \
  -it --rm \
  dlstreamer:main-dev-ubuntu24

Run Inference

# cd /home/dlstreamer/dlstreamer/samples/gstreamer/gst_launch/geti_deployment
# export MODELS_PATH="/home/dlstreamer/intel/dl_streamer/models/"
# bash geti_sample.sh \
  anomaly-detection \
  intel/geti-anomaly-detection/padim/FP16/geti-anomaly-detection.xml \
  CPU \
  ie \
  /home/dlstreamer/intel/dl_streamer/datasets/a.jpg \
  json

a.jpg is the image with anomaly. Following result shows 99% Normal.

# cat output.json 

{"objects":[{"classification_layer_name:output1":{"confidence":0.9920363426208496,"label":"Normal","model":{"name":"Model6"}},"h":506,"tensors":[{"confidence":0.9920363426208496,"label":"Normal","layer_name":"output1","layout":"ANY","model_name":"Model6","name":"classification_layer_name:output1","precision":"FP32"}],"w":640,"x":0,"y":0}],"resolution":{"height":506,"width":640},"timestamp":0}

Root Cause Analysis

No response

[ekurniaw]

Without anomaly_map normalization, the pred_score from the following line is already too high for a normal input as compared to image_threshold. May need to check the pre-processing (channel conversion and input normalization).

edge-ai-libraries/libraries/dl-streamer/src/monolithic/gst/inference_elements/common/post_processor/converters/to_tensor/detection_anomaly.cpp

Line 69 in dd4dd50

cv::minMaxLoc(anomaly_map, NULL, &pred_score);

image_threshold is 136.112. I got the following pred_score. The anomalous progression values are correct but they are just too high.

• Normal image: ~140
• Anomaly image: ~160
• Totally different image: ~180

[ekaakurniawan]

After swapping the input channel (BGR to RGB), I get the correct result. Will rise a PR.

• Normal image: ~40 resulting in 100% normal
• Anomaly image: ~180 resulting in 73% anomaly

[tbujewsk]

Hi @ekaakurniawan
Thank you for pointing out this. Can you paste the jpeg files from your tests?

Regards
Tomasz

[ekaakurniawan]

Original image with wrong result:

Inverted image with correct result:

@tbujewsk , my use case is for printing defect. So, the color anomaly is important. My understanding is that Geti takes RGB channel whereas OpenCV operates in BGR. Need to make BGR to RGB conversion before feeding the input for Geti model (the xml file has reverse_input_channels set to False).

Please also help to advise on where to make pre-processing channel conversion. Thanks.

[tbujewsk]

@ekaakurniawan there have been implemented many optimizations and fixes, could you please verify if you still see the issue?

[ekaakurniawan]

@tbujewsk , the result is still not correct using the following version.

$ sudo apt list --installed | grep intel-dlstreamer
intel-dlstreamer/unknown,now 2025.1.2 amd64 [installed]

Which version are you referring that has the fix? Has the #325 been implemented? Thanks

[tbujewsk]

Please use this version
docker pull intel/dlstreamer:2025.2-ubuntu24-rc1

[ekaakurniawan]

@tbujewsk , the result is still not correct. The pull request has not been implemented even in 2025.2rc2 and main branch.

Also, need to verify the following.

edge-ai-libraries/libraries/dl-streamer/src/monolithic/gst/inference_elements/base/inference_impl.cpp

Line 484 in d8498e9

int reverse_channels = 0; // TODO: verify that channel reversal works correctly with mean and std!