Local Tone Mapping Output has Consistent Flicker

[ajaykumargurrala]

Hi,

I have generated the LTM IP for 10UC3 IN_TYPE and 8UC3 OUT_TYPE. The Max Resolution configured for 3840x2160 and BLOCK_ROWS and BLOCK_COLS set to 32.

I have added this to a camera pipeline to convert 10bit RGB to 8bit RGB. There are two issues 1) a consistent flicker in the output and 2) a dark input scene (say a scene capture with camera closed) results in bright greyish output.

I see that the block size has to be a power of 2 and should be >= 32 and I have ensured this. I have checked if the resolution needs to be an integral multiple of 32 or set block sizes but apparently found that this is not the case after checking the xf_ltm.hpp.

However, I have still tried maintaining the integral ratio between block size and input resolution. The results were persistent.

Am I missing out anything here? Or is this the expected behaviour?

I have attached video snippets for block size 32 and block size 512 for input and output resolution of 800x800 to LTM block.

Tool Version: Vitis HLS 2023.2, Vivado 2023.2 and Vitis Classic 2023.2
Libraries Version: Vision Libraries 2023.2

https://github.com/user-attachments/assets/226eed86-67bf-441e-beec-609826f116df
https://github.com/user-attachments/assets/556c7b89-4e3e-4929-815e-a63b5039fb1e

Thanks
Ajay

[vt-lib-support]

Hi Ajay

1. It is possible that when LTM is applied on a stream of images, there might be highlighted lines at the borders of the blocks. So, it can be considered as a known issue
2. Greyish output for a dark input is not expected.

As an alternative to LTM, we suggest using Quantaization & Dithering (QnD) or Global Tone Mapping (GTM) which produce seamless output for video streams.

[ajaykumargurrala]

Hi

1. I understand the possibility of highlights at the block borders. But that is not the case here. The output stream does not exhibit such a behavior. It has some consistent flicker, the cause for which is not clear.

2. What could be the potential cause for this behavior with dark input?

I have tried QnD and Global Tone Mapping. QnD seems to just perform bit truncation. GTM performs better. However, LTM has the best output quality of all the three. I am just not sure if the issues I have are specific to the algo or are a result of implementation errors.

Has this algorithm been tested in hardware? If yes, can you please share the settings applied for the HLS Synthesis and an output video snippet of the same. If not, can you please test it with the infra available with you (or a KV260 and any MIPI camera) and verify the behaviour.

Thanks,
Ajay.