QPA causes significant detail loss in static parts of video

[joletb]

While QPA gives quite good increases in fidelity in intra coding, it can cause static parts of a video to lose all detail in inter coding, even when there is a good reference frame available.

Furthermore, if we remove the first frame from our sample below, this detail loss is remedied and the encoder properly uses info from previous/next frames, signifying a possible issue with the GOP structure when used with QPA.

This issue does not happen at all in non-QPA encoding, but then detail preservation in general is quite a bit worse.

The following command line was used for the encoding:
vvencffapp -c config/slow --y4m 1 -q 20 -t -1 --PrintFrameMSE 1 --PrintSequenceMSE 1 --ALF 1 -ip 0 -rs 11 --CCALF 1 --STA 0 --WppBitEqual -1 --MaxParallelFrames -1 --SAO 0 --qpa 1 --BIM 0 --MCTF 0 --InputFile - -b slow.266

Here are a couple of comparisons: https://slow.pics/c/koHLpziy
Note also that this issue is significantly more prevalent in anime as there tend to be a lot of duplicate frames.

The encoded bitstreams and the video source used for encoding can be found here: https://drive.google.com/drive/folders/1OAsQxfd6y-Xl8u1syeXkZ9u22scQD0YG?usp=sharing

[jungleboynx]

I encoded the source in x264@CRF21 and h266@QP28 files but couldn't see any difference when playing back the files at the same time in single step mode:
https://www.dropbox.com/scl/fo/j6r4po3naxgsa5z6vk961/h?rlkey=1a0y3i488qdqt226zcssalxfb&dl=0

The video is a bit odd with alternative bright and dark frames in places. The clouds seem to move on every frame but the anime moves on alternative frames - maybe this is normal.

[joletb]

Sure, there's no difference... if you don't count the texture and lineart loss.
Although the main issue here is about QPA making the encoder mangle data instead of reusing it, which by the way, it does in your encode as well: https://slow.pics/c/dwOWsn5s

[adamjw24]

Thanks, we'll have a look.

[crhelmrich]

Hi joletb,

While QPA gives quite good ... fidelity in intra coding, it can cause static parts of a video to lose all detail in inter coding

Can you clarify what you mean by 'static'? The video scene features quite some motion, but 'static' to me means not moving.

This issue does not happen at all in non-QPA encoding, but then detail preservation in general is quite a bit worse.

So, is (to your eyes) the QPA version of the video still better overall than the non-QPA version when played at normal speed?

I have to admit that, when comparing the 'qpa/noqpa' bitstreams you shared (thanks), I cannot reliably spot any differences. Are your observations based on watching the video during playback, or just on inspecting isolated frames (i.e. screenshots)?

Note that VVenC's QP adaptation makes use of spatial as well as temporal psycho-visual modelling. It's based on the XPSNR visual model, details of which you can read up on in this open-access paper: http://handle.itu.int/11.1002/pub/8153d78b-en

This means that VVenC's QPA is solely intended, and optimized, for videos at typical frame rates, NOT for still image coding. Or, in other words, screenshots of individual frames don't tell the whole (perceptual) story.

That said, I'll take another look at the issue with the first frame in the video.

Christian

[joletb]

The "static" part here refers to the to the duplicate parts of the building in the video, there is practically no motion between them there and as such there should be no detail loss. (If we're going this route, should the detail loss be extended to the duplicate parts to make the video more consistent?)
This behavior can happen on scene changes as well, where, with enough of a low bitrate, can be very noticeable.

That being said, I don't think this is entirely a QPA issue as this does not happen when you remove the first frame of the sample. (Both the original sample and this one mentioned below are perceptually the same)
I've included the trimmed sample and an encode of it in the original GDrive link as CSM_test_clip_no_duplicate_first_frame_yuv420p10.y4m and slow-qpa-q20-no-duplicate.266, respectively.

[birdie-github]

I'm an absolute newbie in video encoding but to me this issue looks similar to #212.

[L4cache]

I'm an absolute newbie in video encoding but to me this issue looks similar to #212.

It's not.

[ZenKiyoshi]

I have noticed this before.
#133

#132

[L4cache]

Maybe the problem is the opposite: the subsequent frame of the "problematic" frame is actually getting too many quality.

Tested with the y4m clip provided by joletb ,arguments used:
--preset faster -ip -1 -dr idr_no_radl --poc0idr 1 --sao 0 --bim 0 --mctf 0 -t 16 --wavefrontsynchro 1 --ctusize 128 --ifp 0 --ciip 3 -qpa 1 --qp 20 -fs 1
(skip first frame makes the problem more significant in this case)

With QPA:

POC     0 TId: 0 (  IDR_N_LP, I-SLICE, QP 10)           846416 bits [Y 55.6109 dB    U 60.6117 dB    V 60.1662 dB] [ET     0 ] [L0 ] [L1 ]
POC    16 TId: 1 (      STSA, B-SLICE, QP 22)           410800 bits [Y 47.9555 dB    U 52.5970 dB    V 51.4652 dB] [ET     0 ] [L0 0 ] [L1 0 ]
POC     8 TId: 2 (      STSA, B-SLICE, QP 22)           405784 bits [Y 48.2472 dB    U 52.3015 dB    V 51.2012 dB] [ET     0 ] [L0 0 ] [L1 16 ]
POC     4 TId: 3 (      STSA, B-SLICE, QP 23)           289112 bits [Y 47.6080 dB    U 51.9036 dB    V 50.8292 dB] [ET     0 ] [L0 0 ] [L1 8 ]
POC     2 TId: 4 (      STSA, B-SLICE, QP 24)           182560 bits [Y 46.9975 dB    U 51.3339 dB    V 50.5454 dB] [ET     0 ] [L0 0 ] [L1 4 ]
POC     1 TId: 5 (      STSA, B-SLICE, QP 28)           130064 bits [Y 45.6190 dB    U 49.5797 dB    V 49.1447 dB] [ET     0 ] [L0 0 2 ] [L1 2 4 ]
POC     3 TId: 5 (      STSA, B-SLICE, QP 23)           155544 bits [Y 47.5512 dB    U 52.3980 dB    V 51.5035 dB] [ET     0 ] [L0 2 ] [L1 4 ]
POC     6 TId: 4 (      STSA, B-SLICE, QP 26)           230000 bits [Y 45.8785 dB    U 49.9708 dB    V 48.9032 dB] [ET     0 ] [L0 4 ] [L1 8 ]
POC     5 TId: 5 (      STSA, B-SLICE, QP 23)            82296 bits [Y 47.0603 dB    U 52.7286 dB    V 51.3601 dB] [ET     0 ] [L0 4 ] [L1 6 ]
POC     7 TId: 5 (      STSA, B-SLICE, QP 24)           115256 bits [Y 46.0015 dB    U 51.5641 dB    V 49.9041 dB] [ET     0 ] [L0 6 ] [L1 8 ]
POC    12 TId: 3 (      STSA, B-SLICE, QP 24)           312096 bits [Y 46.9459 dB    U 51.1315 dB    V 50.4386 dB] [ET     0 ] [L0 8 ] [L1 16 ]
POC    10 TId: 4 (      STSA, B-SLICE, QP 26)           239440 bits [Y 45.9486 dB    U 50.0011 dB    V 49.0176 dB] [ET     0 ] [L0 8 ] [L1 12 ]
POC     9 TId: 5 (      STSA, B-SLICE, QP 23)            56528 bits [Y 47.5988 dB    U 52.5054 dB    V 51.3064 dB] [ET     0 ] [L0 8 ] [L1 10 ]
POC    11 TId: 5 (      STSA, B-SLICE, QP 23)           118464 bits [Y 46.0833 dB    U 51.5704 dB    V 50.3236 dB] [ET     0 ] [L0 10 ] [L1 12 ]
POC    14 TId: 4 (      STSA, B-SLICE, QP 27)           243656 bits [Y 44.8992 dB    U 48.9751 dB    V 48.1517 dB] [ET     0 ] [L0 12 ] [L1 16 ]
POC    13 TId: 5 (      STSA, B-SLICE, QP 21)           120608 bits [Y 46.9868 dB    U 52.5304 dB    V 51.4956 dB] [ET     0 ] [L0 12 ] [L1 14 ]
POC    15 TId: 5 (      STSA, B-SLICE, QP 22)           258928 bits [Y 45.0721 dB    U 50.4006 dB    V 49.3436 dB] [ET     0 ] [L0 14 ] [L1 16 ]
POC    24 TId: 2 (      STSA, B-SLICE, QP 21)           379304 bits [Y 48.2156 dB    U 52.6109 dB    V 51.6196 dB] [ET     0 ] [L0 16 ] [L1 16 ]
POC    20 TId: 3 (      STSA, B-SLICE, QP 22)           295616 bits [Y 46.6235 dB    U 51.1552 dB    V 50.1717 dB] [ET     0 ] [L0 16 ] [L1 24 ]
POC    18 TId: 4 (      STSA, B-SLICE, QP 25)           223184 bits [Y 44.9037 dB    U 49.5202 dB    V 48.3823 dB] [ET     0 ] [L0 16 ] [L1 20 ]
POC    17 TId: 5 (      STSA, B-SLICE, QP 20)            97368 bits [Y 47.8897 dB    U 53.2187 dB    V 52.1750 dB] [ET     0 ] [L0 16 ] [L1 18 ]
POC    19 TId: 5 (      STSA, B-SLICE, QP 19)           221616 bits [Y 47.2766 dB    U 52.3488 dB    V 51.0845 dB] [ET     0 ] [L0 18 ] [L1 20 ]
POC    22 TId: 4 (      STSA, B-SLICE, QP 24)           235288 bits [Y 45.3710 dB    U 50.1320 dB    V 49.1783 dB] [ET     0 ] [L0 20 ] [L1 24 ]
POC    21 TId: 5 (      STSA, B-SLICE, QP 19)           128992 bits [Y 47.6393 dB    U 52.9384 dB    V 51.8818 dB] [ET     0 ] [L0 20 ] [L1 22 ]
vvenc [info]: stats:  51.1% frame= 24/47 fps=   4.8 avg_fps=   4.8 bitrate=  5773.15 kbps avg_bitrate=  5773.15 kbps elapsed= 00h:00m:05s left= 00h:00m:05s
POC    23 TId: 5 (      STSA, B-SLICE, QP 19)           202016 bits [Y 47.1692 dB    U 52.5115 dB    V 51.5600 dB] [ET     0 ] [L0 22 ] [L1 24 ]
POC    28 TId: 3 (      STSA, B-SLICE, QP 23)           267192 bits [Y 46.9249 dB    U 51.4836 dB    V 50.4660 dB] [ET     0 ] [L0 24 ] [L1 24 ]
POC    26 TId: 4 (      STSA, B-SLICE, QP 24)           207704 bits [Y 45.1315 dB    U 49.8533 dB    V 49.1088 dB] [ET     0 ] [L0 24 ] [L1 28 ]
POC    25 TId: 5 (      STSA, B-SLICE, QP 19)            81592 bits [Y 48.3754 dB    U 53.3255 dB    V 52.1993 dB] [ET     0 ] [L0 24 ] [L1 26 ]
POC    27 TId: 5 (      STSA, B-SLICE, QP 19)           205680 bits [Y 47.4046 dB    U 52.6119 dB    V 51.5899 dB] [ET     0 ] [L0 26 ] [L1 28 ]
POC    30 TId: 4 (      STSA, B-SLICE, QP 24)           200624 bits [Y 45.5273 dB    U 50.2978 dB    V 49.5801 dB] [ET     0 ] [L0 28 ] [L1 28 ]
POC    29 TId: 5 (      STSA, B-SLICE, QP 20)            88024 bits [Y 47.5471 dB    U 52.8556 dB    V 51.8869 dB] [ET     0 ] [L0 28 ] [L1 30 ]
POC    31 TId: 5 (      STSA, B-SLICE, QP 18)           195600 bits [Y 47.9366 dB    U 53.1944 dB    V 52.3387 dB] [ET     0 ] [L0 30 ] [L1 30 ]
POC    32 TId: 0 (     TRAIL, B-SLICE, QP 18)           551912 bits [Y 49.2734 dB    U 54.3663 dB    V 53.7849 dB] [ET     0 ] [L0 0 ] [L1 0 ]
POC    40 TId: 2 (      STSA, B-SLICE, QP 18)           322016 bits [Y 49.5046 dB    U 54.2149 dB    V 53.9277 dB] [ET     0 ] [L0 32 ] [L1 32 ]
POC    36 TId: 3 (      STSA, B-SLICE, QP 20)           208840 bits [Y 47.9946 dB    U 53.5233 dB    V 52.7678 dB] [ET     0 ] [L0 32 ] [L1 40 ]
POC    34 TId: 4 (      STSA, B-SLICE, QP 22)           164744 bits [Y 46.2822 dB    U 51.9022 dB    V 51.0221 dB] [ET     0 ] [L0 32 ] [L1 36 ]
POC    33 TId: 5 (      STSA, B-SLICE, QP 19)            79096 bits [Y 49.8151 dB    U 54.9774 dB    V 54.3195 dB] [ET     0 ] [L0 32 ] [L1 34 ]
POC    35 TId: 5 (      STSA, B-SLICE, QP 19)           180712 bits [Y 48.7268 dB    U 54.6767 dB    V 53.8849 dB] [ET     0 ] [L0 34 ] [L1 36 ]
POC    38 TId: 4 (      STSA, B-SLICE, QP 22)           145384 bits [Y 46.8786 dB    U 51.6810 dB    V 51.2991 dB] [ET     0 ] [L0 36 ] [L1 40 ]
POC    37 TId: 5 (      STSA, B-SLICE, QP 19)           102520 bits [Y 49.0579 dB    U 55.4315 dB    V 54.8085 dB] [ET     0 ] [L0 36 ] [L1 38 ]
POC    39 TId: 5 (      STSA, B-SLICE, QP 19)           152176 bits [Y 49.0307 dB    U 54.7695 dB    V 54.3352 dB] [ET     0 ] [L0 38 ] [L1 40 ]
POC    44 TId: 3 (      STSA, B-SLICE, QP 23)           320504 bits [Y 46.1009 dB    U 50.4451 dB    V 50.0183 dB] [ET     0 ] [L0 40 32 ] [L1 40 32 ]
POC    42 TId: 4 (      STSA, B-SLICE, QP 24)           195072 bits [Y 45.2652 dB    U 49.9770 dB    V 49.3828 dB] [ET     0 ] [L0 40 ] [L1 44 ]
POC    41 TId: 5 (      STSA, B-SLICE, QP 19)            88920 bits [Y 50.0731 dB    U 55.1672 dB    V 55.0347 dB] [ET     0 ] [L0 40 ] [L1 42 ]
POC    43 TId: 5 (      STSA, B-SLICE, QP 22)           177168 bits [Y 47.3166 dB    U 51.6071 dB    V 51.0253 dB] [ET     0 ] [L0 42 ] [L1 44 ]
POC    46 TId: 4 (      STSA, B-SLICE, QP 27)           254904 bits [Y 42.9397 dB    U 49.1937 dB    V 48.3834 dB] [ET     0 ] [L0 44 ] [L1 44 ]
POC    45 TId: 5 (      STSA, B-SLICE, QP 22)           125168 bits [Y 47.0075 dB    U 51.6647 dB    V 50.9808 dB] [ET     0 ] [L0 44 40 ] [L1 46 44 ]

Without QPA:

POC     0 TId: 0 (  IDR_N_LP, I-SLICE, QP 17)           430544 bits [Y 53.2183 dB    U 58.1984 dB    V 57.8556 dB] [ET     0 ] [L0 ] [L1 ]
POC    16 TId: 1 (      STSA, B-SLICE, QP 20)           485752 bits [Y 49.8717 dB    U 53.5044 dB    V 52.5491 dB] [ET     0 ] [L0 0 ] [L1 0 ]
POC     8 TId: 2 (      STSA, B-SLICE, QP 21)           422848 bits [Y 49.3465 dB    U 52.7126 dB    V 51.6260 dB] [ET     0 ] [L0 0 ] [L1 16 ]
POC     4 TId: 3 (      STSA, B-SLICE, QP 24)           268520 bits [Y 47.9754 dB    U 51.9189 dB    V 50.8579 dB] [ET     0 ] [L0 0 ] [L1 8 ]
POC     2 TId: 4 (      STSA, B-SLICE, QP 25)           187520 bits [Y 47.7614 dB    U 51.2123 dB    V 50.4113 dB] [ET     0 ] [L0 0 ] [L1 4 ]
POC     1 TId: 5 (      STSA, B-SLICE, QP 27)           153736 bits [Y 46.9146 dB    U 49.9578 dB    V 49.5134 dB] [ET     0 ] [L0 0 2 ] [L1 2 4 ]
POC     3 TId: 5 (      STSA, B-SLICE, QP 27)            81448 bits [Y 46.4660 dB    U 50.8436 dB    V 50.1669 dB] [ET     0 ] [L0 2 ] [L1 4 ]
POC     6 TId: 4 (      STSA, B-SLICE, QP 25)           241920 bits [Y 46.9277 dB    U 50.4038 dB    V 49.4034 dB] [ET     0 ] [L0 4 ] [L1 8 ]
POC     5 TId: 5 (      STSA, B-SLICE, QP 27)            44304 bits [Y 46.6058 dB    U 51.5392 dB    V 50.3779 dB] [ET     0 ] [L0 4 ] [L1 6 ]
POC     7 TId: 5 (      STSA, B-SLICE, QP 27)            41664 bits [Y 45.7391 dB    U 50.3303 dB    V 48.9971 dB] [ET     0 ] [L0 6 ] [L1 8 ]
POC    12 TId: 3 (      STSA, B-SLICE, QP 24)           297776 bits [Y 47.5161 dB    U 51.2791 dB    V 50.5252 dB] [ET     0 ] [L0 8 ] [L1 16 ]
POC    10 TId: 4 (      STSA, B-SLICE, QP 25)           245984 bits [Y 46.8500 dB    U 50.3075 dB    V 49.3512 dB] [ET     0 ] [L0 8 ] [L1 12 ]
POC     9 TId: 5 (      STSA, B-SLICE, QP 27)            23920 bits [Y 47.7560 dB    U 52.1199 dB    V 50.9459 dB] [ET     0 ] [L0 8 ] [L1 10 ]
POC    11 TId: 5 (      STSA, B-SLICE, QP 27)            44000 bits [Y 45.7408 dB    U 50.4018 dB    V 49.3620 dB] [ET     0 ] [L0 10 ] [L1 12 ]
POC    14 TId: 4 (      STSA, B-SLICE, QP 25)           276512 bits [Y 46.6408 dB    U 50.0468 dB    V 49.3003 dB] [ET     0 ] [L0 12 ] [L1 16 ]
POC    13 TId: 5 (      STSA, B-SLICE, QP 27)            26000 bits [Y 46.5308 dB    U 51.1114 dB    V 50.2015 dB] [ET     0 ] [L0 12 ] [L1 14 ]
POC    15 TId: 5 (      STSA, B-SLICE, QP 27)           142112 bits [Y 45.6298 dB    U 49.6385 dB    V 48.7745 dB] [ET     0 ] [L0 14 ] [L1 16 ]
POC    24 TId: 2 (      STSA, B-SLICE, QP 21)           385904 bits [Y 49.7278 dB    U 53.2208 dB    V 52.3298 dB] [ET     0 ] [L0 16 ] [L1 16 ]
POC    20 TId: 3 (      STSA, B-SLICE, QP 24)           283608 bits [Y 47.7869 dB    U 51.5517 dB    V 50.7771 dB] [ET     0 ] [L0 16 ] [L1 24 ]
POC    18 TId: 4 (      STSA, B-SLICE, QP 25)           256120 bits [Y 46.8618 dB    U 50.5699 dB    V 49.6870 dB] [ET     0 ] [L0 16 ] [L1 20 ]
POC    17 TId: 5 (      STSA, B-SLICE, QP 27)            13472 bits [Y 48.7428 dB    U 52.7076 dB    V 51.8251 dB] [ET     0 ] [L0 16 ] [L1 18 ]
POC    19 TId: 5 (      STSA, B-SLICE, QP 27)             8648 bits [Y 46.7241 dB    U 50.5560 dB    V 49.6871 dB] [ET     0 ] [L0 18 ] [L1 20 ]
POC    22 TId: 4 (      STSA, B-SLICE, QP 25)           242240 bits [Y 46.9334 dB    U 50.6733 dB    V 49.8129 dB] [ET     0 ] [L0 20 ] [L1 24 ]
POC    21 TId: 5 (      STSA, B-SLICE, QP 27)             3352 bits [Y 47.6381 dB    U 51.4968 dB    V 50.7518 dB] [ET     0 ] [L0 20 ] [L1 22 ]
vvenc [info]: stats:  51.1% frame= 24/47 fps=   6.1 avg_fps=   6.1 bitrate=  4603.30 kbps avg_bitrate=  4603.30 kbps elapsed= 00h:00m:04s left= 00h:00m:04s
POC    23 TId: 5 (      STSA, B-SLICE, QP 27)             2800 bits [Y 46.8888 dB    U 50.7396 dB    V 49.8795 dB] [ET     0 ] [L0 22 ] [L1 24 ]
POC    28 TId: 3 (      STSA, B-SLICE, QP 24)           262632 bits [Y 48.2926 dB    U 51.9777 dB    V 51.1979 dB] [ET     0 ] [L0 24 ] [L1 24 ]
POC    26 TId: 4 (      STSA, B-SLICE, QP 25)           228776 bits [Y 47.0083 dB    U 50.8350 dB    V 50.0467 dB] [ET     0 ] [L0 24 ] [L1 28 ]
POC    25 TId: 5 (      STSA, B-SLICE, QP 27)             9640 bits [Y 49.2313 dB    U 52.8189 dB    V 51.9627 dB] [ET     0 ] [L0 24 ] [L1 26 ]
POC    27 TId: 5 (      STSA, B-SLICE, QP 27)             3832 bits [Y 46.9765 dB    U 50.8188 dB    V 50.0536 dB] [ET     0 ] [L0 26 ] [L1 28 ]
POC    30 TId: 4 (      STSA, B-SLICE, QP 25)           223616 bits [Y 47.3654 dB    U 51.0124 dB    V 50.4057 dB] [ET     0 ] [L0 28 ] [L1 28 ]
POC    29 TId: 5 (      STSA, B-SLICE, QP 27)            14384 bits [Y 47.8390 dB    U 51.5298 dB    V 51.0538 dB] [ET     0 ] [L0 28 ] [L1 30 ]
POC    31 TId: 5 (      STSA, B-SLICE, QP 27)             6264 bits [Y 47.2566 dB    U 51.0768 dB    V 50.4065 dB] [ET     0 ] [L0 30 ] [L1 30 ]
POC    32 TId: 0 (     TRAIL, B-SLICE, QP 19)           512360 bits [Y 50.7091 dB    U 55.4304 dB    V 54.8389 dB] [ET     0 ] [L0 0 ] [L1 0 ]
POC    40 TId: 2 (      STSA, B-SLICE, QP 21)           263560 bits [Y 50.0798 dB    U 54.2828 dB    V 53.9589 dB] [ET     0 ] [L0 32 ] [L1 32 ]
POC    36 TId: 3 (      STSA, B-SLICE, QP 24)           167304 bits [Y 48.3740 dB    U 53.4674 dB    V 52.8648 dB] [ET     0 ] [L0 32 ] [L1 40 ]
POC    34 TId: 4 (      STSA, B-SLICE, QP 25)           157424 bits [Y 47.3372 dB    U 52.2220 dB    V 51.7003 dB] [ET     0 ] [L0 32 ] [L1 36 ]
POC    33 TId: 5 (      STSA, B-SLICE, QP 27)             3480 bits [Y 50.1172 dB    U 55.0982 dB    V 54.5577 dB] [ET     0 ] [L0 32 ] [L1 34 ]
POC    35 TId: 5 (      STSA, B-SLICE, QP 27)             4512 bits [Y 47.2506 dB    U 52.2257 dB    V 51.7575 dB] [ET     0 ] [L0 34 ] [L1 36 ]
POC    38 TId: 4 (      STSA, B-SLICE, QP 25)           137048 bits [Y 47.8498 dB    U 52.4866 dB    V 52.0673 dB] [ET     0 ] [L0 36 ] [L1 40 ]
POC    37 TId: 5 (      STSA, B-SLICE, QP 27)             4584 bits [Y 48.1655 dB    U 53.5156 dB    V 52.9864 dB] [ET     0 ] [L0 36 ] [L1 38 ]
POC    39 TId: 5 (      STSA, B-SLICE, QP 27)             5896 bits [Y 47.6826 dB    U 52.5857 dB    V 52.1876 dB] [ET     0 ] [L0 38 ] [L1 40 ]
POC    44 TId: 3 (      STSA, B-SLICE, QP 24)           305800 bits [Y 47.8379 dB    U 51.2305 dB    V 50.9995 dB] [ET     1 ] [L0 40 32 ] [L1 40 32 ]
POC    42 TId: 4 (      STSA, B-SLICE, QP 25)           207312 bits [Y 47.3572 dB    U 51.1899 dB    V 50.6637 dB] [ET     0 ] [L0 40 ] [L1 44 ]
POC    41 TId: 5 (      STSA, B-SLICE, QP 27)            10056 bits [Y 49.4899 dB    U 53.6706 dB    V 53.4416 dB] [ET     0 ] [L0 40 ] [L1 42 ]
POC    43 TId: 5 (      STSA, B-SLICE, QP 27)            63456 bits [Y 46.8891 dB    U 50.4590 dB    V 49.9426 dB] [ET     0 ] [L0 42 ] [L1 44 ]
POC    46 TId: 4 (      STSA, B-SLICE, QP 25)           327392 bits [Y 46.3278 dB    U 50.5272 dB    V 49.9832 dB] [ET     0 ] [L0 44 ] [L1 44 ]
POC    45 TId: 5 (      STSA, B-SLICE, QP 27)            43880 bits [Y 47.1070 dB    U 50.5357 dB    V 50.1573 dB] [ET     0 ] [L0 44 40 ] [L1 46 44 ]

Notice how frame 17, 19, 21 and so on jumped from QP 27-ish to 19-ish between two samples while other frames are less significantly changed.

[crhelmrich]

Thanks, L4cache, for the detailed analysis! In case you have a hex editor available and know how to use it: could you please

1. open the y4m input file you have used above in the hex editor and change the first few characters from
   YUV4MPEG2 C420p10 W1920 H1080 F24000:1001. . . to YUV4MPEG2 C420p10 W1920 H1080 F60000:1001. . .,
2. save the result in a new file, QPA-encode that with the same settings and check the per-frame QP data?

This effectively changes the frame rate from 23.976 to 59.94 fps for testing and should help to isolate the problem. Thanks,

Christian

[L4cache]

To make things more clear, I'm using the original file CSM_test_clip_yuv420p10.y4m, not the later uploaded one, I didn't download it but I think it's functionally the same with -fs 1.

The 59.94 fps reports as follows

POC     0 TId: 0 (  IDR_N_LP, I-SLICE, QP 10)           846416 bits [Y 55.6109 dB    U 60.6117 dB    V 60.1662 dB] [ET     0 ] [L0 ] [L1 ]
POC    16 TId: 1 (      STSA, B-SLICE, QP 22)           417768 bits [Y 48.2431 dB    U 52.7842 dB    V 51.7518 dB] [ET     0 ] [L0 0 ] [L1 0 ]
POC     8 TId: 2 (      STSA, B-SLICE, QP 21)           454584 bits [Y 48.8063 dB    U 52.9376 dB    V 51.8077 dB] [ET     0 ] [L0 0 ] [L1 16 ]
POC     4 TId: 3 (      STSA, B-SLICE, QP 22)           335592 bits [Y 48.3010 dB    U 52.7306 dB    V 51.5995 dB] [ET     0 ] [L0 0 ] [L1 8 ]
POC     2 TId: 4 (      STSA, B-SLICE, QP 27)           160688 bits [Y 46.3129 dB    U 50.6753 dB    V 49.7035 dB] [ET     0 ] [L0 0 ] [L1 4 ]
POC     1 TId: 5 (      STSA, B-SLICE, QP 28)           131896 bits [Y 45.5920 dB    U 49.5847 dB    V 48.9226 dB] [ET     0 ] [L0 0 2 ] [L1 2 4 ]
POC     3 TId: 5 (      STSA, B-SLICE, QP 23)           140664 bits [Y 46.4539 dB    U 51.9440 dB    V 50.9965 dB] [ET     0 ] [L0 2 ] [L1 4 ]
POC     6 TId: 4 (      STSA, B-SLICE, QP 24)           269008 bits [Y 46.7617 dB    U 50.7232 dB    V 49.6592 dB] [ET     0 ] [L0 4 ] [L1 8 ]
POC     5 TId: 5 (      STSA, B-SLICE, QP 24)            64128 bits [Y 46.9640 dB    U 52.5232 dB    V 51.2525 dB] [ET     0 ] [L0 4 ] [L1 6 ]
POC     7 TId: 5 (      STSA, B-SLICE, QP 25)            62112 bits [Y 45.7728 dB    U 50.8016 dB    V 49.4913 dB] [ET     0 ] [L0 6 ] [L1 8 ]
POC    12 TId: 3 (      STSA, B-SLICE, QP 23)           358920 bits [Y 47.6277 dB    U 51.7516 dB    V 51.0343 dB] [ET     0 ] [L0 8 ] [L1 16 ]
POC    10 TId: 4 (      STSA, B-SLICE, QP 24)           275736 bits [Y 46.7379 dB    U 50.7964 dB    V 49.7902 dB] [ET     0 ] [L0 8 ] [L1 12 ]
POC     9 TId: 5 (      STSA, B-SLICE, QP 26)            32960 bits [Y 47.4694 dB    U 52.4524 dB    V 51.2320 dB] [ET     0 ] [L0 8 ] [L1 10 ]
POC    11 TId: 5 (      STSA, B-SLICE, QP 25)            60960 bits [Y 45.7882 dB    U 50.8931 dB    V 49.9163 dB] [ET     0 ] [L0 10 ] [L1 12 ]
POC    14 TId: 4 (      STSA, B-SLICE, QP 26)           266936 bits [Y 45.5909 dB    U 49.6675 dB    V 48.7914 dB] [ET     0 ] [L0 12 ] [L1 16 ]
POC    13 TId: 5 (      STSA, B-SLICE, QP 26)            34976 bits [Y 46.6737 dB    U 51.5714 dB    V 50.7319 dB] [ET     0 ] [L0 12 ] [L1 14 ]
POC    15 TId: 5 (      STSA, B-SLICE, QP 28)           122304 bits [Y 44.1318 dB    U 49.0740 dB    V 48.0811 dB] [ET     0 ] [L0 14 ] [L1 16 ]
POC    24 TId: 2 (      STSA, B-SLICE, QP 19)           433872 bits [Y 48.9951 dB    U 53.2815 dB    V 52.3878 dB] [ET     0 ] [L0 16 ] [L1 16 ]
POC    20 TId: 3 (      STSA, B-SLICE, QP 21)           335392 bits [Y 47.3971 dB    U 51.7798 dB    V 50.9777 dB] [ET     0 ] [L0 16 ] [L1 24 ]
POC    18 TId: 4 (      STSA, B-SLICE, QP 24)           252528 bits [Y 45.6840 dB    U 50.0415 dB    V 49.0722 dB] [ET     0 ] [L0 16 ] [L1 20 ]
POC    17 TId: 5 (      STSA, B-SLICE, QP 27)            17952 bits [Y 47.4932 dB    U 52.2680 dB    V 51.3135 dB] [ET     0 ] [L0 16 ] [L1 18 ]
POC    19 TId: 5 (      STSA, B-SLICE, QP 25)            13896 bits [Y 45.5879 dB    U 50.3274 dB    V 49.2577 dB] [ET     0 ] [L0 18 ] [L1 20 ]
POC    22 TId: 4 (      STSA, B-SLICE, QP 23)           264272 bits [Y 46.2046 dB    U 50.8005 dB    V 49.7762 dB] [ET     0 ] [L0 20 ] [L1 24 ]
POC    21 TId: 5 (      STSA, B-SLICE, QP 24)             8744 bits [Y 47.2945 dB    U 51.8468 dB    V 50.9791 dB] [ET     0 ] [L0 20 ] [L1 22 ]
POC    23 TId: 5 (      STSA, B-SLICE, QP 24)            10648 bits [Y 46.1858 dB    U 51.0102 dB    V 49.9896 dB] [ET     0 ] [L0 22 ] [L1 24 ]
POC    28 TId: 3 (      STSA, B-SLICE, QP 21)           310552 bits [Y 47.8112 dB    U 52.2543 dB    V 51.3842 dB] [ET     0 ] [L0 24 ] [L1 24 ]
POC    26 TId: 4 (      STSA, B-SLICE, QP 23)           242816 bits [Y 46.0092 dB    U 50.6642 dB    V 49.7180 dB] [ET     0 ] [L0 24 ] [L1 28 ]
POC    25 TId: 5 (      STSA, B-SLICE, QP 24)            21776 bits [Y 48.6274 dB    U 52.9685 dB    V 52.1410 dB] [ET     0 ] [L0 24 ] [L1 26 ]
POC    27 TId: 5 (      STSA, B-SLICE, QP 24)            16600 bits [Y 46.0051 dB    U 50.7933 dB    V 49.9122 dB] [ET     0 ] [L0 26 ] [L1 28 ]
POC    30 TId: 4 (      STSA, B-SLICE, QP 23)           226040 bits [Y 46.3681 dB    U 51.0292 dB    V 50.0967 dB] [ET     0 ] [L0 28 ] [L1 28 ]
POC    29 TId: 5 (      STSA, B-SLICE, QP 24)            28888 bits [Y 47.4146 dB    U 52.0181 dB    V 51.3057 dB] [ET     0 ] [L0 28 ] [L1 30 ]
POC    31 TId: 5 (      STSA, B-SLICE, QP 24)            11040 bits [Y 46.3451 dB    U 51.2104 dB    V 50.1856 dB] [ET     0 ] [L0 30 ] [L1 30 ]
POC    32 TId: 0 (     TRAIL, B-SLICE, QP 17)           623608 bits [Y 50.0547 dB    U 55.2409 dB    V 54.6955 dB] [ET     0 ] [L0 0 ] [L1 0 ]
POC    40 TId: 2 (      STSA, B-SLICE, QP 17)           367440 bits [Y 50.1177 dB    U 54.9260 dB    V 54.4741 dB] [ET     0 ] [L0 32 ] [L1 32 ]
POC    36 TId: 3 (      STSA, B-SLICE, QP 19)           238640 bits [Y 48.6593 dB    U 54.2452 dB    V 53.4073 dB] [ET     0 ] [L0 32 ] [L1 40 ]
POC    34 TId: 4 (      STSA, B-SLICE, QP 21)           191120 bits [Y 47.0548 dB    U 52.2748 dB    V 51.6382 dB] [ET     0 ] [L0 32 ] [L1 36 ]
POC    33 TId: 5 (      STSA, B-SLICE, QP 25)             9992 bits [Y 49.5208 dB    U 54.9424 dB    V 54.4474 dB] [ET     0 ] [L0 32 ] [L1 34 ]
POC    35 TId: 5 (      STSA, B-SLICE, QP 22)            14288 bits [Y 46.9742 dB    U 52.5735 dB    V 51.9073 dB] [ET     0 ] [L0 34 ] [L1 36 ]
POC    38 TId: 4 (      STSA, B-SLICE, QP 21)           165152 bits [Y 47.5891 dB    U 52.4326 dB    V 52.0483 dB] [ET     0 ] [L0 36 ] [L1 40 ]
POC    37 TId: 5 (      STSA, B-SLICE, QP 23)            13968 bits [Y 48.4364 dB    U 54.2124 dB    V 53.5265 dB] [ET     0 ] [L0 36 ] [L1 38 ]
POC    39 TId: 5 (      STSA, B-SLICE, QP 22)            17088 bits [Y 47.4661 dB    U 52.5431 dB    V 52.3188 dB] [ET     0 ] [L0 38 ] [L1 40 ]
POC    44 TId: 3 (      STSA, B-SLICE, QP 22)           349144 bits [Y 47.0165 dB    U 51.2898 dB    V 50.7982 dB] [ET     0 ] [L0 40 32 ] [L1 40 32 ]
POC    42 TId: 4 (      STSA, B-SLICE, QP 23)           218960 bits [Y 46.0866 dB    U 50.8193 dB    V 50.2444 dB] [ET     0 ] [L0 40 ] [L1 44 ]
POC    41 TId: 5 (      STSA, B-SLICE, QP 22)            24912 bits [Y 49.6042 dB    U 54.4916 dB    V 54.1490 dB] [ET     0 ] [L0 40 ] [L1 42 ]
POC    43 TId: 5 (      STSA, B-SLICE, QP 25)            85312 bits [Y 45.8683 dB    U 50.5941 dB    V 49.6878 dB] [ET     0 ] [L0 42 ] [L1 44 ]
POC    46 TId: 4 (      STSA, B-SLICE, QP 26)           296848 bits [Y 44.1138 dB    U 49.8165 dB    V 49.0596 dB] [ET     0 ] [L0 44 ] [L1 44 ]
POC    45 TId: 5 (      STSA, B-SLICE, QP 26)            53112 bits [Y 46.3283 dB    U 50.5932 dB    V 50.1579 dB] [ET     0 ] [L0 44 40 ] [L1 46 44 ]

But... it decodes with an error (both ffvvc and vvdec), in frame 32, it seems like the error is caused by -dr idr_no_radl, so here's the report with changed command-line:
--preset faster -ip -1 --poc0idr 1 --sao 0 --bim 0 --mctf 0 -t 16 --wavefrontsynchro 1 --ctusize 128 --ifp 0 --ciip 3 -qpa 1 --qp 20 -fs 1
It moved frame 32 to second (encode order) and the rest is mostly the same except the bit cost and PSNR, and the decoding error solved!

POC     0 TId: 0 (  IDR_N_LP, I-SLICE, QP 10)           846416 bits [Y 55.6109 dB    U 60.6117 dB    V 60.1662 dB] [ET     0 ] [L0 ] [L1 ]
POC    32 TId: 0 (     TRAIL, B-SLICE, QP 17)           623368 bits [Y 50.0492 dB    U 55.2409 dB    V 54.6856 dB] [ET     0 ] [L0 0 ] [L1 0 ]
POC    16 TId: 1 (      STSA, B-SLICE, QP 22)           403768 bits [Y 48.2369 dB    U 52.8679 dB    V 51.7348 dB] [ET     0 ] [L0 0 ] [L1 32 ]
POC     8 TId: 2 (      STSA, B-SLICE, QP 21)           454344 bits [Y 48.8533 dB    U 52.9276 dB    V 51.7803 dB] [ET     0 ] [L0 0 ] [L1 16 ]
POC     4 TId: 3 (      STSA, B-SLICE, QP 22)           336512 bits [Y 48.3255 dB    U 52.6367 dB    V 51.5535 dB] [ET     0 ] [L0 0 ] [L1 8 ]
POC     2 TId: 4 (      STSA, B-SLICE, QP 27)           155808 bits [Y 46.4918 dB    U 50.5329 dB    V 49.7370 dB] [ET     0 ] [L0 0 ] [L1 4 ]
POC     1 TId: 5 (      STSA, B-SLICE, QP 28)           131320 bits [Y 45.6606 dB    U 49.4500 dB    V 49.0300 dB] [ET     0 ] [L0 0 2 ] [L1 2 4 ]
POC     3 TId: 5 (      STSA, B-SLICE, QP 23)           138672 bits [Y 46.4400 dB    U 51.8202 dB    V 50.9914 dB] [ET     0 ] [L0 2 ] [L1 4 ]
POC     6 TId: 4 (      STSA, B-SLICE, QP 24)           268872 bits [Y 46.7786 dB    U 50.6420 dB    V 49.5962 dB] [ET     0 ] [L0 4 ] [L1 8 ]
POC     5 TId: 5 (      STSA, B-SLICE, QP 24)            64744 bits [Y 46.9892 dB    U 52.5214 dB    V 51.2183 dB] [ET     0 ] [L0 4 ] [L1 6 ]
POC     7 TId: 5 (      STSA, B-SLICE, QP 25)            62448 bits [Y 45.7747 dB    U 50.7248 dB    V 49.4481 dB] [ET     0 ] [L0 6 ] [L1 8 ]
POC    12 TId: 3 (      STSA, B-SLICE, QP 23)           354992 bits [Y 47.6684 dB    U 51.8519 dB    V 50.9932 dB] [ET     0 ] [L0 8 ] [L1 16 ]
POC    10 TId: 4 (      STSA, B-SLICE, QP 24)           277160 bits [Y 46.8051 dB    U 50.8374 dB    V 49.8839 dB] [ET     0 ] [L0 8 ] [L1 12 ]
POC     9 TId: 5 (      STSA, B-SLICE, QP 26)            33168 bits [Y 47.5210 dB    U 52.4773 dB    V 51.2146 dB] [ET     0 ] [L0 8 ] [L1 10 ]
POC    11 TId: 5 (      STSA, B-SLICE, QP 25)            62048 bits [Y 45.8624 dB    U 50.9564 dB    V 49.9489 dB] [ET     0 ] [L0 10 ] [L1 12 ]
POC    14 TId: 4 (      STSA, B-SLICE, QP 26)           268592 bits [Y 45.6074 dB    U 49.8145 dB    V 48.8690 dB] [ET     0 ] [L0 12 ] [L1 16 ]
POC    13 TId: 5 (      STSA, B-SLICE, QP 26)            34712 bits [Y 46.6878 dB    U 51.6670 dB    V 50.7286 dB] [ET     0 ] [L0 12 ] [L1 14 ]
POC    15 TId: 5 (      STSA, B-SLICE, QP 28)           121872 bits [Y 44.1954 dB    U 49.2108 dB    V 48.1351 dB] [ET     0 ] [L0 14 ] [L1 16 ]
POC    24 TId: 2 (      STSA, B-SLICE, QP 19)           420400 bits [Y 49.0538 dB    U 53.3781 dB    V 52.2972 dB] [ET     0 ] [L0 16 ] [L1 32 ]
POC    20 TId: 3 (      STSA, B-SLICE, QP 21)           331152 bits [Y 47.4127 dB    U 51.9657 dB    V 50.8831 dB] [ET     0 ] [L0 16 ] [L1 24 ]
POC    18 TId: 4 (      STSA, B-SLICE, QP 24)           249848 bits [Y 45.6914 dB    U 50.1642 dB    V 49.0045 dB] [ET     0 ] [L0 16 ] [L1 20 ]
POC    17 TId: 5 (      STSA, B-SLICE, QP 27)            17632 bits [Y 47.4971 dB    U 52.3274 dB    V 51.2454 dB] [ET     0 ] [L0 16 ] [L1 18 ]
POC    19 TId: 5 (      STSA, B-SLICE, QP 25)            13544 bits [Y 45.5987 dB    U 50.3947 dB    V 49.1951 dB] [ET     0 ] [L0 18 ] [L1 20 ]
POC    22 TId: 4 (      STSA, B-SLICE, QP 23)           264912 bits [Y 46.2070 dB    U 50.9158 dB    V 49.8489 dB] [ET     0 ] [L0 20 ] [L1 24 ]
POC    21 TId: 5 (      STSA, B-SLICE, QP 24)             8888 bits [Y 47.3088 dB    U 52.0013 dB    V 50.9669 dB] [ET     0 ] [L0 20 ] [L1 22 ]
POC    23 TId: 5 (      STSA, B-SLICE, QP 24)            10800 bits [Y 46.1868 dB    U 51.0704 dB    V 49.9587 dB] [ET     0 ] [L0 22 ] [L1 24 ]
POC    28 TId: 3 (      STSA, B-SLICE, QP 21)           282144 bits [Y 48.0095 dB    U 52.3764 dB    V 51.4839 dB] [ET     0 ] [L0 24 ] [L1 32 ]
POC    26 TId: 4 (      STSA, B-SLICE, QP 23)           244040 bits [Y 46.2076 dB    U 50.6365 dB    V 49.7232 dB] [ET     0 ] [L0 24 ] [L1 28 ]
POC    25 TId: 5 (      STSA, B-SLICE, QP 24)            21808 bits [Y 48.7043 dB    U 53.0521 dB    V 52.0842 dB] [ET     0 ] [L0 24 ] [L1 26 ]
POC    27 TId: 5 (      STSA, B-SLICE, QP 24)            18592 bits [Y 46.2001 dB    U 50.7753 dB    V 49.8590 dB] [ET     0 ] [L0 26 ] [L1 28 ]
POC    30 TId: 4 (      STSA, B-SLICE, QP 23)           197728 bits [Y 47.0115 dB    U 51.1382 dB    V 50.2935 dB] [ET     0 ] [L0 28 ] [L1 32 ]
POC    29 TId: 5 (      STSA, B-SLICE, QP 24)            28688 bits [Y 47.6179 dB    U 52.0992 dB    V 51.3438 dB] [ET     0 ] [L0 28 ] [L1 30 ]
POC    31 TId: 5 (      STSA, B-SLICE, QP 24)             9632 bits [Y 46.9441 dB    U 51.2662 dB    V 50.4237 dB] [ET     0 ] [L0 30 ] [L1 32 ]
POC    40 TId: 2 (      STSA, B-SLICE, QP 17)           363944 bits [Y 50.1363 dB    U 54.8992 dB    V 54.5949 dB] [ET     0 ] [L0 32 ] [L1 32 ]
POC    36 TId: 3 (      STSA, B-SLICE, QP 19)           243640 bits [Y 48.6153 dB    U 54.3134 dB    V 53.5017 dB] [ET     0 ] [L0 32 ] [L1 40 ]
POC    34 TId: 4 (      STSA, B-SLICE, QP 21)           192176 bits [Y 47.0590 dB    U 52.2053 dB    V 51.6733 dB] [ET     0 ] [L0 32 ] [L1 36 ]
POC    33 TId: 5 (      STSA, B-SLICE, QP 25)             9944 bits [Y 49.5105 dB    U 54.9476 dB    V 54.4346 dB] [ET     0 ] [L0 32 ] [L1 34 ]
POC    35 TId: 5 (      STSA, B-SLICE, QP 22)            14368 bits [Y 47.0074 dB    U 52.5142 dB    V 52.0417 dB] [ET     0 ] [L0 34 ] [L1 36 ]
POC    38 TId: 4 (      STSA, B-SLICE, QP 21)           164712 bits [Y 47.6078 dB    U 52.3046 dB    V 52.0812 dB] [ET     0 ] [L0 36 ] [L1 40 ]
POC    37 TId: 5 (      STSA, B-SLICE, QP 23)            13880 bits [Y 48.3947 dB    U 54.2653 dB    V 53.5719 dB] [ET     0 ] [L0 36 ] [L1 38 ]
POC    39 TId: 5 (      STSA, B-SLICE, QP 22)            17608 bits [Y 47.4957 dB    U 52.4954 dB    V 52.3227 dB] [ET     0 ] [L0 38 ] [L1 40 ]
POC    44 TId: 3 (      STSA, B-SLICE, QP 22)           353328 bits [Y 47.0015 dB    U 51.2874 dB    V 50.6600 dB] [ET     0 ] [L0 40 32 ] [L1 40 32 ]
POC    42 TId: 4 (      STSA, B-SLICE, QP 23)           215440 bits [Y 46.0813 dB    U 50.7515 dB    V 50.1705 dB] [ET     0 ] [L0 40 ] [L1 44 ]
POC    41 TId: 5 (      STSA, B-SLICE, QP 22)            25456 bits [Y 49.5983 dB    U 54.4974 dB    V 54.2019 dB] [ET     0 ] [L0 40 ] [L1 42 ]
POC    43 TId: 5 (      STSA, B-SLICE, QP 25)            81712 bits [Y 45.8102 dB    U 50.3893 dB    V 49.5247 dB] [ET     0 ] [L0 42 ] [L1 44 ]
POC    46 TId: 4 (      STSA, B-SLICE, QP 26)           289600 bits [Y 44.2769 dB    U 49.8581 dB    V 49.1816 dB] [ET     0 ] [L0 44 ] [L1 44 ]
POC    45 TId: 5 (      STSA, B-SLICE, QP 26)            53120 bits [Y 46.3434 dB    U 50.5465 dB    V 49.8478 dB] [ET     0 ] [L0 44 40 ] [L1 46 44 ]

[crhelmrich]

Great, thanks! With those statistics, I can confirm that there are 2 separate issues here: one at frame 0 (reported by joletb), and one at frames 17, 19, 21, ... (reported by L4cache). The second one is relatively easy to fix and I'll start working on that soon. The first one is a bit harder to fix, so will take some time.

However, the fix to the second issue will require MCTF to be enabled (to have access to certain picture statistics). @joletb,

is there a particular reason why you chose to set --MCTF 0? This is an adaptive filter, which should be visually transparent even on noiseless content such as this anime clip. If it's picture blurring related, see also #406 to reduce the MCTF effect.

Christian