ffmpeg incorrect pal8 conversion

[therobin]

I was experimenting with some lossless archiving and I think I found a bug in utvideo which I cannot get it to losslessly reproduce some optimized 8-bit color PNG files on input. I was using Elephants Dream 1080p PNGs as the input source.

What I found is that UTVideo will not preserve 8-bit color input images losslessly, but will for 1-bit and 24-bit color images.* I verified by outputting results back to 24-bit BMP and checking the MD5. The UTVideo gives verified lossless output for frames sourced from 1-bit colorspace optimized PNGs and the 24-bit colorspace optimized PNGs. But the frames from PNGs that were 8-bit colorspace when optimized are giving invalid checksums when exported to 24-bit BMP.

I also tested FFV1 encoding to use a control to ensure the issue was not with the way ffmpeg was processing the input images.* All FFV1 frames can be verified lossless whether it was encoded from the original Xiph PNGs or the optimized PNGs. I also tested the latest release build and master branch build from gyan.

I am looking to see if anyone can replicate my findings. You can download the complete 21G source here: https://media.xiph.org/ED/ED-1080-png.tar
But since the problem occurs in the first 1 second, I have packaged the first 30 frames here (both original Xiph source along with Xiph's original MD5 file and my optimized versions): https://www.mediafire.com/file/knisxsozg41001l/Elephants_Dream_First_30_Frames.7z/file
If you want to test optimizing it yourself, I used oxipng with -D command:https://github.com/shssoichiro/oxipng/releases
I also used RapidCRC Unicode to check MD5s on windows: https://www.ov2.eu/programs/rapidcrc-unicode

To replicate my result:
1. Encode the 30 frames to FFV1 and UTVideo:

ffmpeg -i ED-30-xiph-source\%05d.png -c:v ffv1 ED-xiph_ffv1.mkv
ffmpeg -i ED-30-xiph-source\%05d.png -c:v utvideo ED-xiph_utvideo.mkv
ffmpeg -i ED-30-opt\%05d.png -pix_fmt bgr0 -c:v ffv1 ED-opt_ffv1.mkv
ffmpeg -i ED-30-opt\%05d.png -c:v utvideo ED-opt_utvideo.mkv

NOTE: The pix_fmt bgr0 command is necessary when encoding from the optimized files to prevent ffmpeg from encoding to greyscale since the first optimized PNG is 1-bit color. It is actually not necessary for UTvideo since it does support a grey or mono colorspace so will default to the appropriate pixel format.

2. Export all videos and PNGs to BMPs:

ffmpeg -i ED-30-xiph-source\%05d.png BMP\ED-xiph-%02d.bmp
ffmpeg -i ED-30-opt\%05d.png -pix_fmt bgr24 BMP\ED-opt-%02d.bmp
ffmpeg -i ED-xiph_ffv1.mkv BMP\ED-xiph_ffv1-%02d.bmp
ffmpeg -i ED-xiph_utvideo.mkv BMP\ED-xiph_utvideo-%02d.bmp
ffmpeg -i ED-opt_ffv1.mkv BMP\ED-opt_ffv1-%02d.bmp
ffmpeg -i ED-opt_utvideo.mkv BMP\ED_opt_utvideo-%02d.bmp

3. Perform checksum on all results.
I find all BMPs have the same MD5 checksum except:

ED_opt_utvideo-12.bmp 02a34fc1e347889d5546b547fa63e6af
ED_opt_utvideo-13.bmp ca40ed9fc4acc827a4a77051433f0102
ED_opt_utvideo-14.bmp e3666199f35234756dddac97b8e18488

The above are sourced from the 8-bit optimized PNGs.

The correct MD5 is as following:

ED-xiph_utvideo-12.bmp ba5d92ca99726d837859b4b05795d317
ED-xiph_utvideo-13.bmp 735ee7c8b493855480fd0f5847a66132
ED-xiph_utvideo-14.bmp b509c3ac24e3543e1f59a7f1456e924d

Which are sourced from the original Xiph PNGs that are all 24-bit.

*EDIT: I change the topic title to reflect the real error being pal8 in ffmpeg as to prevent spreading misinformation about UTVideo. See discussion.

[poisondeathray]

No issue here. Check your ffmpeg version

md5 matches for for original png converted to bmp sequence, and ut video converted to bmp sequence

Also, PSNR is inf for UT vs. original png sequence

[therobin]

No issue here. Check your ffmpeg version

md5 matches for for original png converted to bmp sequence, and ut video converted to bmp sequence

Also, PSNR is inf for UT vs. original png sequence

Could you tell me which build you are using? I tested three builds from gyan.

[poisondeathray]

Could you tell me which build you are using? I tested three builds from gyan.

I had been using ffmpeg-2021-09-05-git-a947098558-full_build from gyan, and retested with ffmpeg-2021-11-18-git-85a6b7f7b7-full_build , still no problems - psnr inf


ffmpeg -r 25 -i ED-xiph_utvideo_new.mkv -r 25 -i "%05d.png" -lavfi "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr="stats_file=ff_psnr_r25_new.log"" -f null -

[therobin]

I had been using ffmpeg-2021-09-05-git-a947098558-full_build from gyan, and retested with ffmpeg-2021-11-18-git-85a6b7f7b7-full_build , still no problems - psnr inf


ffmpeg -r 25 -i ED-xiph_utvideo_new.mkv -r 25 -i "%05d.png" -lavfi "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr="stats_file=ff_psnr_r25_new.log"" -f null -


Xiph_UTvideo is not the problem. Opt_Utvideo is the problem.

[poisondeathray]

sorry, I misread

Yes I can reproduce the issue

[poisondeathray]

It might have to do with monob to gbrp conversion; manually specifying -pix_fmt gbrp or -vf format=gbrp for the ut conversion does not help

ffmpeg -h encoder=utvideo
Supported pixel formats: gbrp gbrap yuv422p yuv420p yuv444p


If you use avisynth / utvideo vfw RGB encoding - psnr inf for the opt series compared to xiph source bmp seq

If you use avs input to ffmpeg (pixel format is already piped as bgr24, ImageSource decodes as RGB24), ffmpeg utvideo encoding (ffmpeg internal bgr24 to gbrp conversion), psnr inf for the opt series too compared to xiph source bmp seq

[poisondeathray]

If you convert the opt sequence to bgr0, then to gbrp, then utvideo is lossless . So those observations above combined with this one suggests the issue is ffmpeg internal pixel format conversions of pal8 sources (12,13,14 were pal8; others were monob and rgb24) to gbrp . If utvideo supported bgr0 like ffv1, then it should have worked "out of the box" too. I suggest you report the issue on the ffmpeg bug tracker


ffmpeg -i "PATH\ED-30-opt\%05d.png" -vf format=bgr0,format=gbrp -c:v utvideo ED-opt_bgr0_gbrp_utvideo.mkv

[Balling]

Is this a regression?

[therobin]

It might have to do with monob to gbrp conversion; manually specifying -pix_fmt gbrp or -vf format=gbrp for the ut conversion does not help

monob to gbrp is correct (First 11 frames from ED_opt_utvideo are valid). pal8 to gbrp is the problem (Frames 12-14 in the optimized sample set).

I submitted a ticket (https://trac.ffmpeg.org/ticket/9520). Turns out there is a decade old ticket regarding errors doing conversions TO pal8. Which I can confirm as well converting TO pal8 is also incorrect.

There may also be other formats that produce errors when converting from pal8. Right now we know:

incorrect: pal8 -> gbrp
correct: pal8 -> bgr0
correct: pal8 -> bgr0 -> gbrp
correct: pal8 -> pal8

incorrect: rgb24 -> pal8
incorrect: bgr24 -> pal8

I am not sure what downconversions if any to pal8 are correct.

[poisondeathray]

Dithering (and algorithm choice) might account for the differences observed in the the "something" to pal8 step between programs like irfanview (there were a bunch of dithering related bugs recently on ffmpeg bug tracker)

Disabling dithering should not help for the pal8 => gbrp step (but I tested it anyways)