DNX120 indexing is broken in LWLibav, FFMpegSource2, DirectShowSource(LAV), FFPlay

[FranceBB]

Hi there,
DNX120 indexing is broken in:

- LWLibavVideoSource
- FFVideoSource
- DirectShowSource (LAV)
- FFMpeg
- Davinci Resolve

The following players also fail to decode it properly:

- MPV
- MPC-HC (LAV)
- PotPlayer


The following software decode it correctly:

- AVID Media Composer
- AVID Media Director
- Adobe Premiere
- EVS Hardware Playout Ports


Here's the mediainfo:

https://i.imgur.com/JFQWYwr.png
https://i.imgur.com/Jlr9sBX.png


This is how it's indexed:

https://i.imgur.com/UdEp4Nt.png

And this is how it should be instead:

https://i.imgur.com/dfPB74v.png

How to reproduce:


[FranceBB@router-localhost ~]$ ffplay -i "/home/FranceBB/Downloads/20210429WiganvHullFCX_C921h25m29s09.mxf"
ffplay version 4.4 Copyright (c) 2003-2021 the FFmpeg developers
built with gcc 11 (GCC)
configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2
-flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
-fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed
-Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld ' --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3
--enable-bzlib --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libaom --enable-libdav1d --enable-libass --enable-libbluray --enable-libcdio
--enable-libdrm --enable-libjack --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-nvenc --enable-openal --enable-opencl --enable-opengl
--enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librav1e --enable-libsmbclient --enable-version3 --enable-libsoxr --enable-libspeex --enable-libsrt
--enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-version3 --enable-vapoursynth --enable-libvpx --enable-vulkan
--enable-libglslang --enable-libx264 --enable-libx265 --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzvbi --enable-lv2 --enable-avfilter --enable-avresample --enable-libmodplug
--enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-lto --enable-libmfx --enable-runtime-cpudetect
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
Input #0, mxf, from '/home/FranceBB/Downloads/20210429WiganvHullFCX_C921h25m29s09.mxf':
Metadata:
operational_pattern_ul: 060e2b34.04010101.0d010201.01010100
application_platform: Microsoft Windows 7 Professional Service Pack 1 (Build 7601)
uid : f3687fb0-a928-11eb-92b0-0cc47a8346df
generation_uid : f3687fb0-a928-11eb-92b1-0cc47a8346df
company_name : OpenCube
product_name : MXFTk Advanced
product_uid : 3a4fe380-0d01-11e4-869f-3cd92b5c1dfc
product_version : 2.10.4.20180828
product_version_num: 2.10.4.0.1
toolkit_version_num: 2.10.4.0.1
modification_date: 2021-04-29T21:25:14.052000Z
material_package_umid: 0x060A2B34010101050101052013000000F36858A0A92811EB92A40CC47A8346DF
timecode : 21:25:24:09
Duration: 00:00:35.68, start: 0.000000, bitrate: 121346 kb/s
Stream #0:0: Video: dnxhd (DNXHD), yuv422p(bt709/unknown/bt709, top first), 1920x1080, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 25 tbn, 25 tbc
Metadata:
file_package_umid: 0x060A2B34010101050101052013000000F3683190A92811EB929C0CC47A8346DF
track_name : Picture
5.24 M-V: 0.045 fd= 0 aq= 0KB vq=15392KB sq= 0B f=0/0




Unfortunately, I cannot share the footage of the game (for obvious reasons, it's a copyrighted event), however I asked them to send me a non-sensitive content and they put a blue cloth on the camera and recorded a sample...
I know it kind of sucks as a sample, but here we go:

Sample - https://we.tl/t-SvCQRPGikp


This was my desperate attempt to work around the indexing issue before actually realizing that AVID Media Composer, AVID Media Director and Adobe Premiere had no issues at all with the file and they were able to decode it just fine:


#Indexing
LWLibavVideoSource("20210429WiganvHullFCX_C921h25m29s09.mxf", seek_mode=1)

#Getting rid of green fields (1920x540 25p)
SeparateFields()
selectevery(4,0,3)

#Getting rid of temporal decoding error
FreezeFrame(16, 16, 15)
FreezeFrame(22, 23, 21)
FreezeFrame(34, 34, 33)
FreezeFrame(37, 37, 36)
FreezeFrame(42, 43, 41)
FreezeFrame(51, 51, 50)
FreezeFrame(53, 53, 52)
FreezeFrame(59, 60, 58)
FreezeFrame(66, 67, 65)
FreezeFrame(87, 90, 86)
FreezeFrame(114, 114, 115)
FreezeFrame(119, 120, 118)
FreezeFrame(129, 129, 128)
FreezeFrame(144, 144, 143)
FreezeFrame(153, 153, 154)
FreezeFrame(170, 170, 169)
FreezeFrame(172, 172, 171)
FreezeFrame(175, 176, 174)
FreezeFrame(195, 195, 194)
FreezeFrame(199, 199, 198)
FreezeFrame(207, 208, 206)
FreezeFrame(215, 216, 214)
FreezeFrame(219, 219, 218)
FreezeFrame(220, 221, 219)
FreezeFrame(226, 226, 225)
FreezeFrame(245, 247, 244)
FreezeFrame(249, 249, 248)
FreezeFrame(252, 254, 251)
FreezeFrame(258, 258, 257)
FreezeFrame(263, 263, 262)
FreezeFrame(267, 271, 266)
FreezeFrame(277, 277, 276)
FreezeFrame(282, 283, 281)
FreezeFrame(288, 288, 287)
FreezeFrame(301, 302, 300)
FreezeFrame(307, 308, 306)
FreezeFrame(312, 312, 311)
FreezeFrame(314, 314, 313)
FreezeFrame(322, 322, 321)
FreezeFrame(325, 325, 324)
FreezeFrame(333, 333, 332)
FreezeFrame(357, 357, 356)
FreezeFrame(380, 380, 379)
FreezeFrame(384, 384, 383)
FreezeFrame(406, 407, 405)
FreezeFrame(413, 413, 412)
FreezeFrame(415, 415, 414)
FreezeFrame(430, 430, 429)
FreezeFrame(432, 432, 431)
FreezeFrame(434, 434, 433)
FreezeFrame(436, 436, 435)
FreezeFrame(441, 441, 440)
FreezeFrame(443, 443, 442)
FreezeFrame(453, 453, 452)
FreezeFrame(460, 460, 459)
FreezeFrame(465, 466, 464)
FreezeFrame(487, 490, 486)
FreezeFrame(496, 496, 495)
FreezeFrame(501, 501, 500)
FreezeFrame(520, 520, 519)
FreezeFrame(522, 523, 521)
FreezeFrame(532, 532, 531)
FreezeFrame(538, 538, 537)
FreezeFrame(541, 541, 540)
FreezeFrame(543, 545, 542)
FreezeFrame(550, 551, 549)
FreezeFrame(553, 553, 552)
FreezeFrame(558, 558, 557)
FreezeFrame(563, 563, 562)
FreezeFrame(570, 570, 569)
FreezeFrame(574, 574, 573)
FreezeFrame(576, 576, 575)
FreezeFrame(577, 577, 576)
FreezeFrame(579, 579, 578)
FreezeFrame(581, 581, 580)
FreezeFrame(586, 586, 585)
FreezeFrame(590, 590, 589)
FreezeFrame(598, 599, 597)
FreezeFrame(601, 602, 600)
FreezeFrame(608, 608, 607)
FreezeFrame(613, 613, 612)
FreezeFrame(624, 624, 623)
FreezeFrame(633, 633, 632)
FreezeFrame(648, 648, 647)
FreezeFrame(650, 650, 649)
FreezeFrame(654, 656, 653)
FreezeFrame(661, 661, 660)
FreezeFrame(667, 667, 666)
FreezeFrame(670, 670, 669)
FreezeFrame(672, 672, 671)
FreezeFrame(674, 675, 673)
FreezeFrame(681, 681, 680)
FreezeFrame(686, 686, 685)
FreezeFrame(700, 700, 699)
FreezeFrame(713, 713, 712)
FreezeFrame(718, 718, 717)
FreezeFrame(736, 737, 735)
FreezeFrame(742, 742, 741)
FreezeFrame(759, 762, 758)
FreezeFrame(771, 771, 770)
FreezeFrame(773, 773, 772)
FreezeFrame(784, 784, 783)
FreezeFrame(786, 786, 785)
FreezeFrame(787, 787, 786)
FreezeFrame(792, 792, 791)
FreezeFrame(809, 813, 808)
FreezeFrame(820, 820, 821)
FreezeFrame(832, 832, 831)
FreezeFrame(834, 836, 833)
FreezeFrame(838, 838, 837)
FreezeFrame(860, 862, 859)
FreezeFrame(866, 866, 865)
FreezeFrame(868, 868, 867)
FreezeFrame(873, 873, 872)
FreezeFrame(888, 889, 887)
FreezeFrame(891, 891, 890)

#Getting rid of the initial green frames
trim(5, 0)

#Going from 8bit planar to 10bit planar for additional precision
ConvertBits(10)

#Denoising to make life easier for motion vector analysis
dfttest(sigma=64, tbsize=1, lsb_in=false, lsb=false, Y=true, U=true, V=true, opt=0, dither=0)


#Interpolating the previously removed frames which were incorrectly decoded
super = MSuper()
backward_vectors = MAnalyse(super, isb = true, delta=2, blksize=16, blksizev=16, searchparam=3, plevel=0, search=3, badrange=(-24))
forward_vectors = MAnalyse(super, isb = false, delta=2, blksize=16, blksizev=16, searchparam=3, plevel=0, search=3, badrange=(-24))
inter = MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
ConditionalFilter(inter.Loop(2,0,0), last, "YDifferenceFromPrevious", "<", ".001")
super = MSuper()
backward_vectors = MAnalyse(super, isb = true, delta=2, blksize=16, blksizev=16, searchparam=3, plevel=0, search=3, badrange=(-24))
forward_vectors = MAnalyse(super, isb = false, delta=2, blksize=16, blksizev=16, searchparam=3, plevel=0, search=3, badrange=(-24))
inter = MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
ConditionalFilter(inter.Loop(2,0,0), last, "YDifferenceFromPrevious", "<", ".001")
super = MSuper()
backward_vectors = MAnalyse(super, isb = true, delta=2, blksize=16, blksizev=16, searchparam=3, plevel=0, search=3, badrange=(-24))
forward_vectors = MAnalyse(super, isb = false, delta=2, blksize=16, blksizev=16, searchparam=3, plevel=0, search=3, badrange=(-24))
inter = MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
ConditionalFilter(inter.Loop(2,0,0), last, "YDifferenceFromPrevious", "<", ".001")
super = MSuper()
backward_vectors = MAnalyse(super, isb = true, delta=2, blksize=16, blksizev=16, searchparam=3, plevel=0, search=3, badrange=(-24))
forward_vectors = MAnalyse(super, isb = false, delta=2, blksize=16, blksizev=16, searchparam=3, plevel=0, search=3, badrange=(-24))
inter = MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
ConditionalFilter(inter.Loop(2,0,0), last, "YDifferenceFromPrevious", "<", ".001")

#Interpolating 25p to 50p
super=MSuper(pel=1, hpad=0, vpad=0)
backward_1=MAnalyse(super, chroma=false, isb=true, blksize=16, blksizev=16, searchparam=3, plevel=0, search=3, badrange=(-24))
forward_1 =MAnalyse(super, chroma=false, isb=false, blksize=16, blksizev=16, searchparam=3, plevel=0, search=3, badrange=(-24))
backward_2 = MRecalculate(super, chroma=false, backward_1, blksize=8, blksizev=8, searchparam=0, search=3)
forward_2 = MRecalculate(super, chroma=false, forward_1, blksize=8, blksizev=8, searchparam=0, search=3)
MBlockFps(super, backward_2, forward_2, num=50000, den=1000, mode=0)

#Spatially removing (some) artifacts introduced by the interpolation
neo_dfttest(sigma=64, tbsize=1, Y=3, U=3, V=3, dither=0, opt=0)

#Upscale to create the full resolution from a single field using Neural Network Edge Directed Interpolation 3
nnedi3_rpow2(cshift="Spline64ResizeMT", rfactor=2, fwidth=1920, fheight=1080, nsize=4, nns=4, qual=1, etype=0, pscrn=2, threads=0, csresize=true, mpeg2=true)

#Recreate two fields to get a proper 25i
assumeTFF()
separatefields()
selectevery(4,0,3)
weave()




Anyway, you can easily see that there are green fields that are not supposed to be there for whatever reason and I have no idea why, but I would very much like to have this fixed.

I've opened a ticket in the bug tracker of FFMpeg...

[FranceBB]

Ok, so -thread_type slice actually solves the issue in FFMpeg and FFPlay:

ffplay -i "/home/FranceBB/Downloads/20210429WiganvHullFCX_C921h25m29s09.mxf" -thread_type slice

So... the question is: how can we get it fixed in ffms2 and LWLibav?

[patul]

Have you tried to force the LWLibavVideoSource to use single thread? The default for thread parameter is 0, of which,

int threads = 0

The number of threads to decode a stream by libavcodec.
The value 0 means the number of threads is determined automatically and then the maximum value will be up to 16.

[FranceBB]

LWLibavVideoSource("Z:\20210429WiganvHullFCX_C921h25m29s09.mxf", threads=1)

Still shows green fields:

https://i.imgur.com/LffgTtf.png

Same goes for:


FFVideoSource("Z:\20210429WiganvHullFCX_C921h25m29s09.mxf", threads=1)


https://i.imgur.com/XajWgeU.png


While

ffplay -i "/home/FranceBB/Downloads/20210429WiganvHullFCX_C921h25m29s09.mxf" -thread_type slice

solves the issue:

https://i.imgur.com/mhAMi0w.png


I guess that HollyWu and Myrsloyk have to do something about this on developer side rather than on user side, like detecting when it's DNX interlaced and force the -thread_type slice.

There are also some DNX120 that play ok from some cameras and some other DNX120 that have green fields from some totally different cameras.
Going deeper in the analysis, those from the camera brands that are decoded correctly have:

[dnxhd @ 000001f4fd9d7c80] interlaced 3, cur field 1

those from the camera brands that are decoded incorrectly (unless -thread_type slice is specified) have:

[dnxhd @ 0000013fc0277c80] interlaced 2, cur field 0

the values are bits in the frame header, what i don't get is why we have the entries 2 times



[dnxhd @ 0000025bc07c7c80] interlaced 2, cur field 0
[dnxhd @ 0000025bc07c7c80] Profile cid 1242.
[dnxhd @ 0000025bc07c7c80] 1920x1080, 4:2:2 8 bits, MBAFF=0 ACT=0
[dnxhd @ 0000025bc07c7c80] interlaced 2, cur field 0
[dnxhd @ 0000025bc07c7c80] 1920x1080, 4:2:2 8 bits, MBAFF=0 ACT=0


in the cameras that produce the version with the decoding issue, it's 2 times "interlaced 2, cur field 0" while in the camera brands that produce a file correctly decoded without any flag, only the first entry is the same while the second entry is

[dnxhd @ 00000222415e7c80] interlaced 3, cur field 1


I opened a ticket in the ffms2 (https://github.com/FFMS/ffms2/issues/384) and LSMASH (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/commit/e5bb1ecb71f0edb5b7632b1013faaed77277e2b4) repository as well.

[FranceBB]

Ok, so I've been testing several DNX120 and those that are decoded correctly in FFMpeg without the -thread_type slice have:

[dnxhd @ 0000025bc07c7c80] interlaced 2, cur field 0
[dnxhd @ 00000222415e7c80] interlaced 3, cur field 1

while the ones that are decoded incorrectly and need the -thread_type slice in order to decode them correctly have:

[dnxhd @ 0000025bc07c7c80] interlaced 2, cur field 0
[dnxhd @ 0000025bc07c7c80] interlaced 2, cur field 0

Besides, if I use ffprobe to see what's going on in the files that are not decoded correctly I get:

ffprobe "test.dnxhd" -show_frames |findstr top_field_first
top_field_first=0
top_field_first=1
top_field_first=0
top_field_first=1
top_field_first=0
top_field_first=1
top_field_first=0
top_field_first=1
top_field_first=0
top_field_first=1
top_field_first=0
top_field_first=1

Now, according to the DNX standard, we have one frame header per field (unlike in other codecs like MPEG-2 where we only have one frame header for each pair of fields).
The frame header start sequence is 00 00 02 80 01 - the next byte tells us if we deal with the first or second field of an interlaced frame
00 00 02 80 01 02 indicates first field
00 00 02 80 01 03 indicates second field
If I take one of the DNX120 which is decoded correctly in ffmpeg by default, I have 02 and 03 that change with every frame header, like so:

frame 0, first field 00 00 02 80 01 02
frame 0, second field 00 00 02 80 01 03
00 00 02 80 01 02
00 00 02 80 01 03
00 00 02 80 01 02
00 00 02 80 01 03

but this is not the case for the DNX120 files we have received from the pitch in which we have:

00 00 02 80 01 02
00 00 02 80 01 02
00 00 02 80 01 03
00 00 02 80 01 03
00 00 02 80 01 02
00 00 02 80 01 02
00 00 02 80 01 03
00 00 02 80 01 03


See the issue here?

[FranceBB]

Ok, more progress: using gsar (https://trac.ffmpeg.org/attachment/ticket/9255/gsar.exe) to modify the header works and makes the files possible to decode/index like so:

type "INPUTFILE" | C:\temp\gsar.exe -s:x83:x09:x40:x00:x00:x00:x02:x80:x01:x03 -r:x83:x09:x40:x00:x00:x00:x02:x80:x01:x02 -F | C:\temp\gsar.exe -s:x60:x0D:xC0:xDE:x00:x00:x02:x80:x01:x02 -r:x60:x0D:xC0:xDE:x00:x00:x02:x80:x01:x03 -F > "OUTPUTFILE"

Still, I believe this should be addressed anyway without using gsar (https://trac.ffmpeg.org/attachment/ticket/9255/gsar.exe).

[FranceBB]

cat.exe "INPUTFILE" | gsar.exe -s:x83:x09:x40:x00:x00:x00:x02:x80:x01:x03 -r:x83:x09:x40:x00:x00:x00:x02:x80:x01:x02 -F |
gsar.exe -s:x60:x0D:xC0:xDE:x00:x00:x02:x80:x01:x02 -r:x60:x0D:xC0:xDE:x00:x00:x02:x80:x01:x03 -F > "OUTPUTFILE"

this also fixes the decoding issue and it can be executed on Windows on the original file before decoding it.

gsar.exe: Download (https://trac.ffmpeg.org/attachment/ticket/9255/gsar.exe)

cat.exe: Download (https://trac.ffmpeg.org/attachment/ticket/9255/cat.exe)

[FranceBB]

Ok, I've noticed that asd-g attempted to fix the indexing issue using a workaround in LWLibavVideoSource here: https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/commit/e5bb1ecb71f0edb5b7632b1013faaed77277e2b4

I look forward to test the new build to see if the problem has been solved.
If it does solve the problem, ffms2 devs should follow the same approach.

[FranceBB]

Unfortunately the asd-g workaround didn't work... :(
Further investigation is required.

[FranceBB]

Update: New fix by ASD-G and this time he got it right! It works! ;)

https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/commit/e5bb1ecb71f0edb5b7632b1013faaed77277e2b4

He also made a custom build with the fix: https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/files/6956417/ls1.zip


So, hats off to LSMASH, now all we need is ffms2 to follow and apply a similar (if not the same) workaround.
Myrsloik? Are you up to the task?

[richardpl]

Nice, projects full of hacks, Good riddance!

[FranceBB]

Fixed in FFMpeg as well by Paul B Mahol:

http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=507fdcd1b09deed0cfd274d6afb284a99963168f

I haven't tested it yet but looks like we're finally gonna have some light at the end of what has been a very long tunnel and, considering that MPV and ffms2 are both based on FFMpeg decoders, I very much believe that we can finally forget about this nasty bug.

Thanks everyone for your patience, time and cooperation.
I owe you one this time. ;)

EDIT: Fixed in MPV as well now https://i.imgur.com/0PGbiB0.png