IMF master file support (avisynth)

[FranceBB]

Hi everybody!

I have been using avisynth for years so far both for hobby (fansub) and work (broadcasting).
Despite the fact that there are professional programs like Premier, Final Cut etc, I love the way avisynth allows me to filter images (like the dither tool that allows me to filter at 16bit) and I obtain better result than using professional softwares in several videos.
So, I started using it at work as well, and the fact that it's able to index master files like ProRes files, HDCAM, CanopusHQ and mxf files is absolutely cool.
That said, I received an email from some companies, saying that they will start releasing masters in .imf in the very next future.
AFAIK, .imf is a container meant to contain JPEG2000 files, MPEG-2 files, MPEG-4 Part 10 (AVC H.264) files, as well as audio files.
I googled a bit and I didn't find anything about its support in avisynth.
Do you know whether avisynth supports it or not? (I think it doesn't).
I think it would be great if avisynth could support it as well as other master file formats.
Besides, I would like to download a sample .imf file to make a few tests and analyze it using an hex editor (AFAIK it should be possible to extract both audio and video by looking at them comparing the code with the one in a normal mxf file).
So... where can I get an IMF sample?
Besides, apart from avisynth, do you know any software and/or transcoding solutions that support IMF files?

Thank you in advance.

[Wilbert]

No, AviSynth doesn't support imf files. Even ffmpeg (https://www.mail-archive.com/ffmpeg-user@ffmpeg.org/msg09074.html) doesn't support it yet.

[filler56789]

I had never heard or read about the «Interoperable Master Format» until now:

http://www.imfforum.com/IMF_Forum/index.html

[TheFluff]

If a file format format has "interoperable" or "exchange" in the name somewhere, it's almost always an enterprise-ese way of saying it's a proprietary format which, while based on open standards, is just incompatible enough to not work with standard tools without some significant amount of effort. If you want to implement support for it, you either need to reverse engineer it and risk an IP lawsuit, or pay some vendor a hundred grand in licensing fees and sign an NDA for the privilege of getting to see the specification. IMF seems to be almost MXF, but not quite. MXF was apparently supported a bit too well by too many tools, so a new format was needed for another round of vendor lock-in.

[kolak]

It's standardised by SMPTE and idea behind is nice, but so far slow to catch up. Disney are the ones who heavily promote it.
It's more about metadata, having one main file and than e.g. credits as other small ones with metadata which stitches it together. Inside you have essence (like in MXF) which can be JPEG2000, uncompressed etc (loads of debate about possible formats). It's nice idea, specially metadata part, but still quite early stages.

Netflix requires UHD files to be delivered as IMF.

This video gives good overview:
https://www.youtube.com/watch?v=bmhv36hmSP4

[FranceBB]

I have contacted Archimedia regarding their Iris software, but, unfortunately, it's not what I was looking for.

"Iris does play IMF files but it does not transcode them to another format. Its something that has been asked for by many of our customers but it currently does not..
Would raw DPX frames and PCM WAV files be sufficient for your needs?"

It might be an acceptable compromise.

[kolak]

If you want conversion than MTI Film or Amberfin are more suited for this, but this is rather expensive software.
Cheapest would be Telsetream Switch (but this is more a player with just few transcoding options).

[raffriff42]

This video gives good overview:
https://www.youtube.com/watch?v=bmhv36hmSP4It does indeed. Thanks for the link; it's very enlightening. And entertaining for a geek like me.

https://www.dropbox.com/s/7ulipoibd20q4r3/SMPTE%20Standards%20IMF%201%2000.30.17%20Summary.jpg?raw=1

It sounds like an incredibly complex standard. Not something for a couple of guys to reverse-engineer over the weekend. Especially since it is not a completed standard! And BTW there are digital signatures involved, and possible encryption.

The good news (it seems to me) is that it might be OK for someone like FranceBB (as a service provider) to support a small subset of IMF. The example was given of someone providing French subtitles for a Disney movie and delivering a 'supplementary' (?) IMF with only the changed elements.

Around 0:48, mention is made of cinecert.com, who seem to be experts in this area. You may want to talk to them:Products:
* 'asdcplib' open source implementation of SMPTE and MXF Interop "Sound & Picture Track File" format
http://www.cinecert.com/asdcplib/
supports reading and writing MXF files containing sound (PCM),
picture (JPEG 2000 or MPEG-2) and timed-text (XML) essence.
* 'Anini' IMF/DCP Processing Node: Compress • Wrap • Automate
http://www.cinecert.com/product/anini/
* 'Wailua' D-Cinema Packaging Tools: Wrap • Encrypt • Validate
http://www.cinecert.com/product/wailua/

There was also mention of Avid having a (planned?) "export to IMF" feature. Mind, this video was from 2012 and things have presumably moved forward since then.

[FranceBB]

Fast forward from 2016 to 2022 and we have an IMF Decoder inside FFMpeg.

Just go to the folder you've received and look for a file called CPL_something.xml:

https://i.imgur.com/oExKP2g.png

Create a Bat and call FFMpeg with:


ffmpeg.exe -f imf -i "CPL_something.xml" -map 0:0 -map 0:1 -map 0:2 -map 0:3 -map 0:4 -c:v copy -c:a copy -f mov -y "IMF.mov"

pause


where -i "CPL_something.xml" is the name of the CPL file, in my example it would be: "CPL_IMF10326568_11130320_CARFIL_D_EP201_HDR_FF178_REC2020_2398_48K_ITA_DS_ENG_DS_ITA_51_ENG_51_TXT_ITA_DF9F8FDC_1B5F_484A_9D58_097AC6770A91.xml" so:

ffmpeg.exe -f imf -i "\\mibcisilonsc\avisynth\Scambio\IMF\CPL_IMF10326568_11130320_CARFIL_D_EP201_HDR_FF178_REC2020_2398_48K_ITA_DS_ENG_DS_ITA_51_ENG_51_TXT_ITA_DF9F8FDC_1B5F_484A_9D58_097AC6770A91.xml" -map 0:0 -map 0:1 -map 0:2 -map 0:3 -map 0:4 -c:v copy -c:a copy -f mov -y "\\mibcisilonsc\avisynth\Scambio\Raw\IMF.mov"

pause


Then, run the BAT and you'll get your final file that you can work on:

https://i.imgur.com/q0WxDyA.png

just wait and you'll get the final .mov file.
Alternatively, if you prefer to work with .mxf, you can do -f mxf and call the file IMF.mxf:

ffmpeg.exe -f imf -i "\\mibcisilonsc\avisynth\Scambio\IMF\CPL_IMF10326568_11130320_CARFIL_D_EP201_HDR_FF178_REC2020_2398_48K_ITA_DS_ENG_DS_ITA_51_ENG_51_TXT_ITA_DF9F8FDC_1B5F_484A_9D58_097AC6770A91.xml" -map 0:0 -map 0:1 -map 0:2 -map 0:3 -map 0:4 -c:v copy -c:a copy -f mxf -y "\\mibcisilonsc\avisynth\Scambio\Raw\IMF.mxf"

pause

however I found the mxf muxer inside FFMpeg to be quite unreliable at times, so I wouldn't risk it.

Inside an IMF package, you're gonna find a Motion JPEG2000 4K file in RGB (or YUV 4:4:4) 12bit planar either SDR or HDR and different separate audio tracks in PCM lossless 24bit 48'000Hz muxed as mxf.

This is an example:

https://i.imgur.com/0yl8T5z.png

Cheers,
Frank

[DTL]

Oh - RGB 12bit still alive somewhere in motion pictures industry ? I think it will died with ugly old 4:2:0 YUV.

[FranceBB]

Oh - RGB 12bit still alive somewhere in motion pictures industry ? I think it will died with ugly old 4:2:0 YUV.

Yep and it's not the first time I've seen something like this.
Back when we were getting DCP packages, inside I used to find YUV 4:4:4 XYZ 12bit and apparently IMF packages are close to this by being RGB 12bit Full Range.


For anyone else, sometimes remuxing goes wrong with FFMpeg. Technically, Motion JPEG-2000 is all intra and PCM can be trimmed anywhere, so technically by appending the parts depicted in the CPL from different videos and audios, it shouldn't be a problem and yet it reports:

https://i.imgur.com/SXdBezR.png

so, whenever you get something like that and you wanna play safe, just re-encode it to any lossless codec that matches the bit depth (12bit) and can work in RGB like HuffYUV, FFV1, etc
AFAIK Lagarith is too old to support high bit depth and last time I checked UTVideo was limited to 10bit planar, so I don't know how many choices you have.
Nonetheless, this is what I've done and it went through the infamous frame 870 without the error any longer:

ffmpeg.exe -f imf -i "\\mibcisilonsc\avisynth\Scambio\IMF\CPL_IMF10326568_11130320_CARFIL_D_EP201_HDR_FF178_REC2020_2398_48K_ITA_DS_ENG_DS_ITA_51_ENG_51_TXT_ITA_DF9F8FDC_1B5F_484A_9D58_097AC6770A91.xml" -map 0:0 -map 0:1 -map 0:2 -map 0:3 -map 0:4 -c:v ffvhuff -c:a pcm_s32le -ar 48000 -f avi -y "\\mibcisilonsc\avisynth\Scambio\Raw\IMF.avi"

pause


flying through the infamous frames with no errors:

https://i.imgur.com/C5FnOBe.png


and I've got a nice lossless RGB 16bit output in HuffYUV:


General
Complete name : I:\Scambio\Raw\IMF.avi
Format : AVI
Format/Info : Audio Video Interleave
Format profile : OpenDML
File size : 18.7 GiB
Duration : 38 s 160 ms
Overall bit rate : 4 208 Mb/s
Writing application : Lavf59.16.100

Video
ID : 0
Format : HuffYUV
Format version : Version 3
Codec ID : FFVH
Duration : 37 s 538 ms
Bit rate : 4 253 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 (24000/1001) FPS
Color space : RGB
Bit depth : 16 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 21.387
Stream size : 18.6 GiB (99%)

Audio #1
ID : 1
Format : PCM
Format settings : Little / Signed
Codec ID : 00000001-0000-0010-8000-00AA00389B71
Duration : 38 s 160 ms
Bit rate mode : Constant
Bit rate : 3 072 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Bit depth : 32 bits
Stream size : 14.0 MiB (0%)
Alignment : Aligned on interleaves
Interleave, duration : 39 ms (0.94 video frame)

Audio #2
ID : 2
Format : PCM
Format settings : Little / Signed
Codec ID : 00000001-0000-0010-8000-00AA00389B71
Duration : 38 s 160 ms
Bit rate mode : Constant
Bit rate : 3 072 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Bit depth : 32 bits
Stream size : 14.0 MiB (0%)
Alignment : Aligned on interleaves
Interleave, duration : 39 ms (0.94 video frame)

Audio #3
ID : 3
Format : PCM
Format settings : Little / Signed
Codec ID : 00000001-0000-0010-8000-00AA00389B71
Duration : 38 s 160 ms
Bit rate mode : Constant
Bit rate : 9 216 kb/s
Channel(s) : 6 channels
Channel layout : L R C LFE Ls Rs
Sampling rate : 48.0 kHz
Bit depth : 32 bits
Stream size : 41.9 MiB (0%)
Alignment : Aligned on interleaves
Interleave, duration : 39 ms (0.94 video frame)

Audio #4
ID : 4
Format : PCM
Format settings : Little / Signed
Codec ID : 00000001-0000-0010-8000-00AA00389B71
Duration : 38 s 160 ms
Bit rate mode : Constant
Bit rate : 9 216 kb/s
Channel(s) : 6 channels
Channel layout : L R C LFE Ls Rs
Sampling rate : 48.0 kHz
Bit depth : 32 bits
Stream size : 41.9 MiB (0%)
Alignment : Aligned on interleaves
Interleave, duration : 39 ms (0.94 video frame)

[DTL]

Can it simply remux with -vcodec copy and regenerate all TS tables ? Lossless huff is much larger in compare with initial jpg2k.

FFmpeg have some options for regenerating timestamps: 2 ways listed at https://superuser.com/questions/1159056/ffmpeg-recreate-timestamps-without-reencoding

1. -vsync drop
2. extract raw video bitstream and remux with new rate -r

[FranceBB]

Right, however for the time being I opened a ticket in their bug tracker.
I'm not sure whether there's something I don't understand behind this or if it's a bug that they can fix.

https://trac.ffmpeg.org/ticket/9611#ticket

[palemieux]

Fix proposed at https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=5849

Development at https://github.com/sandflow/ffmpeg-imf/pull/94

[FranceBB]

I can confirm that the patch indeed fixes the pts/dts problem.
Well done, really! :D

Mediainfo of the final and correctly remuxed file:


General
Complete name : I:\Scambio\Raw\IMF.mxf
Format : MXF
Format version : 1.3
Format profile : OP-1a
Format settings : Closed / Complete
File size : 774 GiB
Duration : 2 h 8 min
Overall bit rate : 861 Mb/s
Encoded date : 0-00-00 00:00:00.000
Writing application : FFmpeg OP1a Muxer 59.17.101.0.0
Writing library : Lavf (mingw32) 59.17.101.0.0

Video
ID : 2
Format : JPEG 2000
Format profile : IMFS4k@ML6SL4
Format settings, wrapping mode : Frame
Codec ID : 0D010301020C0100-0401020203010100
Duration : 2 h 8 min
Bit rate : 843 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:4:4
Bit depth : 12 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 4.239
Stream size : 757 GiB (98%)
Color primaries : BT.2020
Transfer characteristics : PQ
BitDepth_Container : 16

Audio #1
ID : 3
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (AES)
Codec ID : 0D01030102060300
Duration : 2 h 8 min
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 23.976 FPS (2002 SPF)
Bit depth : 24 bits
Stream size : 2.07 GiB (0%)
Locked : Yes

Audio #2
ID : 4
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (AES)
Codec ID : 0D01030102060300
Duration : 2 h 8 min
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 23.976 FPS (2002 SPF)
Bit depth : 24 bits
Stream size : 2.07 GiB (0%)
Locked : Yes

Audio #3
ID : 5
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (AES)
Codec ID : 0D01030102060300
Duration : 2 h 8 min
Bit rate mode : Constant
Bit rate : 6 912 kb/s
Channel(s) : 6 channels
Sampling rate : 48.0 kHz
Frame rate : 23.976 FPS (2002 SPF)
Bit depth : 24 bits
Stream size : 6.21 GiB (1%)
Locked : Yes

Audio #4
ID : 6
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (AES)
Codec ID : 0D01030102060300
Duration : 2 h 8 min
Bit rate mode : Constant
Bit rate : 6 912 kb/s
Channel(s) : 6 channels
Sampling rate : 48.0 kHz
Frame rate : 23.976 FPS (2002 SPF)
Bit depth : 24 bits
Stream size : 6.21 GiB (1%)
Locked : Yes

Other #1
ID : 1-Material
Type : Time code
Format : MXF TC
Frame rate : 24.000 FPS
Time code of first frame : 00:00:00:00
Time code settings : Material Package
Time code, striped : Yes

Other #2
ID : 1-Source
Type : Time code
Format : MXF TC
Frame rate : 24.000 FPS
Time code of first frame : 00:00:00:00
Time code settings : Source Package
Time code, striped : Yes

Other #3
Type : Time code
Format : SMPTE TC
Muxing mode : SDTI
Frame rate : 24.000 FPS
Time code of first frame : 00:00:00:00

[FranceBB]

Remuxed another title.
Works like a charm.


General
Complete name : \\mibcisilonsc\avisynth\Scambio\Raw\Venom - Let There be Carnage UHD HDR PQ BT2020 23,976p 12bit.mxf
Format : MXF
Format version : 1.3
Format profile : OP-1a
Format settings : Closed / Complete
File size : 553 GiB
Duration : 1 h 41 min
Overall bit rate : 779 Mb/s
Encoded date : 0-00-00 00:00:00.000
Writing application : FFmpeg OP1a Muxer 59.17.101.0.0
Writing library : Lavf (mingw32) 59.17.101.0.0

Video
ID : 2
Format : JPEG 2000
Format profile : IMFS4k@ML6SL4
Format settings, wrapping mode : Frame
Codec ID : 0D010301020C0100-0401020203010100
Duration : 1 h 41 min
Bit rate : 761 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:4:4
Bit depth : 12 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 3.825
Stream size : 540 GiB (98%)
Color primaries : BT.2020
Transfer characteristics : PQ
BitDepth_Container : 16

Audio #1
ID : 3
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (AES)
Codec ID : 0D01030102060300
Duration : 1 h 41 min
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 23.976 FPS (2002 SPF)
Bit depth : 24 bits
Stream size : 1.63 GiB (0%)
Locked : Yes

Audio #2
ID : 4
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (AES)
Codec ID : 0D01030102060300
Duration : 1 h 41 min
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 23.976 FPS (2002 SPF)
Bit depth : 24 bits
Stream size : 1.63 GiB (0%)
Locked : Yes

Audio #3
ID : 5
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (AES)
Codec ID : 0D01030102060300
Duration : 1 h 41 min
Bit rate mode : Constant
Bit rate : 6 912 kb/s
Channel(s) : 6 channels
Sampling rate : 48.0 kHz
Frame rate : 23.976 FPS (2002 SPF)
Bit depth : 24 bits
Stream size : 4.90 GiB (1%)
Locked : Yes

Audio #4
ID : 6
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (AES)
Codec ID : 0D01030102060300
Duration : 1 h 41 min
Bit rate mode : Constant
Bit rate : 6 912 kb/s
Channel(s) : 6 channels
Sampling rate : 48.0 kHz
Frame rate : 23.976 FPS (2002 SPF)
Bit depth : 24 bits
Stream size : 4.90 GiB (1%)
Locked : Yes

Other #1
ID : 1-Material
Type : Time code
Format : MXF TC
Frame rate : 24.000 FPS
Time code of first frame : 00:00:00:00
Time code settings : Material Package
Time code, striped : Yes

Other #2
ID : 1-Source
Type : Time code
Format : MXF TC
Frame rate : 24.000 FPS
Time code of first frame : 00:00:00:00
Time code settings : Source Package
Time code, striped : Yes

Other #3
Type : Time code
Format : SMPTE TC
Muxing mode : SDTI
Frame rate : 24.000 FPS
Time code of first frame : 00:00:00:00

[FranceBB]

Fast forward to a few more days and I've used the FFMpeg imf decoding made by Pierre-Anthony Lemieux to create an Avisynth workflow which I'm testing and I'll hopefully soon gonna use in production.

So, in a nutshell, I get the IMF downloaded from Aspera in a watchfolder and then I grab the CPL, create the muxed .mxf file with the new Pierre's IMF decoder, index it in Avisynth with ffms2 and then perform the eventual PQ to HLG conversion, frame rate conversion and encoding in XAVC Intra Class 300 with x264. The final video + audio mux is done with the BBC muxer: BBC BMX Transwrap. :)

https://i.imgur.com/xW3GKCL.png

In particular, this shows the workflow step by step:

https://i.imgur.com/20trzbg.png
https://i.imgur.com/CXr9kdd.png
https://i.imgur.com/GZEdcuv.png
https://i.imgur.com/TvN8Dht.png


and at the end of the workflow, my file is indeed ready:

https://i.imgur.com/ZKVsXqZ.png
https://i.imgur.com/WmkR5qm.png


It's perfectly fine, in HLG and it plays back just fine on Versio hardware playout port.
I love opensource.
Avisynth forever, Doom9 forever.

[Frank62]

Looks nice. Can you explain why fps>70 is encoded directly?

[FranceBB]

Looks nice. Can you explain why fps>70 is encoded directly?

Because it's decimated.
You can't blend to 50p if you have more than 70fps, so I just drop FPS to 50p at that point. ;)

[Frank62]

What about SelectEven.Blend? :D

[FranceBB]

What about SelectEven.Blend? :D

Why? It would make things worse xD
When you have high enough fps, you can just drop and it will be as smooth as if it was truly 50p ;)

[Frank62]

I don't get the point, but you will know what you do.

[FranceBB]

I don't get the point, but you will know what you do.

Ok, so, assume you gotta output a 50p.
If you have like 120fps by your approach I would use selecteven to get 60fps and then blend down to 50p. This will leave me with blended frames in the 50p, so I will have still frames that look horrible, particularly at UHD resolution, 'cause they're two images overlayed one on top of the other. If I just drop to 50p, however, each frame will be perfectly clear and the motion won't judder or anything 'cause it's actually as smooth as if it was shot at 50p. ;)

[kolak]

60p with dropped frames to 50p doesn't look smooth at all as you create motion gaps.
It's the same (just less visible) as converting 25p to 24p by dropping 1 frame every second.

[FranceBB]

60p with dropped frames to 50p doesn't look smooth at all as you create motion gaps.
It's the same (just less visible) as converting 25p to 24p by dropping 1 frame every second.

Yep, in fact the conditional in the workflow says that it blends back to 50p 'till the source is 70fps and if it's higher then it just drops to 50p. Frank62 asked me why I drop if it's higher than 70fps and the reason is that if the source is like 90fps and I use ConvertFPS(50) Avisynth throws an error, so I just decimate and that's it. ;)

[FranceBB]

P.s I'm totally open to new ideas though.
What would you do if you received something larger than 70fps and your target is 50p?

[StainlessS]

so I just decimate and that's it.

ChangeFPS

Changes the frame rate by deleting or duplicating frames.
http://avisynth.nl/index.php/FPS#ChangeFPS
Only real advantage, is builtin.

[FranceBB]

http://avisynth.nl/index.php/FPS#ChangeFPS
Only real advantage, is builtin.

That's exactly what I use XD
I use AssumeFPS for 23,976 -> 25 (then Change FPS to 50 to duplicate), ConvertFPS for things like 29,970 -> 50 and for things like 60 -> 50 and 70 -> 50 and I use ChangeFPS for anything larger than 70 -> 50.

[Frank62]

P.s I'm totally open to new ideas though.
What would you do if you received something larger than 70fps and your target is 50p?
I would try something motion flow based, this would fail in several scenes, so I would blend, as you also prefer. This could again be achieved by

-dropping frames
-blending the result.

Of course this way you generate motion gaps by dropping, but you also cause gaps by simply dropping (decimating) some to get directly 50fps. The difference is that my proposal generates REGULAR gaps (which are much less obvious than one or two each second!), followed by blending which makes it even more smooth, and makes it look more as if it had been shot with a lower framerate with longer exposure time.

You seem to think that if the framerate is high enough, no matter what you drop the result will be perfectly smooth(?), this is IMHO not the case.

[FranceBB]

-dropping frames
-blending the result.




So, ok, suppose I get something like 120p, what you're saying is that I should drop to something that can be blended (like 70p) and then blend to 50p? Like:

#Indexing 120p
FFMpegSource2("test.mxf", atrack=-1)

#Dropping to 70p
ChangeFPS(70)

#Blending to 50p
ConvertFPS(50)

or perhaps I misinterpreted what you're saying.
Sometimes the code is worth thousands words eheheheh

[kolak]

You seem to think that if the framerate is high enough, no matter what you drop the result will be perfectly smooth(?), this is IMHO not the case.

Exactly. Any fps has unique timings and in order for video to be smooth you need frame in those "precise points in time". Only way to do it is motion estimation or "perfect" frame dropping. For this you would need 300fps based source to be ale to get "good" 50 or 60p (it would still have wrong motion blur). What you can assume is any imperfection in 60p is less visible/obvious for the eye than one in eg 25p due to much smaller motion gaps in 60p.
Does any broadcaster accept blended conversion ? I've never sent blended conversion to a broadcast as it would be rejected straight away.

[Frank62]

@FranceBB
Your script is almost what I thought, but not exactly: With your example I would go first to 60fps, what you could achieve by simply SelectEven() (or Odd...). The clue is to decimate REGULARLY, in this case drop each second frame. So the motion won't become jerky in the first place.

@kolak
In the ninetees blending conversions were standard in PAL countries, and done hundreds or thousands of times. One good example: Star Trek Next Generation.
Or simply every series shot in original 29.97, where until today no better way exists (yes, one can try vectorized motion flow, but this fails too often) f. e. The Wonder Years. Also most cartoon series were converted with blends. We released some, where we didn't have better masters than those converted ones and used SRestore, for it was the best method to get at least something progressive.

[FranceBB]

Ok cool.
I'll make a few tests and I'll modify the logic.
Thanks everyone. ;)

[kolak]

@kolak
In the ninetees blending conversions were standard in PAL countries, and done hundreds or thousands of times. One good example: Star Trek Next Generation.
Or simply every series shot in original 29.97, where until today no better way exists (yes, one can try vectorized motion flow, but this fails too often) f. e. The Wonder Years. Also most cartoon series were converted with blends. We released some, where we didn't have better masters than those converted ones and used SRestore, for it was the best method to get at least something progressive.

Yes, sometimes you do get those masters and think why they are so bad :)
Today's motion adaptive conversion is ok, specially when you have 50/60i/p to start with.

[Frank62]

My opinion: Motion adaptive approaches incl. A. I. are still far away from being perfect. Of course less/no ghosting, but a lot of other artificial looking things. Not my taste. Also I only saw very few examples (up to now only documentaries) that were aired on tv. Also nearly everything available incl. A. I. fails in some high movement scenarios, which makes it hard to use professionally in higher quality segments.
Seems old-fashioned, but I still prefer blending, especially with
original (not pulldowned) 29.97 --> 25

Also: When there were low-movement things like interviews when we used motion adaption we preferred to use the good old MBlockFps (of mvtools), instead of MFlow or the more sophisticated avisynth framerateconverter-script, because IF there were extreme motion fails the result is still more watchable if done with blocks.
We also tested software like S&W Alchemist - the results were in nearly every case slightly worse than simple MVTools functions.

[kolak]

If you have 50i/60i or better (50p+) to start with there are really few artefacts in latest Alchemist (software version not hardware) or Tachyon. Only challange is 24/25p as start but then you rather do speed change.
AI can be crazy good, but can fail quickly as well. It's rather sensitive to source nature, needs more polishing.
Blending?- looks terrible for typical case and no one accepts it. It's good as fallback where motion adaptive method produces too many artefacts (Tachyon uses it with some masking like framerateconverter-script script).

Best is to avoid fps conversion :) Then use speed change if possible or advanced pulldown like 25p to 29.97i. 23.976p to 29.97i for me is jerky, but USA is fine with it.

Alchemist software worse than plain mvtools? Hmmm- try 10 random/different sources. Judging on 1 is not representative. It's worse in some cases, but better in others+ thanks to GPU use it's 2x faster than realtime for HD 50i to 60i on single 1080Ti. It also has important feature which is clean cut. It can avoid making scene change on 2nd field, which sometimes can be a problem. It's fairly well tweaked for any source. With mvtools you can get better results, but it will be so time consuming. When you have 1000 files to run through there is really no time for manual tweaking.

[Frank62]

Best is to avoid fps conversion :)
I agree. :)

Then use speed change if possible
Of course, I also agree.

or advanced pulldown like 25p to 29.97i. 23.976p to 29.97i for me is jerky, but USA is fine with it.
I feel exactly the same.

But I look to it from the PAL side, and you didn't mention the other case: Going from 29.97 to 25.
Dropping frames is extremely jerky, a speed change would make it slow motion, and all kinds of pulldowns are too jerky for PAL-used eyes.

[kolak]

But I look to it from the PAL side, and you didn't mention the other case: Going from 29.97 to 25.
Dropping frames is extremely jerky, a speed change would make it slow motion, and all kinds of pulldowns are too jerky for PAL-used eyes.

This is point 1- avoid :)

[FranceBB]

Going from 29.97 to 25.
Dropping frames is extremely jerky, a speed change would make it slow motion, and all kinds of pulldowns are too jerky for PAL-used eyes.

Yep, the only thing is that generally over here 29.970 is blended to 50p and then divided in fields to get 25i for everything but sports. If we do receive a sports content for which we can do whatever we like, then linear interpolation with MVTools it is (we actually make both version automatically and then the editor picks the one he wants). For movies or TV Series, though, this cannot be done for legal reasons. The overwhelming majority of those contents are native in 23,976p (and sometimes we receive the telecined 29,970 version with 3:2 pulldown in which case we use TIVTC to go back to the original framerate), so in both cases we start from a 23,976p which is speeded up by 4% to 25p and flagged as interlaced. With the tiny minority of contents coming over as 29,970p, blending to 50p and SeparateFields() to 25i it is as that's accepted by content providers (i.e Sony etc) while linear interpolation / optical flow etc is not.

[Frank62]

This is point 1- avoid :)
Or do something else like drinking or smoking a good cigar, what is exactly what I am going to do now. ;)

@FranceBB
Exactly how we did our DVDs in these cases. :)

[FranceBB]

Or do something else like drinking or smoking a good cigar, what is exactly what I am going to do now. ;)

Cheers, mate.

https://c.tenor.com/R4n0cDzr6RgAAAAM/shots-drinks.gif

[kolak]

...With the tiny minority of contents coming over as 29,970p, blending to 50p and SeparateFields() to 25i it is as that's accepted by content providers (i.e Sony etc) while linear interpolation / optical flow etc is not.

And they accept blending and are not happy with 29.97p to 50i conversion with eg. Tachyon. Interesting :)
Tachyon is quite good with low fps source and 29.97p is not anymore that low fps. They use masking and blended fallback for scenes which produce too many artefacts. It works quite well.

Do you have a small sample of such a blended file. I would like to see it on TV :)

[excellentswordfight]

Hey @FranceBB, I just downloaded the IMF package of the stem2 movie from https://theasc.com/asc/stem2 and im not sure what the best way to feed the encoder with that source, I'm not even sure how to validate that I do the conversion correctly. I thought that I could just use the ProRes version as a reference, but it looks to me that they messed up that conversion as well.

I'm starting now with the SDR version, mediainfo states YUV 4:4:4, but ffmpeg reports "rgb48le(10 bpc, bt709, progressive)" (and that does seem to be correct as it matched the info in CPL xml). And if just try FFVideoSource with avisnyth (LSMASHVideoSource cant open the file at all), info says 16 bits per component.

So first of all, what would be the best way just to get an correct conversion to yuv 4:2:0 8bit in avsinyth, just so that I can get a reference, and secondly, do you think its even worth trying to mimic the conversion in ffmpeg? Cause it would be nice just do everything there, I think that I would be able to do it in ffmpeg, but I need something that im confident is correct to compare the test results to.

[FranceBB]

Ok, it's actually pretty easy and I even automated it in FFAStrans at work.

So, first thing first, the first thing would be to get the file they want you to have using the CPL.
Given that MJPEG2000 is all intra (so each frame is encoded individually) it's possible to "remux" by reading the CPL to get the final file losslessly (and the same goes for the audio, since it's PCM, so it can be trimmed anywhere).

To do that with FFMpeg, it's gonna be as easy as:

ffmpeg.exe -f imf -i "CPL_something.xml" -map 0:0 -map 0:1 -map 0:2 -map 0:3 -map 0:4 -c:v copy -c:a copy -f mxf -y "IMF.mxf"

(I use map 'cause my stream has 1 video and 4 audio, but you can use any map you want in that regard, like if you only want the video and the first audio track you can skip the map).

From there, it's gonna be business as usual just like you would do with literally any other file as you're gonna have one normal mezzanine file derived from the CPL in .mxf.


I'm starting now with the SDR version, mediainfo states YUV 4:4:4, but ffmpeg reports "rgb48le(10 bpc, bt709, progressive)" (and that does seem to be correct as it matched the info in CPL xml).

Ignore the 4:4:4 YUV info inside Mediainfo, IMF files in MJPEG2000 are almost always delivered as RGB48 in Full PC Range. Actually, someone should really tell Jerome and open an issue on the Mediainfo bugtracker.
I'll do it eventually tomorrow morning (if I forget, guys, remind me, 'cause the older and more stressed out / burned out I get, the more things I forget - and I'm still relatively young).



And if just try FFVideoSource with avisnyth (LSMASHVideoSource cant open the file at all), info says 16 bits per component.


Yeah, so FFVideoSource converts anything that can't directly index inside Avisynth to 4:4:4 16bit internally. It does it for things like XYZ when it's not YUV and it does the same for RGB48.

Now, about LSMASH, it obviously cannot index it 'cause it's not an ISO standard, however if you use LWLibavVideoSource() and LWLibavAudioSource() you will be able to skip 1 step and index the CPL directly without going through an intermediate file in FFMpeg at all!
So everything stays inside Avisynth and that's my suggested "go to" method.

In other words, just do:

video=LWLibavVideoSource("CPL_something.xml")
audio=LWLibavAudioSource("CPL_something.xml")
AudioDub(video, audio)

and you'll end up with RGB48 indexed directly inside Avisynth

https://i.imgur.com/Ic3WacT.png


Now, in my case, the example is RGB48 HDR PQ BT2020 Full PC Range, however if you're starting with the RGB48 SDR BT709 Full PC Range version, going to YUV 4:2:0 8bit it's gonna be as easy as:


#Indexing
video=LWLibavVideoSource("CPL_something.xml")
audio=LWLibavAudioSource("CPL_something.xml")
AudioDub(video, audio)

#Screw frame properties
propclearall()

#Going to yv12
ConverttoYUV420(matrix="Rec.709")
ConvertBits(bits=8, dither=1)



if you have the BT2020 HDR PQ then it's slightly more complicated depending on what you wanna do:

1) Encode as BT2020 PQ 4:2:0 Type 2 10bit H.265
2) Bring it to BT2020 HLG 4:2:0 10bit and encode it
3) Bring it to BT709 SDR 4:2:0 8bit and encode it with H.264

### Encoding in PQ ###

The first option is probably the easiest one as it's gonna be as easy as:


#Indexing
video=LWLibavVideoSource("CPL_something.xml")
audio=LWLibavAudioSource("CPL_something.xml")
AudioDub(video, audio)

#Screw frame properties
propclearall()

#From RGB planar Full Range to YUV420 10bit planar Narrow Range with dithering
z_ConvertFormat(pixel_type="YUV420P10", colorspace_op="rgb:st2084:2020:full=>2020:st2084:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")


now, the output of that will still be 4:2:0 with the MPEG-2 chroma placement, so you need to convert it to Type 2, so



ffmpeg.exe -i "AVS Script.avs" -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p10le -strict -1 -an -f yuv4mpegpipe - | x265.exe --y4m - --preset medium --level 5.0 --tune fastdecode --no-high-tier --ref 4 --profile main10 --bitrate 25000 --deblock -1:-1 --hdr-opt --hrd --aud --min-luma 64 --max-luma 940 --chromaloc 2 --range limited --videoformat component --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,0.0050)" --max-cll 1000,400 --overscan show --no-open-gop --min-keyint 1 --keyint 24 --repeat-headers --rd 3 --vbv-maxrate 25000 --vbv-bufsize 25000 --wpp -o "raw_video.hevc"

mkvmerge.exe --output "raw_video.mkv" --language 0:und ^"^(^" raw_video.hevc ^"^)^"

ffmpeg.exe -i "AVS Script.avs" -map 0:1 -af loudnorm=I=-24:LRA=14:tp=-2 -c:0:1 ac3 -b:0:1 384k -ar 48000 "audio.ac3"

ffmpeg.exe -i "AVS Script.avs" -map 0:1 -af loudnorm=I=-24:LRA=14:tp=-2 -c:0:1 libtwolame -b:0:1 192k -ar 48000 "audio.mp2"


ffmpeg.exe -i "raw_video.mkv" -i "audio.ac3" -i "audio.mp2" -map 0:0 -map 1:0 -map 2:0 -c:0:0 copy -c:1:0 copy -c:2:0 copy -f mpegts -metadata provider_name="ABC" -metadata service_provider="ABC" -metadata service_name="ABC" -mpegts_original_network_id 0x1122 -mpegts_transport_stream_id 0x3344 -mpegts_service_id 0x5566 -mpegts_pmt_start_pid 0x1500 -mpegts_start_pid 0x150 -mpegts_service_type "digital_tv" -mpegts_flags "resend_headers" "\\VBOXSVR\Share_Windows_Linux\final.ts"

pause


where of course you can change the settings, namely --max-cll must be the same one as your content and --keyint must be equal to the framerate of your file. 23,976 = keyint 24, 25 = keyint 25, 29,970 = keyint 30, 50 = keyint 50, 59,940 and 60 = keyint 60.

This of course depends on what your target audience is and you can change anything.
In my case, my target was to create a .ts stream with AC3 audio and legacy MP2 compatibility stream.

Oh and of course, "ABC" should be replaced with the name of your company when you make the final file ;)


### Encoding in HLG (from PQ) ###


The second option would be to convert to HLG. That can be tricky 'cause some studios DO NOT allow us to do anything we want and might wanna negotiate a specific conversion.
For instance, we have a signed agreement with some companies that want us to use a particular LUT they made according to the reference white in PQ etc.
Anyway, assuming that you're not hog-tied to a particular contract, you can either use HLG Tools by William (https://forum.doom9.org/showthread.php?t=182499) to generate the PQ to HLG LUT for you (it can even use PQ Stats to detect the MaxCLL if they didn't provide it to you) OR you can use my LUT based on the official BBC White Paper (https://forum.doom9.org/showthread.php?t=176091), but beware that it will assume a reference white of 1000 nits.
There are also other ways like using avsresize.
I mean, you have plenty of ways to do it.
I'm gonna show one:


#Indexing
video=LWLibavVideoSource("CPL_something.xml")
audio=LWLibavAudioSource("CPL_something.xml")
AudioDub(video, audio)

#Screw frame properties
propclearall()

#From RGB48 Full Range to RGB Planar 16bit Full Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:full=>rgb:st2084:2020:full", resample_filter_uv="spline64", dither_type="error_diffusion")

#From PQ to HLG with 16bit precision
Cube("PQ_to_HLG.cube", fullrange=true)

#From RGB 16bit planar Full Range to YUV420 10bit planar Narrow Range with dithering
z_ConvertFormat(pixel_type="YUV420P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")



From here it's really easy 'cause we have a nice cool HLG BT2020 4:2:0 10bit planar Limited TV Range stream and we don't need to convert to Type 2 due to backwards compatibility reasons (one more advantage of the HLG transmission standard, which I love), so...

ffmpeg.exe -i "AVS Script.avs" -pix_fmt yuv420p10le -strict -1 -an -f yuv4mpegpipe - | x265.exe --y4m - --preset medium --level 5.0 --tune fastdecode --no-high-tier --ref 4 --profile main10 --bitrate 25000 --deblock -1:-1 --hdr-opt --hrd --min-luma 64 --max-luma 940 --range limited --videoformat component --colorprim bt2020 --transfer arib-std-b67 --colormatrix bt2020nc --atc-sei 18 --overscan show --no-open-gop --min-keyint 1 --keyint 24 --repeat-headers --rd 3 --vbv-maxrate 25000 --vbv-bufsize 25000 --wpp -o "raw_video.hevc"

mkvmerge.exe --output "raw_video.mkv" --language 0:und ^"^(^" raw_video.hevc ^"^)^"

ffmpeg.exe -i "AVS Script.avs" -map 0:1 -af loudnorm=I=-24:LRA=14:tp=-2 -c:0:1 ac3 -b:0:1 384k -ar 48000 "audio.ac3"

ffmpeg.exe -i "AVS Script.avs" -map 0:1 -af loudnorm=I=-24:LRA=14:tp=-2 -c:0:1 libtwolame -b:0:1 192k -ar 48000 "audio.mp2"

ffmpeg.exe -i "raw_video.mkv" -i "audio.ac3" -i "audio.mp2" -map 0:0 -map 1:0 -map 2:0 -c:0:0 copy -c:1:0 copy -c:2:0 copy -f mpegts -metadata provider_name="ABC" -metadata service_provider="ABC" -metadata service_name="ABC" -mpegts_original_network_id 0x1122 -mpegts_transport_stream_id 0x3344 -mpegts_service_id 0x5566 -mpegts_pmt_start_pid 0x1500 -mpegts_start_pid 0x150 -mpegts_service_type "digital_tv" -mpegts_flags "resend_headers" "\\VBOXSVR\Share_Windows_Linux\final.ts"

pause

same rule as before, keyint = framerate and "ABC" in the muxing should be changed to whatever the company you're working for is called ;)


### Encoding in BT709 SDR (from PQ) ###

It's really unlikely that a company will ever allow you to do this without having them send samples back and forth as they wanna approve each and every thing you do 'cause there are many things that could go wrong.
(Well, I mean, not go wrong, but maybe the director doesn't agree with it as it doesn't respect his original intent/creation etc).
Anyway, assuming that they allow you to do this, you can use plenty of methods.
There's HDR Tools by JPSDR (Jean Philippe Scotto di Rinaldi) which I used in the past and I remember working with him on testing his latest implementations with 12bit DNX which was a lot of fun.
Anyway, you can also use avsresize as always or you can use one my LUTs.
Again, I'll show just one way, but feel free to use anything you like. :)



#Indexing
video=LWLibavVideoSource("CPL_something.xml")
audio=LWLibavAudioSource("CPL_something.xml")
AudioDub(video, audio)

#Screw frame properties
propclearall()

#From RGB48 Full Range to RGB Planar 16bit Full Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:full=>rgb:st2084:2020:full", resample_filter_uv="spline64", dither_type="error_diffusion")

#From PQ to BT709 SDR with 16bit precision
Cube("PQ_to_BT709_v1.cube", fullrange=true)

#From RGB 16bit planar Full Range to YUV420 8bit planar
z_ConvertFormat(pixel_type="YUV420", colorspace_op="rgb:709:709:full=>709:709:709:limited", resample_filter_uv="spline64", dither_type="error_diffusion", use_props=0)

#Downscale to FULL HD
SinPowResizeMT(1920, 1080)



and at this point it's all nice and dandy 'cause we have a FULL HD yv12 BT709 SDR stream, so:


x264.exe "AVS Script.avs" --x264-binary "x264.exe" --bitrate 25000 --preset medium --profile High --level 4.1 --ref 4 --keyint 24 --slices 4 --deblock -1:-1 --bluray-compat --overscan show --range tv --log-level info --thread-input --opencl --colormatrix bt709 --transfer bt709 --colorprim bt709 --videoformat component --nal-hrd vbr --vbv-maxrate 40000 --vbv-bufsize 30000 --aud --output "raw_video.h264"


ffmpeg.exe -i "AVS Script.avs" -vn -sn -af loudnorm=I=-24:LRA=14:tp=-2 -c:a dca -b:a 1536k -ar 48000 -strict -2 "audio.dts"

ffmpeg.exe -i "AVS Script.avs" -vn -sn -af loudnorm=I=-24:LRA=14:tp=-2 -c:a ac3 -b:a 384k -ac 2 -ar 48000 "audio.ac3"

"mkvmerge.exe" --output "raw_video.mkv" --language 0:und ^"^(^" raw_video.h264 ^"^)^"

ffmpeg.exe -i "raw_video.mkv" -i audio.dts" -i "audio.ac3" -map 0:0 -map 1:0 -map 2:0 -c:v copy -c:a copy -c:a copy -metadata provider_name="ABC" -metadata service_provider="ABC" -metadata service_name="ABC" -mpegts_original_network_id 0x1122 -mpegts_transport_stream_id 0x3344 -mpegts_service_id 0x5566 -mpegts_pmt_start_pid 0x1500 -mpegts_start_pid 0x150 -mpegts_service_type "digital_tv" -mpegts_flags "resend_headers" "final.m2ts"

pause


where, as always, keyint = framerate and in the muxing step, "ABC" should be changed to whatever your company is.



I hope this clarifies your doubts and solves your issue ;)



p.s it occurs to me that there's really a company called "ABC" in the U.S. Please note that I used "ABC" as an example of the first 3 letters of the alphabet and I DO NOT work for ABC nor have anything to do with them. I work for Sky.



Oh one last thing, we discussed about Indexing IMF with LWLibav here: https://forum.doom9.org/showthread.php?t=167435&page=71
and I was so happy to be able to do all those things in Avisynth that I brought it to the attention to the IMF User Group meeting I attended in Amsterdam: Link (https://www.imfug.com/workshop-report-imf-in-broadcast-workflows-ibc-2022/)

https://www.imfug.com/wp-content/uploads/2022/09/report-header-825x510.png

To quote the relevant part of the article:

the recent addition of an IMF demuxer to the FFmpeg, Avisynth and VapourSynth open-source toolkits has significantly reduced the barrier to adopting IMF.

Long live to open source and long live to Avisynth.
I'm the second guy on the right hand side, after Andrew from the BBC.
The only "sad" thing is that Amazon didn't send our Ben Waggoner (which was somewhere else in Amsterdam) but another guy. :'(

[excellentswordfight]

Thank you FranceBB, very much appriciate the help.

Yeah, so FFVideoSource converts anything that can't directly index inside Avisynth to 4:4:4 16bit internally. It does it for things like XYZ when it's not YUV and it does the same for RGB48.

Now, about LSMASH, it obviously cannot index it 'cause it's not an ISO standard, however if you use LWLibavVideoSource() and LWLibavAudioSource() you will be able to skip 1 step and index the CPL directly without going through an intermediate file in FFMpeg at all!
That was exatcly the info I was looking for, thank you.



#Going to yv12
ConverttoYUV420(matrix="Rec.709")
ConvertBits(bits=8, dither=1)


Yes thats pretty much what I was already trying, I guess that when you specify Rec709 its always limited range so I dont have to think about that in a separate step right?

This gives me the same results as when playing the mxf back in MPC with lavfilters (well apart from a loss in chroma res) .

if you have the BT2020 HDR PQ then it's slightly more complicated depending on what you wanna do:

1) Encode as BT2020 PQ 4:2:0 Type 2 10bit H.265

Yes, Im going for option 1, so that should be easier (i've dabbed in HDR conversions before, and I dont wanna go back in that rabbit hole again in my free time :) ).


#Indexing
video=LWLibavVideoSource("CPL_something.xml")
audio=LWLibavAudioSource("CPL_something.xml")
AudioDub(video, audio)

#Screw frame properties
propclearall()

#From RGB planar Full Range to YUV420 10bit planar Narrow Range with dithering
z_ConvertFormat(pixel_type="YUV420P10", colorspace_op="rgb:st2084:2020:full=>2020:st2084:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")


Why z_ConvertFormat? Cant you just do:

ConverttoYUV420(matrix="Rec2020")
ConvertBits(bits=10, dither=1)


Its still a bit interesting that even now when when I convert it in rather controlled fassion, there is both a luminance shift and colorshift (mostly reds as usual) compared to the Prores & AVC version that they offer. But I also saw encoding errors on the prores version, so Im not sure how carefully they have treated those versions...

[FranceBB]

Yes thats pretty much what I was already trying, I guess that when you specify Rec709 it's always limited range so I don't have to think about that in a separate step right?


Correct, when you go from RGB to YUV and you specify Rec709, Avisynth will convert from full range to limited tv range, so you're gonna be fine. :)



Why z_ConvertFormat? Can't you just do:

ConverttoYUV420(matrix="Rec2020")
ConvertBits(bits=10, dither=1)


You absolutely can, it was just me being a bit pedantic in specifying manually which resizing kernel to use for chroma downscale when downscaling to 4:2:0.
Anyway, yes, you totally can use just ConverttoYUV420() with the right colormatrix and it will automatically convert from Full Range to Limited TV Range.



Its still a bit interesting that even now when it seems like I can convert it correctly to yuv 420 8bit, there is both a luminance shift and colorshift (mostly reds as usual) compared to the Prores & AVC version that they offer.


Assuming that IMF is the real master from which they made the other mezzanine files, the answer is simple: they screwed up during *their* conversion to ProRes and H.264.
IMF decoding is absolutely accurate in Avisynth (so much so that we worked with Pierre Luis Lumiex from the IMF User Group on that) and I've compared it with other paid software like AVID Media Composer and Colorfront Transkoder and it was absolutely fine.


But I also saw encoding errors on the prores version, so Im not sure how carefully they have treated those versions...

Sloppiness, the great disease affecting companies ever since the digital era began...

TL;DR don't worry about that, they did something wrong, probably 'cause the operator doing that didn't care (which sadly is almost always the case in most companies), we did it right ;)

[excellentswordfight]

Assuming that IMF is the real master from which they made the other mezzanine files, the answer is simple: they screwed up during *their* conversion to ProRes and H.264.
IMF decoding is absolutely accurate in Avisynth (so much so that we worked with Pierre Luis Lumiex from the IMF User Group on that) and I've compared it with other paid software like AVID Media Composer and Colorfront Transkoder and it was absolutely fine.

Sloppiness, the great disease affecting companies ever since the digital era began...

TL;DR don't worry about that, they did something wrong, probably 'cause the operator doing that didn't care (which sadly is almost always the case in most companies), we did it right ;)
yeah, i suspect that you are right. Maybe ffmpeg actually handled it "automagically" with just using -pix_fmt, I discarded that directly cause I compared it to the Prores & AVC versions. Im actually gonna revisit that and compare it to the avisynth workflow. Eitherway, this at least gave me a reason to install avisynth+ for the first time, havnt been using avisynth for years.

While im at it, do you know if there is any benefit of feeding x264 with 10bit and use --input-depth 10 --output-depth 8, will it use those extra bits or just do a standard bit conversion then encode it as usual?

edit.
Yes, my original encode of just going through ffmpeg with pix_fmt does look alright (pngs exported from avspmod).

IMF source https://ibb.co/dkNhgqB

LWLibavVideoSource("StEM2_SDR_Rec709_444F_IMF_2160p24_178.mxf")
ConverttoYUV420(matrix="Rec709")
ConvertBits(bits=8, dither=1)
Spline36Resize(1920,1080)
converttorgb(matrix="Rec709")


Prores source https://ibb.co/MCFvGF7

LSMASHVideoSource("ASC_StEM2_178_UHD_24_100nits_Rec709_Stereo_ProRes422HQ.mov")
ConverttoYUV420(matrix="Rec.709")
ConvertBits(bits=8, dither=1)
Spline36Resize(1920,1080)
converttorgb(matrix="Rec709")


ffmpeg piped to x264 encode https://ibb.co/PrQky9V
ffmpeg.exe" -ss 00:00:08 -r 24000/1001 -i "StEM2_SDR_Rec709_444F_IMF_2160p24_178.mxf" -s 1920x1080 -sws_flags spline -pix_fmt yuv420p10le -an -f yuv4mpegpipe -strict -1 - |
"x264.exe" --demuxer y4m --fps 24000/1001 --input-depth 10 --output-depth 8 --profile high --level 4.1 --preset veryslow --tune film --bluray-compat --bitrate 25000 --vbv-maxrate 40000 --vbv-bufsize 30000
--keyint 24 --sar 1:1 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --pass 2 --stats x264_2pass.log --colorprim bt709 --transfer bt709 --colormatrix bt709 - -o stem2.1080p.x264.SDR.mp4"
LSMASHVideoSource("stem2.1080p.x264.mp4")
converttorgb(matrix="Rec709")

@benwaggoner you have used the dnxhd file right? Did you see the same issue there as with the prores?


And thank you for reminding me about chroma location, gonna start with the hdr10 version now:
"ffmpeg.exe" -ss 00:00:08 -r 24000/1001 -i "StEM2_HDR_Rec2020PQ_444F_IMF_2160p24_178.mxf" -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p10le -an -f yuv4mpegpipe -strict -1 - | "x265.exe" --y4m --fps 24000/1001 --profile main10 --level-idc 51 --preset slower --uhd-bd --no-sao --deblock -1:-1 --bitrate 50000 --vbv-maxrate 98000 --vbv-bufsize 98000 --pass 2 --hdr10-opt --chromaloc 2 --range limited --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --master-display "G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,50)" - -o stem2.2160p.HDR10.x265.hevc

[FranceBB]

While im at it, do you know if there is any benefit of feeding x264 with 10bit and use --input-depth 10 --output-depth 8, will it use those extra bits or just do a standard bit conversion then encode it as usual?


If you feed x264 with high bit depth, it will still perform dithering, however it will use Sierra-2-4A error diffusion instead of the Floyd Steinberg error diffusion that you have in Avisynth. Unless you have a good reason to use Floyd Steinberg inside Avisynth, it's better to feed x264 with the high bit depth and let it use Sierra-2-4A dithering 'cause the encoder will know where those dithered bits are and therefore will be able to encode it in a more efficient way compared to receiving an 8bit floyd-steinberg dithered down source (which isn't bitrate friendly). Now, here's the catch: unfortunately x264 only supports 16bit planar, 16bit interleaved and 8bit input, but nothing in between, therefore if you try to open an "AVS Script.avs" in 10bit planar with x264.exe it will say "unsupported bit depth". This is one of the things that annoys me most when I'm encoding professional XAVC Intra Class 300 10bit stuff with x264, 'cause if my source is 10bit and target is 10bit, I still have to go to 16bit planar and let x264 go down to 10bit.




IMF source https://ibb.co/dkNhgqB
Prores source https://ibb.co/MCFvGF7
x264 encode https://ibb.co/PrQky9V


Woah. :eek:
Whoever made the ProRes didn't bother to properly convert from Full PC Range to Limited TV Range, hence the screw up.

https://media.tenor.com/BKigh0ExUggAAAAM/congratulations-good.gif

Your x264 version looks fine. ;)

[FranceBB]

Actually, someone should really tell Jerome and open an issue on the Mediainfo bugtracker.


Done: https://github.com/MediaArea/MediaInfo/issues/657

[FranceBB]

Well, turns out it was an FFMpeg mxf muxer fault all along, so Mediainfo was actually right in reading the info out of a faulty file...

I reported it to the FFMpeg bug tracker using the info provided by Jerome: https://trac.ffmpeg.org/ticket/10001


Summary of the bug:

When muxing a MJPEG2000 RGB Full Range stream, the mxf muxer is using the CDCI essence descriptor UL instead of the RGBA essence descriptor UL.
In other words, it's using



{ { 0x06,0x0e,0x2b,0x34,0x02,0x53,0x01,0x01,0x0d,0x01,0x01,0x01,0x01,0x01,0x28,0x00 }, mxf_read_generic_descriptor, sizeof(MXFDescriptor), Descriptor }, /* CDCI */


instead of



{ { 0x06,0x0e,0x2b,0x34,0x02,0x53,0x01,0x01,0x0d,0x01,0x01,0x01,0x01,0x01,0x29,0x00 }, mxf_read_generic_descriptor, sizeof(MXFDescriptor), Descriptor }, /* RGBA */


On top of this, the mxf muxer adds the color sampling descriptor as 4:4:4 which doesn't make sense as it's RGB, not YUV.

Last but not least, instead of writing 10bit for 10bit RGB sources and 12bit for 12bit RGB sources, it always writes 16bit, which of course is also wrong.
This results from the fact that FFMpeg is trying to the first compatible match when there is no colr atom.


I also included the steps to reproduce as the following:

1) Download the standard IMF evaluation package from here: Link (https://aswf-dpel-assets.s3.amazonaws.com/asc-stem2/IMF/StEM2_TST_SDR_Rec709_IMF_App2e_Lossy.zip)

2) Create the final mxf file by reading the CPL with FFMpeg like ffmpeg.exe -f imf -i "CPL.xml" -c:v copy -c:a copy -f mxf -y "IMF.mxf"

3) Check the resulting file: it was supposed to be rgb48le 10bit BT709, however the muxed file has all the above mentioned things wrong; wrong bit depth inside the container and CDCI wrongly populated instead of RGBA, causing it to be interpreted as YUV 4:4:4 16bit by other softwares other than ffmpeg and ffprobe themselves.