handling mxf source

[Lebowsky]

Long story short, someone is sending me a file in .mxf format, and I need to add subtitles to it and render the whole as a DVD. Normally I'd use multiavchd to do it all, but it seems like it doesn't support mxf as input source. Said someone is not able to send it to me in another format.

I was told multiavchd can be fed an avs script. What is the best way to read an mxf file in avisynth?

[FranceBB]

AFAIK FFMpeg actually supports .MXF files, but you'll have to mux your file in .mkv. After that, you should be able to index it. As to the audio, it should be able to read it, but I strongly suggest you to extract it and convert it to WAV (lossless) and then use AudioDub(). If you are not too comfortable with FFMPEGSOURCE2, you could just convert the MXF in a lossless AVI file using Xmedia Recode, but I discourage it, since FFMpeg should work.

[Lebowsky]

thanks, I'll try with FFMPEGSOURCE2!

[FranceBB]

FFmpegSource2("video.mkv", fpsnum=24000, fpsden=1001, atrack=-1, seekmode=1, rffmode=0, colorspace="YV12")
# 23,976fps. Adjust settings to your source.
#OR:

video=FFVideoSource("video.mkv", fpsnum=24000, fpsden=1001)
audio=FFAudioSource("audio.wav")
Audiodub(video, audio)

#Remember to mux the MXF using mkvmerge and export the audio track to wav first ;)

[Lebowsky]

yeah I don't know. thanks for the suggestion above, but for some reason the result only is 10 minutes long instead of 1h16m. I had the same problem with Xmedia Recode, it was also outputting a 10 minutes file long. I guess the source is fubar.

[Sparktank]

Try using ffprobe on it to read source info.
"C:\Apps\VidEnc\ffmpeg_64\ffprobe.exe" "F:\Samples\Sintel 2010\Sintel [4K Remastered] [DCP]\video.mxf" 2>>"Sintel-FFProbe.txt"

Sintel-FFProbe.txt
Duration: 00:14:48.00, start: 0.000000, bitrate: 236980 kb/s
Stream #0:0: Video: jpeg2000 (JPEG 2000 digital cinema 4K), xyz12le, 4096x2160, SAR 1:1 DAR 256:135, 24 tbr, 24 tbn

xyz12le
12-bit video in XYZ colorspace. Not supported by ffms2 or lsmash, iirc.

DSS2 works with XYZ and converts to whatever is set in LAV VIDEO.
I have LAV Video set to output YV12 for FFDShow (to use with SVP interpolation).
So when I view Sintel MXF/DCP, it shows me YV12 colors.

Going to be pretty slow. Depending on whatever the source format is.
If it turns out to be yuv420p or yuv422p, that'll be easier as it's 8-bit.

[Lebowsky]

Try using ffprobe on it to read source info.

Thanks for the tip. I'm guessing theres a problem with the file's index...

W:\FFmpeg\bin>ffprobe source.mxf
ffprobe version N-79440-g0efafc5 Copyright (c) 2007-2016 the FFmpeg developers
built with gcc 5.3.0 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmfx --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --
enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
libavutil 55. 20.100 / 55. 20.100
libavcodec 57. 34.100 / 57. 34.100
libavformat 57. 34.100 / 57. 34.100
libavdevice 57. 0.101 / 57. 0.101
libavfilter 6. 41.101 / 6. 41.101
libswscale 4. 1.100 / 4. 1.100
libswresample 2. 0.101 / 2. 0.101
libpostproc 54. 0.100 / 54. 0.100
[mxf @ 02e2b160] broken or empty index
Input #0, mxf, from 'source.mxf':
Metadata:
uid : 0003475a-f756-4ab5-b736-d9c5484397ed
generation_uid : 0000475a-f756-4ab5-b736-d9c5484397ed
company_name : Apple Inc.
product_name : AppleMXFExportFCP
product_version : 12
product_uid : 00000000-0000-0000-0000-000000000000
modification_date: 2015-09-30 18:21:52
material_package_umid: 0x060A2B340101010501010D00130000000073081E121534000267A3F5AA4A102B
timecode : 09:58:30:00
Duration: 01:17:55.16, start: 0.000000, bitrate: 7974 kb/s
Stream #0:0: Video: mpeg2video (4:2:2), yuv422p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 50000 kb/s, 25 fps, 25 tbr, 25 tbn
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:1: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:2: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:3: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:4: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:5: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:6: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:7: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:8: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD

W:\FFmpeg\bin>

[Sparktank]

That may be why you had indexing issues with it in ffms2.

Good news is that it's yuv422p. 8-bit YV16/YUV2 (try to use YV16 instead of YUY2 plugins, as YUY2 resamples? [needs citation]).
So if you get anything working on it, it won't take a very long time compared to >10-bit sources.

Try L-Smash Works on it:
http://forum.doom9.org/showthread.php?t=167435
updated libraries in the Dropbox link.

LoadPlugin ("LSMASHSource.dll")
LWLibavVideoSource ("soure.mxf", stacked = false, format = "yuv422p")
and it should create a small index file. The indexing should be faster than ffsm2.

If there's problems with latest lsmash, try downloading the previous version. They keep them in the "OLD" folder in the dropbox. Which is handy.

If none of that really works at all, try use ffmpeg to convert to a lossless intermediate.
Don't know how well ffmpeg + UT Video work. But if that manages to work, you can try that then use Avisource("lossless-ULH0.avi").

Add -report to your command so it saves a log with info to check what happened during process. Might reveal something else.

[Lebowsky]

Try L-Smash Works on it:
http://forum.doom9.org/showthread.php?t=167435
updated libraries in the Dropbox link.

If there's problems with latest lsmash, try downloading the previous version. They keep them in the "OLD" folder in the dropbox. Which is handy.

yeah no go. Just says "Unable to open file". Tried with older version as well. And I know I'm using it properly, because the first time I had another error due to a missing VC++ lib.


If none of that really works at all, try use ffmpeg to convert to a lossless intermediate.

Add -report to your command so it saves a log with info to check what happened during process. Might reveal something else.

I'm getting the same problem as with ffprobe. Conversion fails after 10mins in the video:

[mpeg2video @ 0476fe40] ac-tex damaged at 30 17=00:10:20.80 bitrate=3385.3kbits/s speed=4.11x
[mpeg2video @ 0476fe40] Warning MVs not available
[mpeg2video @ 0476fe40] concealing 6120 DC, 6120 AC, 6120 MV errors in I frame

Logfile is hereunder, if that helps... But I'm guessing the file really is borked...

ffmpeg-20160415-181351.log (http://www.0wned.ch/tmp/ffmpeg-20160415-181351.log)

edit: tried converting to huffyuv as well, same result...

[Lebowsky]

I got a new MXF source, 'slightly' bigger (35gb). But this time ffprobe does not complain. I was able to index it using ffmpeg2source. Now two questions,

how do I extract the audio from it?
how should I handle the interlacing? I was kinda hoping for a progressive file...


Format : MXF
Commercial name : XDCAM HD422
Format version : 1.2
Format profile : OP-1a
Format settings : Closed / Complete
File size : 32.7 GiB
Duration : 1h 17mn
Overall bit rate : 60.0 Mbps
Encoded date : 2015-09-30 18:21:52.332
Writing application : Apple Inc. AppleMXFExportFCP 12

Video
ID : 2
Format : MPEG Video
Commercial name : XDCAM HD422
Format version : Version 2
Format profile : 4:2:2@High
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : M=3, N=12
Format settings, picture structure : Frame
Format settings, wrapping mode : Frame
Codec ID : 0D01030102046001-0401020201040300
Duration : 1h 17mn
Bit rate : 50.0 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 fps
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.965
Stream size : 27.2 GiB (83%)
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Input #0, mxf, from 'source.mxf':
Metadata:
uid : 0003475a-f756-4ab5-b736-d9c5484397ed
generation_uid : 0000475a-f756-4ab5-b736-d9c5484397ed
company_name : Apple Inc.
product_name : AppleMXFExportFCP
product_version : 12
product_uid : 00000000-0000-0000-0000-000000000000
modification_date: 2015-09-30 18:21:52
material_package_umid: 0x060A2B340101010501010D00130000000073081E121534000267A3F5AA4A102B
timecode : 09:58:30:00
Duration: 01:17:55.16, start: 0.000000, bitrate: 59993 kb/s
Stream #0:0: Video: mpeg2video (4:2:2), yuv422p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 50000 kb/s, 25 fps, 25 tbr, 25 tbn
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:1: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:2: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:3: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:4: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:5: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:6: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:7: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD
Stream #0:8: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
Metadata:
file_package_umid: 0x060A2B340101010501010D00130000000073081E121534002A3A4B1BC39B2AAD

[Sparktank]

That's great news.

Extracting audio is going to be tedious:
Windows command prompt to use ffmpeg:
cls
"ffmpeg.exe" -i "source.mxf" -map 0:1 -acodec pcm_s24le -map_metadata -1 "output-01.wav"

cls
"ffmpeg.exe" -i "source.mxf" -map 0:2 -acodec pcm_s24le -map_metadata -1 "output-02.wav"

"cls" is not really needed, but I like it to keep the console window clean and neat.

Do that all the way up to stream 8.

I don't know what the audio actually is. If it was recorded for 7.1 or not.
It could be stereo really, with a 7.1 mask and the rest of the channels blanked.
You could run a .bat on the extracted mono wave files to see if there's any audio in them, if you don't want to play them all or if your source didn't specify.
@echo off
%~dp1
CD %~dp1
FOR %%A IN (*.wav) DO "fmpeg.exe" -i "%%~nxA" -y -filter_complex "compand,aformat=channel_layouts=mono,showwavespic=s=1920x1080" -frames:v 1 "%%~nA-mono-compand.png"
You'll get one waveform.png for each the mono wave files. And looking at them all, you'll see whether or not it's real 7.1 or 5.1 in a 7.1 or 2.0 in a 7.1 or something.

There's page for audio manipulation in ffmpeg:
https://trac.ffmpeg.org/wiki/AudioChannelManipulation

for 5.1:
"ffmpeg.exe" -i "output-01.wav" -i "output-02.wav" -i "output-03.wav" -i "output-04.wav" -i "output-05.wav" -i "output-06.wav" -filter_complex "[0:a][1:a][2:a][3:a][4:a][5:a]amerge=inputs=6[aout]" -map "[aout]" -acodec pcm_s24le "output-6ch.wav"

For interlace, when it comes to mxf/prores/etc, it could be just progressive really.
It's all 25fps, even by ffprobe readings.
Open the script (without addnig any deinterlacing) in avspod and hit F5 to 'refresh' (load) video preview.
At bottom left, and it should say what the FPS is.
It could just be progressive in fake-interlace.

Otherwise, simple tfm().TDecimate() should suffice?
You would have to play and double-check that it would work like that.

[Lebowsky]

ok I'll try the audio extraction later.. if I can get the video working. Actually ffmpegsource2 gave me weird results. Like same frame 15 times in a row, very choppy... LWLibavVideoSource worked fine.

I assume you mean avspmod? It says 25fps. Tried tfm().TDecimate() but it seems to decimate far too many frames, and leave a lot of interlaced ones anyway... nnedi3 seems to do the trick though. Dunno how good it's supposed to be but it seems satisfying for now.

I'll encode to x264 tomorrow and report back. Sparktank HUGE thanks.

[Sparktank]

I assume you mean avspmod?

Ah yeah, sorry. had a long day yesterday and fragmented sleep.
Also, I meant to say bottom-right of the interface.
Bottom left is other stuff.

[Lebowsky]

ok, the nnedi3 x264 encode looks good! I'm now trying using QTGMC to see if the quality is better but it's gonna take some time...

I extracted the audio only first 2 channels actually had audio, and the content is the same. I'm guessing I could either:
ffmpeg -i output-0.wav -filter_complex "[0:a][0:a]amerge=inputs=2[aout]" -map "[aout]" -acodec pcm_s24le output.wav

or:
ffmpeg -i output-0.wav -i output-1.wav -filter_complex "[0:a][1:a]amerge=inputs=2[aout]" -map "[aout]" -acodec pcm_s24le output.wav

Since the files seem to be the same I suppose there is zero difference, but in the case there might be a difference I didn't notice, second one would be more appropriate?

[Sparktank]

You could ask the source how they recorded.
But, probably sounds like dual-mono or something inside a stereo.

Run both through Foobar2000 to do "bit-compare" on them and see if they turn out different.
If it has any stereo differences, FB2K will report a difference in volume in the samples.

[FranceBB]

As to the video, QTGMC is one of the best deinterlace filters.

Depending on whether your source is actually interlacing or telecined, you could try this way:

#if interlaced
tdeint(mode=2, order=-1, field=-1, mthreshL=6, mthreshC=6, map=0, type=2, debug=false, mtnmode=1, sharp=true, cthresh=6, blockx=16, blocky=16, chroma=false, MI=64, tryWeave=true, link=1, denoise=true, slow=2, opt=4)

#if telecined
tfm(mode=1,pp=5,slow=2,micmatching=2,clip2=tdeint(mode=2,type=3))
tdecimate()

#If you feel more conformatble with QTGMC, (interlaced source), try:
QTGMC(Preset="placebo", Sharpness=1.00)