Decoding ProRes - which way is correct?

[Lyris]

It doesn't seem like Quicktime is going away any time soon, so taming the myriad Quicktime problems is something I'll need to get on top of.

It seems like there is an almost infinite number of ways of decoding these files, the question is, which is actually correct? Let's say I want to funnel these into x264 or another AVC encoder using an AVISynth script - what's the best way? Has anyone mastered this yet? It seems ridiculous that there should be this many different results and that in 2013 we can't just define a pixel and have it come out of the other end of the system intact!

To give you an idea of what I mean:

Image 1: Copied out of the clipboard from the Apple Quicktime Player for Windows. This appears to either decode to video levels - or perhaps is doing something to the gamma - that makes the image look lighter. Banding in grayscale gradient.

http://img29.imageshack.us/img29/797/tpkn.png

---

Image 2: AVISynth script with QTInput(), loaded in VirtualDub, copied and pasted into Photoshop. The colour decoding is incorrect. (Virtualdub bug?) Also notice that there's somewhat messy dither in the magenta area. Banding in grayscale gradient.

http://img836.imageshack.us/img836/5071/51pj.png

---

Image 3: VLC Media Player. Same colour decoding problem as AviSynth QTinput -> VirtualDub but no dither. Banding in grayscale gradient.

http://img9.imageshack.us/img9/8525/6kyi.png

---

Image 4: Davinci Resolve, frames exported a TIFF sequence. Obviously that involves a conversion to RGB. This seems to be the best result - the colour bars line up with the software vectorscope graticules in the program. (FINALLY - some consistency!)

http://img826.imageshack.us/img826/4487/e99x.png

---

Image 5: Davinci Resolve TIFF output, loaded into AVISynth via ImageSource(), with a ConvertTOyv12 command, viewed in VirtualDub. Mostly correct but unsurprisingly given the RGB<->YV12 conversions, we have banding in the gradient now.

http://img9.imageshack.us/img9/2267/sb4w.png


---

So, is there some known-good way of dealing with these?

[poisondeathray]

RE: wrong colors

The reason is vdub converts to rgb (for display) using rec601 , so the preview is wrong

If you want to view it correctly in vdub , temporarily use ConvertToRGB(matrix="rec709") - but don't forget to comment or remove it before you actually encode (assuming you want YUV=>YUV, no other RGB filtering) .Or use avspmod which has view options / matrices you can toggle

Same with #5 - The TIFF output is correct because Resolve correctly uses 709 to convert to RGB. But you inadvertently used the wrong matrix when converting to YV12, that's the ony reason that looked "correct" in vdub (because vdub used 601 to convert it back to RGB for display)




Assuming you mean 8bit AVC encodes, there is no "best " way to do it, but usually it involves some type dithering in the 10bit =>8bit step. Some people do it with dither tools, or vapoursynth

[Lyris]

Thanks! It's about time I get used to AVSpmod. That discounts the colour decoding errors then.

[poisondeathray]

It's not really a decoding issue; For example, if you were using vdub video=>fast compress, there is no colorspace conversion (only the colorspace conversion to RGB for display in vdub) . The colors would look correct in the end product because it's a YUV preserved workflow (assuming whatever you used to view the end AVC product played it back correctly)

[Lyris]

That's what I've been doing - setting Vdub up to avoid conversions to and from RGB. Thanks for clarifying though - it's good to know that only the view is incorrect.

The gunk in the magenta area is bugging me - it's probably not really an issue for film content since anything like that will get lost in the grain anyway, but it rubs my video OCD the wrong way :P

[poisondeathray]

There shouldn't be any "gunk" in the magenta with a clean test pattern. Or are you talking about quicktime for windows ?

How were your source files derived ?

[Lyris]

The original source was HDCAM SR. Take a look at image 2 and look at the horizontal streaks in the magenta colour bar, and compare it to the output of Resolve.

[poisondeathray]

It must be an issue specific to your source.

I can't replicate those results with prores and various test patterns (yes, the wrong Rec601 colors in vdub - only for the preview, but no magenta streaks) , with QTInput() or FFVideoSource()

Can you upload a small cut ? (e.g. you can cut in QTpro if you have it)

[poisondeathray]

I'm going to take that back; and eat my words :D

qtinput default settings has a bit of horizontal striping at default settings. FFMS2 does not. You can see it with the histogram("luma") , the enhanced view .

It might have to do with the version of QT installed, or the version of QTSource.dll . I'll look into it deeper and report back

[poisondeathray]

Not sure what's going on. Sometimes QTInput decodes ok, other times it doesn't (almost invisible horizontal bars, seen easily with histogram("luma") ). FFMS2 is consistently ok . At first I thought it might be ffmpeg/ffmbc's version of prores encoding vs. Apple certified prores encoder, but the pattern isn't consistent. Another thing I don't like about QTInput is you lose overbrights

Here is a ffmbc encoded example (75% colorbars with a gradient) that exhibits the problem with QTInput() , ok with FFVideoSource() . Almost looks as if some dithering pattern applied to the gradient when using QTInput() (even though it's off by default, and I set dither=0 to double check) . I included an uncompressed 10bit 422 source (v210), and QTInput decodes that ok , only problems with the Prores version
QTInput("v210.mov", mode=1, raw="v210")

http://www.mediafire.com/download/xi28k4n8tngtm1f/prores.7z

[Lyris]

The plot thickens...

Too bad I can't get FFVideoSource to work... I have grey boxes jumping around all over the image when there's motion.

[poisondeathray]

The plot thickens...

Too bad I can't get FFVideoSource to work... I have grey boxes jumping around all over the image when there's motion.


What version of FFMS2 are you using?

[mp3dom]

Download the modified FFMS2 version of SAPikachu (here: http://nmm.me/vg). Use AviSynth 2.6 and open the video with:

FFVideoSource("video.mov",enable10bithack=true) #10bit in stacked format
f3kdb_dither() #flash3kyuu_deband, you can also use DitherTools
#You are in YV16 colorspace now.

That's it.

[kolak]

Download the modified FFMS2 version of SAPikachu (here: http://nmm.me/vg). Use AviSynth 2.6 and open the video with.

This is the best way.
QTinput+Vdub also works (thought 10->8bit is done without any dithering)- you have to set color depth to YUY2, to avoid going to RGB.
You can also add color=2 to make sure QT decoding is done to YUV.

update: new Vdub supports 709 color space now (if you need for some reason to go to RGB).

[Lyris]

Thanks both - just got myself set up with FFMS2 and it's working nicely. Glad I started this thread!