Sane ProRes 422 to RAW 10-bit YUV?

[kolak]

It's not my post :)

I just needed 10bit image from DNxHD to feed my dithering tool, but.....it's not easy :) As I said- it looks like only one software, which can see 10bit in DNxHD (except AVID) is AE. I rendered v210 in AE and feed this to get dithered 8bit file for encoding. It's just so much time and work- wish it would be simpler :)
It's RED footage and simple conversion to 8bit shows so much banding in many scenes, so it needs dithering.

In this case Cineform is very good- it has nice decoder, which can output many 10bit pins, including v210. I wish more people use Cineform:)

[jmac698]

Oh, well there's the r3dsource import filter, I just asked the author to update it to support 16bit depth. http://arenafilm.hu/alsog/avisynthr3d/
If you are on red forums, could you also ask? http://reduser.net/forum/showthread.php?25398

@daemon404: redirect what I said ;)

[kolak]

Soure is DNxHD file and this is what I have to work with:)

[Daemon404]

Awesome update... FFmpeg now supports prores decoding!

http://git.videolan.org/?p=ffmpeg.git;a=commit;h=5554de13b29b9bb812ee5cfd606349873ddf0945

problems = solved, for me.

[kolak]

In 10bit???


Andrew

[Daemon404]

In 10bit???


Andrew

Yes. It decodes to 10-bit properly.

[kolak]

Great!!

Speed?

Andrew

[poisondeathray]

wow that's good news

did anyone catch this:



author: Elvis Presley



:)

[mp3dom]

The only unsupported ProRes seems the 4:4:4:4 which is anyway very rare. It also support the ProRes Proxy.

[kolak]

Where can I download latest build?

[mp3dom]

Latest binary Win builds are here: http://ffmpeg.zeranoe.com/builds/ but I don't know if the ProRes decoder git is already builtin

Edit: Yes, it should already contains the ProRes decoder

[kolak]

It does not work- invalid pixel format -1

??? :(

[tateu]

Having ProRes support in ffmpeg and ffmpegsource is great, especially for those who refuse to install Quicktime and/or need 10 bit.

I compiled my own ffmpeg version and ffmpegsource plugin with ProRes support for testing but I haven't checked anything with 10 bit because I don't know if any of my ProRes files actually are 10 bit.

ffmpegsource decoded my test files to 8 bit data almost identically to qtinput in YUY2 mode. I had to create a histogram of both and zoom into them 400% to even see any differences. However, ffmpeg/ffmpegsource decodes about 3 times slower than qtinput.

Hopefully official builds off ffmpeg/ffmpegsource with 10 bit ProRes support get released soon.

[TheFluff]

Having ProRes support in ffmpeg and ffmpegsource is great, especially for those who refuse to install Quicktime and/or need 10 bit.

I compiled my own ffmpeg version and ffmpegsource plugin with ProRes support for testing but I haven't checked anything with 10 bit because I don't know if any of my ProRes files actually are 10 bit.

ffmpegsource decoded my test files to 8 bit data almost identically to qtinput in YUY2 mode. I had to create a histogram of both and zoom into them 400% to even see any differences. However, ffmpeg/ffmpegsource decodes about 3 times slower than qtinput.

Hopefully official builds off ffmpeg/ffmpegsource with 10 bit ProRes support get released soon.

If the ffmpeg decoder supports decoding 10-bit ProRes, ffms2 will support opening that as well. Output will be 8-bit only though, because no currently existing version of Avisynth supports high bitdepth YUV (the existing hacks that use various trickery to store 10-bit and up as something else do not count, and I am not particularly eager to implement such things in FFMS2).

[kolak]

Shame, because very soon we should have stable way of importing 10bit (or more) to avisynth. There is big progress and interest- all look very promising.

[TheFluff]

Shame, because very soon we should have stable way of importing 10bit (or more) to avisynth. There is big progress and interest- all look very promising.

Avisynth is already more than messy enough; especially the crazy MT extensions. Adding even more "interesting" hacks is not the Right Thing. If Avisynth is to continue being an useful tool, it needs a major overhaul, and the current state of 2.6 is not anywhere near enough.

[kolak]

So, the only thing left are hacks.
Maybe people are asking for to much for free- give us new avisynth 3, which will do everything- who is going to make it?
10bit support hack is quite simple (as far as I understand it) and already working properly- nothing like MT extension.

Andrew

[jmac698]

As I see it, if you insist on the one way and it's too big a job, it's not gonna get done by the volunteers. We live in an imperfect world. I'm perfectly fine coding to the 16bit standards, and there is a lot of support for it already. This support has meant that professionals are now able to use Avisynth in their work. It *is* a useful tool and becoming more so all the time.
I agree though, I'd like to see it redesigned someday *in an ideal world*.

[TheFluff]

As I see it, if you insist on the one way and it's too big a job, it's not gonna get done by the volunteers. We live in an imperfect world. I'm perfectly fine coding to the 16bit standards, and there is a lot of support for it already. This support has meant that professionals are now able to use Avisynth in their work. It *is* a useful tool and becoming more so all the time.
I agree though, I'd like to see it redesigned someday *in an ideal world*.

Just as you say, I'm an unpaid volounteer, and I don't want to work on code that makes me cringe.

On the other hand, FFMS2 is open source and if you want to add hacks to output high bitdepth YUV to the Avisynth plugin, feel free. It shouldn't be particularly hard; the FFMS2 API is clean and well documented and the Avisynth plugin is only a few hundred lines of code. If you do it well, you can even have your own branch on the main repository if you want it. I don't want it in the official plugin, though.

[jmac698]

The requested URL /ffmpegsource.googlecode.com/files/ffms-2.16-src.tar.bz2 was not found on this server. That’s all we know.

Make your source available and I'll look into it.

[TheFluff]

The requested URL /ffmpegsource.googlecode.com/files/ffms-2.16-src.tar.bz2 was not found on this server. That’s all we know.

Make your source available and I'll look into it.

The .tar.bz2 is right there and the download link works fine for me. I suggest you use the SVN repository (http://code.google.com/p/ffmpegsource/source/checkout), though.

[jmac698]

My bad.
I don't see where I can change things. I see that each output frame is blitted from the decoded copy made by ffmpeg, I don't see any codec source code here, so I need the source to ffmpeg too? I'm thinking you have to dig deep to get stuff to start passing high-bit data. Btw, VFW has the same problem, it's designed for 8bit, and v210 packs 10bit into an existing structure, probably a hack as you'd call it. Anyhow, there is someone writing an extension to avisynth to directly pass high-bit data. It already works, would you use it?

http://forum.doom9.org/showthread.php?t=162436

Here's how you would adapt ffms2:

PVideoFrame __stdcall hqas::ConvertHQ16ToRGB24::GetFrame(int n, IScriptEnvironment* env)
22 {
23 PVideoFrame src = child->GetFrame(n, env);
24 frame_info const &srcfi = get_info(*env, src);
25
26 PVideoFrame dst = new_frame(*env, vi);
27
28 bitmap<uint8_t, 3> out = bitmap<uint8_t, 3>(dst->GetWritePtr(), dst->GetRowSize() / 3, dst->GetHeight(), dst->GetPitch());
29
30 bitmap<uint16_t const> r = get_read_bitmap<uint16_t>(src, srcfi, planes::r);
31 bitmap<uint16_t const> g = get_read_bitmap<uint16_t>(src, srcfi, planes::g);
32 bitmap<uint16_t const> b = get_read_bitmap<uint16_t>(src, srcfi, planes::b);
33
34 for(int y = 0; y < out.height(); ++y)
35 {
36 uint8_t *outbgr = out.get_row(y);
37 uint16_t const *inr = r.get_row(y), *ing = g.get_row(y), *inb = b.get_row(y);
38
39 for(int x = 0; x < out.width(); ++x)
40 {
41 *outbgr++ = uint8_t(*inb++ >> 8);
42 *outbgr++ = uint8_t(*ing++ >> 8);
43 *outbgr++ = uint8_t(*inr++ >> 8);
44 }
45 }
46
47 return dst;
48 }

Here's hoping that didn't make you cringe :)

[surfdog]

I understand that once a video comes out of avisynth that it is 8-bit. What I'm wondering is: at what point, inside avisynth, does a 10-bit video get converted to 8-bit? Is it by the input filters (such as qtinput or rawinput) or is it by the use of other "non 10 bit" filters? I'm sure those here know that the person doing the dithering filter has a way of operating on more than 8 bits(http://forum.doom9.org/showthread.php?t=153589). Does qtinput, for prores, (or any input filter for that matter) preserve all 10-bits of each color channel when it opens the video? Thanks.

[jmac698]

qtinput has access to the 10bit video, but two bits are just discarded and 8 bits are sent to avisynth. Avisynth normally only allows to pass 8bit video internally. We get around this by using an extra large height video, stuffing the extra bits into the bottom half. Avisynth still thinks it's 8bit, but we process the bottom half separately to preserve the data. When it's time to output, we again arrange it in a special way to feed to avs2yuv and to x264.

[TheFluff]

My bad.
I don't see where I can change things. I see that each output frame is blitted from the decoded copy made by ffmpeg, I don't see any codec source code here, so I need the source to ffmpeg too?
You need the ffmpeg source code if you intend to compile FFMS2 yourself, yes.

I'm thinking you have to dig deep to get stuff to start passing high-bit data.
No. I don't think you've understood how the FFMS2 Avisynth plugin works at all.

What you need to do:

First you need to get 10-bit raw video frames from the FFMS2 API. You can do this by first getting the appropriate constant (either by calling FFMS_GetPixFmt (http://code.google.com/p/ffmpegsource/source/browse/trunk/src/avisynth/avssources.cpp#196) or by looking at libavutil/pixfmt.h directly) and then passing it to FFMS_SetOutputFormatV2 (http://code.google.com/p/ffmpegsource/source/browse/trunk/src/avisynth/avssources.cpp#222). The appropriate pixel format in this case is probably PIX_FMT_YUV420P10LE.
Next you need to decide under what circumstances the Avisynth plugin should output this pixel format. Since it is decidedly nonstandard, I'd say it's a bad idea to output it by default. You should probably make up a name for it and add it to CSNameToPIXFMT (http://code.google.com/p/ffmpegsource/source/browse/trunk/src/avisynth/avsutils.cpp#42), so the user can request it via the colorspace parameter to FFVideoSource.
Finally you need to add support for copying this new output format into an Avisynth frame in whatever funny way the other plugins supporting this hack wants.


Taking a closer look at the wiki page about high bitdepth makes me even more negative to the entire thing. Why on earth would you split each sample into two parts and store them in different halves of the frame? Seems like a recipe for a big performance hit to me. It also makes it harder to implement support for it in FFMS2. You can't blit the frames delivered from FFmpeg directly, since YUV420P10LE stores each sample as a 16-bit unsigned little-endian integer, so if you want to implement this I hope you like pointer arithmetics and bit shifts.

I'm not 100% sure what that code you posted does, but I don't think it does what you think it does and I don't think it's relevant to adding 10-bit support to FFMS2.

[poisondeathray]

The appropriate pixel format in this case is probably PIX_FMT_YUV420P10LE.

Wouldn't it be 422 subsampled instead of 420 ?

Does this follow ffmpeg naming conventions?

If so, the pix fmt for v210 in ffmpeg is "yuv422p10le"

[cretindesalpes]

Why on earth would you split each sample into two parts and store them in different halves of the frame?
I am responsible for this monstrosity ^^. It's absolutely disgusting on a programming point of view and terribly inefficient, but it was very convenient when I started the experiment with simple Masktools hacking. Anyway, I fear we'll have to live with it for a while. I clearly don't have time to add a well-designed high bitdepth support to the Avisynth code base, moreover it's not planned (http://forum.doom9.org/showpost.php?p=1179134&postcount=33) in v2.6.

[TheFluff]

Wouldn't it be 422 subsampled instead of 420 ?

Does this follow ffmpeg naming conventions?

If so, the pix fmt for v210 in ffmpeg is "yuv422p10le"

Well yeah, if the input is v210 that'd be the correct pixel format. I just sorta assumed 4:2:0 for some reason.

Anyway, I fear we'll have to live with it for a while. I clearly don't have time to add a well-designed high bitdepth support to the Avisynth code base, moreover it's not planned (http://forum.doom9.org/showpost.php?p=1179134&postcount=33) in v2.6.

That's exactly the problem. Avisynth 2.6 really doesn't add very much. At this point I'd almost say it's time to start something new from scratch, because hacking on Avisynth doesn't seem to cut it at all, but I also think that down that road lies madness.


Anyway, to contribute, I give you a quick and dirty example patch (http://privatepaste.com/c1ab0c40fa) to get any interested parties started on hacking the support into FFMS2. It's obviously not complete and I have not even tested if it compiles at all, but it should give you an idea of where to start modifying things.

[jmac698]

We can change the standard to exactly the one ffmpeg uses. It is efficient for 16bit plugins, and can be used in script too. It's not even a hack; there's only one problem that the width reported will be double, but that's easy to work around.

I'm trying to tell you there is already true high-bit depth support internally for plugins for avisynth, and there's working code. What I posted showed reference to true 16bit integers as they were converted down to 8bit rgb. In fact, simply adding the video type without changing anything else is enough to do a lot.

[TheFluff]

I'm trying to tell you there is already true high-bit depth support internally for plugins for avisynth, and there's working code. What I posted showed reference to true 16bit integers as they were converted down to 8bit rgb.
You said "here's how to adapt FFMS2" and posted code that converts 16-bit samples to 8-bit ones. FFMS2 already does that internally and the discussion was about how to add >8-bit output to FFMS2, so I don't see how the code was relevant at all.

In fact, simply adding the video type without changing anything else is enough to do a lot.
What?

edit: if you want to change the standard to storing samples together (doubling the width instead of the height), adding support for that to FFMS2 would be absolutely trivial, at least for 4:2:0. Unfortunately 4:2:2 won't be as easy, since Avisynth only supports packed 4:2:2 (YUY2) while FFmpeg only supports planar 4:2:2 when using >8 bits per component.

[jmac698]

Hey.. thanks for contributing an example! And yes, the trivial support for 4:2:0 would immediately make a huge difference to the practical application of high bit-depth support.

Yes, I know the code doesn't do anything useful in itself, I simply wanted to point out the fact that 16bit integer video was now an internal video type. What that code for a plugin did was take avisynth's new internal high-bit video type and convert it to an 8 bit avisynth internal type. Anyhow, I've invited PhrostByte (the author) here to explain it to you.

If you could make your example do the "trivial" thing you mentioned, that would be great! Really, it's almost all we need.

>simply adding the video type
What I mean is just having a clip that can be 16bit (yuv) and being able to read the correct properties from it, set interlace, etc. even if no internal filters work with it.. is enough to do practical and *clean* scripting with it. For example I could write a script function that checks for 16bit video and uses masktools or 16bit plugins to do some very simple operations on it, including converting back to 8bit, or simply leaving it as it is and encoding directly to 10bit with avs2yuv.

[TheFluff]

If you could make your example do the "trivial" thing you mentioned, that would be great! Really, it's almost all we need.

Something like this (http://privatepaste.com/bcc541cb3a) maybe? Not tested at all, haven't even tested if it compiles.

>simply adding the video type
What I mean is just having a clip that can be 16bit (yuv) and being able to read the correct properties from it, set interlace, etc. even if no internal filters work with it.. is enough to do practical and *clean* scripting with it. For example I could write a script function that checks for 16bit video and uses masktools or 16bit plugins to do some very simple operations on it, including converting back to 8bit, or simply leaving it as it is and encoding directly to 10bit with avs2yuv.

This can't be done without patching Avisynth itself.
I mean, it's certainly possible to treat env->NewVideoFrame() as if it was malloc() and just put random data into the frame buffer, and even make filters use that, but until someone modifies Avisynth itself it'll be impossible to tell such random data apart from ordinary video, so you'll have to trust the user to tell filters what to expect. At least unless you do something even more hackish like hiding metadata outside the active image area or in the LSB of certain pixels like telecide does with its hinting, but even then you probably can't detect it within a script.

[Wilbert]

Unfortunately 4:2:2 won't be as easy, since Avisynth only supports packed 4:2:2 (YUY2)
planar YUY2: YV16.

[TheFluff]

planar YUY2: YV16.

Well, Avisynth 2.5 then. If you want to use Avisynth 2.6's new features you'll need to use the C plugin though, because the native one doesn't support those yet.

[Wilbert]

If you want to use Avisynth 2.6's new features you'll need to use the C plugin though, because the native one doesn't support those yet.
I'm sure IanB will incorperate your modifications of the header file if you send it to him. I guess it's this one?

http://code.google.com/p/ffmpegsource/source/browse/branches/c_plugin/src/avisynth_c/avisynth_c.h

[TheFluff]

I'm sure IanB will incorperate your modifications of the header file if you send it to him. I guess it's this one?

http://code.google.com/p/ffmpegsource/source/browse/branches/c_plugin/src/avisynth_c/avisynth_c.h

No, I meant the native FFMS2 plugin cannot be built as an Avisynth 2.6 plugin yet.

The modded avisynth_c.h is kemuri_-9's department and I leave any comments regarding it to him.

[jmac698]

>until someone modifies Avisynth itself
Yep,
http://svn.int64.org/viewvc/int64/hqas/hqas/avisynth.h?view=log
It's a modded version of avisynth 2.58.

[surfdog]

qtinput has access to the 10bit video, but two bits are just discarded and 8 bits are sent to avisynth. Avisynth normally only allows to pass 8bit video internally. We get around this by using an extra large height video, stuffing the extra bits into the bottom half. Avisynth still thinks it's 8bit, but we process the bottom half separately to preserve the data. When it's time to output, we again arrange it in a special way to feed to avs2yuv and to x264.

Thanks for the response jmac, I've tried out your deep color tools before too. Yes I get what you're saying , what I'm really wondering is with the following line of avisynth code:
videoHolder = qtinput(10bit_yuy2_proresVideo.mov,color=2,audio=true,mode=1,raw="v210")
Once avisynth passes this line of code, does the variable 'videoHolder' hold the true 10-bit version of the video or just a 8-bit version?

[jmac698]

qtiinput by itself only returns 8 bit. There's two ways to get the rest; one is an option in qtinput that can round up. By comparing the roundup and normal you can tell what is the extra bit. The other way is to use sashimi, this lets me read the raw file. In this case I get a video that looks like garbage but happens to have the 10bit data. I unscramble the garbage into 16bits with masktools. Then I have the true 10 bits in way way I want, but some other tools use vertically stacked msb/lsb for further processing.

[surfdog]

qtiinput by itself only returns 8 bit. There's two ways to get the rest; one is an option in qtinput that can round up. By comparing the roundup and normal you can tell what is the extra bit. The other way is to use sashimi, this lets me read the raw file. In this case I get a video that looks like garbage but happens to have the 10bit data. I unscramble the garbage into 16bits with masktools. Then I have the true 10 bits in way way I want, but some other tools use vertically stacked msb/lsb for further processing.

Yes I have tried the roundup method with little success (dither =1). Do you mind sharing the script in which you open a 10-bit .mov file (preferably prores format) using the sashimi plugin method you mentioned above?

[jmac698]

Just setting dither to 1 doesn't show much difference, I actually take dither=1 and subtract dither=0, this leaves *only* the low bits (which looks black with some noise). Anyhow, I wrote a script to read 9 bit video from any quicktime file include prores http://forum.doom9.org/showthread.php?p=1467907#post1467907
I haven't tested prores, only uncompressed 10 bit as v210 in .mov.

There is also readv210, ffvideosource(fn) and qtinput(fn) also work but they are only 8bit, in the case of qtinput you can fudge 9bit with a trick. With this script you can access the full 10bits in script.
The latest version:
ReadV210 - http://forum.doom9.org/archive/index.php/t-158985.html Can read uncompressed v210 in .mov container as greyscale.

Readv210mod - http://forum.doom9.org/showthread.php?p=1526131#post1526131 Can read uncompressed v210 in .mov container as color. Both need some manual tweaking to skip the file header. Update: link fixed

[surfdog]

Just setting dither to 1 doesn't show much difference, I actually take dither=1 and subtract dither=0, this leaves *only* the low bits (which looks black with some noise). Anyhow, I wrote a script to read 9 bit video from any quicktime file include prores http://forum.doom9.org/showthread.php?p=1467907#post1467907
I haven't tested prores, only uncompressed 10 bit as v210 in .mov.

There is also readv210, ffvideosource(fn) and qtinput(fn) also work but they are only 8bit, in the case of qtinput you can fudge 9bit with a trick. With this script you can access the full 10bits in script.
The latest version:
OK. Ya I've tried you're deep color script before and stopped when I realized i was only getting 9 out of the 10 bits from qtinput (that's not to say that you're script is at fault or anything). I'm wondering if your last post was submitted the way you meant it to be. The last sentence says "With this script you can access the full 10bits in script.
The latest version:" and then there is no link. Is this what you meant to put?
edit: oh I see that you changed it. thanks.

edit:
I tried the Readv210mod link and it doesn't seem to be working.

[jmac698]

Fixed.

[kolak]

There are new ffmpeg builds, but still no ProRes decoding :(

[jmac698]

http://www.videohelp.com/tools/ffmbc

FFmbc-0.7:
- Sync on FFmpeg git 7cbb856efe6ccab7485bb96ad3887472a6519ffa
- AVCIntra decoding is now supported.
- Parallel frame multi-threaded decoding is now enabled.
- Prores 422 and Prores 4444 decoding.
- When writing MXF files, use more correct key for KLV Fill packets.
- Fix detection of some DVCAM files.
- Fix decoding artefact in bottom rows of Quantel DVCPROHD 1080i files.

[kolak]

Cool :)
Where can I get compiled version?
I assume 8bit only?

[jmac698]

I dunno, I'll see what I get it compiled :)
Btw, if you're looking for a guide to compile ffmbc, it's probably the same as a guide to compiile ffmpeg.

[smok3]

has anyone get the prores decoding to work? (with ffmpeg & friends)? (on any os)

[the_weirdo]

AFAIK, libavcodec's ProRes decoder is licensed under GPLv2, so you would need to compile ffmpeg/libav as GPLv2 (use options --enable-gpl --enable-version2 for configure). Many ffmpeg/libav builds are compiled as GPLv3 so ProRes decoder doesn't work.

[jmac698]

Wow, good tip, I'd never think of that. Oh, and it just finished compiling. But I had passed --enable-version3 before and it said it didn't recognize that option?
Hey, there's an exe here! And... I'm missing pthreadGC2.dll :(