Compressing 10b v210 video to 8b YUY2?

[frustum]

First, my question:

Is there any means of mapping the full 10b range of component data to 8b before it hits the 8b choke point on entry to avisynth?

What would be perfect would be if virtualdub's codec for this format had the ability to specify a transfer function to map the original value down to 8b components. It doesn't, as far as I know, but hoped someone here knew of some tool chain that would allow doing something like that. Otherwise, I guess I need to wait until avisynth has fat components.

Despite the poorly chosen title, converting to 8b YV12 would be as useful to me.

Background:

I had a film transfer house (cinepost, did a nice job) capture my family's 8mm movies. I'm using videofred's excellent script to enhance the quality and upconvert to 30 fps. Yes, it does result in more of a video look, but I don't care about duplicating the 8mm feel; I just wanna see the pictures.

Anyway, cinepost uses a rank cintel machine and I had them capture to uncompressed v210 AVI format -- 10b components, 4:2:2 sampling. I did this despite knowing that avisynth can't deal with 10b components. I have hope I can make use of those extra bits in some way, later, if not sooner.

Currently I just take the avi file, open it with virtualdub, and ask it to write it out as huffyv encoded yv12. Yes, I realize I just lost half of my chroma there, but fred's conversion scripts use plugins that can't deal with yuy2. At some point I might see if I can work around that, but that isn't the main issue right now.

On a number of reels, there are scenes which are way underexposed, or worse yet, the foreground subject is way underexposed and the background is bright. Doing the usual tricks only gets so far; those extra two bits would be useful to restore detail in the dark areas.

Thanks in advance for any suggestions.

[dansrfe]

I don't really think you can do anything with the extra 2 bits. If you want it at 8bit then it means what it says, losing 2 bits. Also, I'm not quite sure why you didn't have your 8mm film capture saved to uncompressed 8 bit RGB at 4:4:4. Would have made things a lot simpler although 4:2:2 doesn't make any visual difference I presume.

[Dark Shikari]

I don't really think you can do anything with the extra 2 bits. If you want it at 8bit then it means what it says, losing 2 bits. Also, I'm not quite sure why you didn't have your 8mm film capture saved to uncompressed 8 bit RGB at 4:4:4. Would have made things a lot simpler although 4:2:2 doesn't make any visual difference I presume.x264 has a proper high bit depth pipeline that can dither down to 8-bit for you.

[poisondeathray]

On a number of reels, there are scenes which are way underexposed, or worse yet, the foreground subject is way underexposed and the background is bright. Doing the usual tricks only gets so far; those extra two bits would be useful to restore detail in the dark areas.



You could do parts of the restoration in 10bit or higher capable software, until the last few stages where you dither down to 8bit

Use masks/rotoscoping to isolate foreground vs. background elements

[frustum]

Thanks for the quick replies, everyone.

dansrfe -- I realize that the avisynth pipeline supports 8b components only. However, saying it is mandatory to lose 2b is wrong. Imagine I had a scene which was dark, with 10b value that ranged only from 0-160. My current flow maps these to the range 0 to 40. When I boost the brightness/contrast/gamma, the result is very noisy as the steps between values is large.

If instead I was able to leave them 0-160, I could do my image work in 8b in avisynth, and at the very end quash the range to whatever looked appropriate, with much less quantization noise.

Why didn't I use 8b 4:4:4 -- it wasn't an option, and ultimately I have more issues with dark scenes than I do with chroma fringing.

Dark Shikari -- (btw, I love reading your blog; thanks for spending the time on your writeups) -- I can recover some of the 2b via dithering, but how much do I lose due to compression? Or does x264 offer a lossless option (sorry to be ignorant -- right now I'm only dealing with mpeg 2 so I can press standard DVDs)

poinsondeathray -- my flow currently is to do all the image processing and NLE in avisynth scripts, then I drop it into vegas movie studio 10 platinum for titling and audio work. According to the product comparison page, they state that pro (which I'm not going to buy) supports floating point components, and the lower products don't, but I imagine it also means 8b components. Is there any other free/cheap 10b capable software you could recommend?

I've attached a frame with histogram to show what it looks like.

[Dark Shikari]

Dark Shikari -- (btw, I love reading your blog; thanks for spending the time on your writeups) -- I can recover some of the 2b via dithering, but how much do I lose due to compression? Or does x264 offer a lossless option (sorry to be ignorant -- right now I'm only dealing with mpeg 2 so I can press standard DVDs)--qp 0

[poisondeathray]

poinsondeathray -- my flow currently is to do all the image processing and NLE in avisynth scripts, then I drop it into vegas movie studio 10 platinum for titling and audio work. According to the product comparison page, they state that pro (which I'm not going to buy) supports floating point components, and the lower products don't, but I imagine it also means 8b components. Is there any other free/cheap 10b capable software you could recommend?


No, and I doubt you will find any inexpensive software with 10+ bit support

For 8-bit workflow in avisynth, you could try smoothlevels instead of levels , which will dither and you will get less banding or quantization

I can't see your attachment (pending approval), but I suspect you won't be able to "fix" your foreground/background problem adquately unless you use masks - It sounds like you will blowout the background trying to fix the foreground exposure by using a "global" filter

[videoFred]

I've attached a frame with histogram to show what it looks like.

Please show us something, this attachment wont work.

Interesting thread! We have done some testing. The same filmreel transfered with my 8 bits machine vision camera vs a high end 16 bits machine vision camera. The difference is way less spectacular then I tought.

The benefit of having more bits is in the post processing: you will have some more 'playroom' before you get artefacts like banding for example.

As for the details in the dark parts: very often there is no detail in the dark parts on the original film. But this depends on the quality of the original of cource. The used filmstock, exposure etc. Very often it is better to leave the dark parts dark, instead of trying to make not existing details visible.

If you want to get max. quality out of my script, please return result4 and do the levels and color corrections in VDub.

Fred.

[Mug Funky]

@ videofred: most cintel machines with 8mm gates aren't exactly "high end" by today's standards... i'm not at all surprised if your gear gives as good a result :)

btw, an Ursa i've used in the past would certainly return 10-bit formatted video, but when tested only yielded 8 active bits. after hard-patch bypassing everything in the TK chain, it still only returned 8 active bits. odd, for a 10 bit machine.

here's a way to test:

- get hold of the qtinput plugin for avisynth - this can natively load v210 without quicktime itself touching the video. it has 2 ways to deal with 10 bits (none ideal mind you) - truncate and rounding.

- load the video twice, one with truncation, one with rounding

- with mt_lutxy, use this expr: "x y - abs 32 *"

this will immediately show any differences between the two. if all you get is a perfectly flat picture, then you've got 8 bit. if you get random noise (or anything at all), you've got 10 bit.

i've confirmed this with true 10-bit stuff.

[videoFred]

@ videofred: most cintel machines with 8mm gates aren't exactly "high end" by today's standards... i'm not at all surprised if your gear gives as good a result :)

Thank you for the compliment, but what I'm trying to say is that often there is nothing to see in the dark parts of old 8mm film. Only darkness. :D

Even a high end $1.000.000 machine will not magically create detail out of nothing. :eek:

But sometimes I get an old 8mm film from someone, made witch good equipment by someone who was knowing what he was doing. Then, with a decent transfer, Avisynth can do his magic:

http://www.facebook.com/album.php?aid=2058406&id=1298449764&l=9830569e79

And even here you will see I have not tried to get anything out of the shadows.
For the simple reason there is nothing in the shadows. It's K40, a very high contrast film.

Fred.

[wonkey_monkey]

Imagine I had a scene which was dark, with 10b value that ranged only from 0-160. My current flow maps these to the range 0 to 40. When I boost the brightness/contrast/gamma, the result is very noisy as the steps between values is large.

If instead I was able to leave them 0-160, I could do my image work in 8b in avisynth, and at the very end quash the range to whatever looked appropriate, with much less quantization noise.

I may be barking up the wrong tree, but if you can open the video with AviSynth twice, once as 10bit shifted down to 8bit (all values divided by 4), and then also as 10bit truncated to 8bit (anything over 255 set to 255), you can merge the two to give you the detail you want without blowing the highlights - would that help, or have I misunderstood what you're trying to do?

David

[frustum]

It has taken me a while to return to this topic, but I spent some time trying to figure out whether the 10b components in my video contain anything meaningful. videoFred says there probably isn't a whole lot of information in the darkest parts of my film, and he is likely right, but I don't want to give up so easily, as I paid a premium to capture 10b.

davidhorman said: if you can open the video with AviSynth twice, once as 10bit shifted down to 8bit (all values divided by 4), and then also as 10bit truncated to 8bit (anything over 255 set to 255)

At this point I'm not worried how to combine them; I'm stuck at the point of figuring out how to get those other bits. I believe the current code truncates (shifts right two bits). David, what you called truncating I would call clamping or saturating.

Mug Funky: I don't think qtinput is of help to me because my files are avi files, not quicktime files.

Dark Shikari: I'm a video noob, so your directions didn't help me. That x264 has a fat pixel pipe is great, but I don't know how this trickles down to me. As far as I understand, x264 is a library, so I need to find software which uses this library so I can use x264 to dither the video for me. I already have ffdshow installed, and that incorporates x264, but I don't see a v210 codec. I installed winff which is a wrapper around ffmpg, which incorporates x264, but specifying "--qp 0" as an extra command line parameter causes errors.

I found the code in virtualdub which reads v210 video and I could easily change it to do what I want, but my attempt to get it to compile under MSVC++ express 2010 edition failed (running on xp under vmware player running on windows 7). When I opened the project file, it said it needed to convert it to the most recent style, but just about every file so converted failed. This approach holds the most promise as I can do arbitrary things with the data before feeding it to avisynth. I'll look more into this approach.

[mp3dom]

Mug Funky: I don't think qtinput is of help to me because my files are avi files, not quicktime files.

QTInput works for AVI too, as long as they contains uncompressed data supported by QTInput. UYVY and V210 works even in AVI containers (probably others too).


As far as I understand, x264 is a library, so I need to find software which uses this library so I can use x264 to dither the video for me.

Go to x264.nl and download (and use) the CLI encoder.

[jmac698]

Yes, there's lots you can do with 10bits. And yes YUV is much better than RGB, try the script below to prove it.
I'd love to have Deep Color support in Avisynth. It's coming very soon to a consumer device near you! Don't be surprised when we look back at this message in 10 years and laugh at the thought that "8 bits are enough". It's supported in HDMI and video cards already, there's some expensive displays for it but it will be common soon. With those extra bits you can: dither it, do an AGC, do HDR tone mapping, get a film look by going back to LOG mapping, and get a lot out of dark scenes. It also converts to 8bit with more accuracy; I can prove that pure yellow needs at least 10 bits YUV to convert properly to RGB. And you know that blue square in the colorbars? That's "-I" which can't be displayed in RGB. We had to reduce saturation to map it.
There's information in your dark scenes; at least the colors, and the luma just needs some denoising to bring it out. At worst it will look like my script below.
I'd love to have a sample to play with; give me a dark scene and I will try to make an AGC for it.
I'd like to propose an intermediate format for Deep Color; just two videos of 8bits each. With scripting alone, we could do all the basic color tweaking in this format. Most plugins would only operate effectively on the high 8 bit video clip however.
Did you know many old analog formats have more than 8 bits? They just need to be brought out. I've always wanted to digitize an old video tape in two passes with different (analog) brightness settings, and combine them with HDR techniques, because I know there's more information in there.

http://www.geocities.com/hanfrunz/iuf_v1.5.zip read uncompressed formats avi, omf(avid), pxr(pixar), mov(24/32bit quicktime), cineon. Can export as well.
There's rawreader, qtinput etc. as well. I'd really like to see this store to a temporary format as two 8bit videos as I described.
Also, to keep the same tint when converting YUV to RGB, you must either increase the luma or reduce the saturation.

x264 iin VFW codec that works with Virtualdub, also lossless x264 compression option
http://sourceforge.net/projects/x264vfw/

x264 lossless doesn't work too well in general though:
http://forum.doom9.org/showthread.php?p=1464097

Ciineon 10bit log:
http://en.wikipedia.org/wiki/Cineon

Deep Color:
http://en.wikipedia.org/wiki/Color_depth#Deepcolor

Conversion accuracy and mapping out of gamut YUV to RGB
http://forum.doom9.org/showthread.php?t=107682&page=3


#Demonstration that YUV stores more information than YUV
#You can recover picture detail from pure YUV by brightening it.
#If you stored your movie only as RGB, you could recover nothing from dark areas.
colorbars(pixel_type="YV12").tweak(bright=-255)
tweak(bright=30)
#Uncomment the line below to see what can be recovered from RGB (hint-nothing!)
#colorbars.rgbadjust(-255,-255,-255).rgbadjust(30,30,30)

[Gavino]

And yes YUV is much better than RGB, try the script below to prove it.
colorbars(pixel_type="YV12").tweak(bright=-255)
tweak(bright=30)
#Uncomment the line below to see what can be recovered from RGB (hint-nothing!)
#colorbars.rgbadjust(-255,-255,-255).rgbadjust(30,30,30)

That is a very misleading demonstration.
Tweak(bright=-255) adjusts only luma, preserving chroma.
Your RGBAdjust destroys everything (all 3 channels r, g and b), so obviously there is nothing to recover.
(Also, you have used a multiplier of -255 instead of a gain, ie the rb,gb and bb parameters.)

In fact, RGB contains more information than YUV, since the range of possible values is greater. This is not simply because of the [0,255]->[16,235] mapping, but more significantly because a large part of YUV space (even using a [0,255] mapping) is not used. See this thread (http://forum.doom9.org/showthread.php?t=154731) for a demonstration of this.

[jmac698]

a large part of YUV space (even using a [0,255] mapping) is not used.
I agree, but that is my point :) I'm adjusting the brightness of these "unused" values to recover the "hidden" picture information.
Let's say you have two encodings of the same video. If you adjust the brightness of the RGB version you don't really get anything more. If you brighten the YUV version then convert to RGB, you do access some hidden picture.
If his film scan truly has full YUV info in it, he can recover picture.

My script above simulates a shadow region of the filmscan, showing that there's still color there if you tweak the brightness in YUV before converting back to RGB.
Make sense now?
If you still don't believe me, I'll write an example using the Rec601 formula.

ps I just wrote a script to simulate tweak in 48bit color. Now Avisynth is the first free program to support Deep Color :) Next I have to write an input filter. Really, post your Y210 videos so I can test it.
I need new features in masktools, this is really a pain.

[Gavino]

My script above simulates a shadow region of the filmscan, showing that there's still color there if you tweak the brightness in YUV before converting back to RGB.
Make sense now?
If you still don't believe me, I'll write an example using the Rec601 formula.
I still don't get what you're saying, so I'd like to see it explained in mathematical terms. The script comparison looks like comparing apples and oranges to me, for the reasons I explained.

[frustum]

mp3dom -- thanks for the tip -- I downloaded the cli program and it works. However, it seems that it supports yv12 output format (4:2:0), but not yuy2 (4:2:2), which means I'd have to throw away half of my chroma. It still might be useful to help investigate whether the lsbs have any data, but dithering only buys back a bit, and a noisy one at that.

I'm on the verge of just writing a program that can extract some frames from my file to establish what is actually there. It is uncompressed v210, and I'm not writing a general avi decoder, so it shouldn't be too much work.

jmac698 -- thanks for the links. I have no doubt that 10b is better than 8b; the first question I need to answer, though, is how many bits of usable data I have, which may be less than the advertised 10b the format can represent.

[jmac698]

Use rawreader, will save you a bunch of time
http://sites.google.com/site/ourenthusiasmsasham/contributions/open-source-software#TOC-Sashimi

v210 format:
http://wiki.multimedia.cx/index.php?title=V210

I think this can all be done in script, bit tedious though. Guess what, we've got a new use for boolean arithmetic in Masktools :)
Update: Can't be done in script, need per pixel bit extraction. Maybe you can modify rawsource, or qt input.

Update2: Yep! There's stuff hidden in the xtra bits. It's been done here:
http://www.dvinfo.net/forum/canon-eos-5d-mk-ii-hd/144428-premiere-pro-cs4-solution-2.html

[Blue_MiSfit]

I'd use x264 in lossless mode to dither the 10 bit input down to 8 bit properly. You can use x264 to do this natively (and I have done it myself).

x264.exe input.avi output.mp4 --qp 0

You will get an MP4 file as an output, containing 8 bit 4:2:0, lossless H.264, which has been properly dithered down. Now you can use this as your soruce for HDRAGC or any other AviSynth tools without worrying about rounding / truncation.

Derek

[frustum]

I installed File::Format::RIFF from CPAN in an attempt to write a perl script to extract frame data. However, it choked on the V210 files as they are huge (> 4GB), but was OK on smaller AVI files (which were not v210).

I then found and installed the freeware program avihex, a hex file viewer with some avi-specific parsing capabilities. I found what looked like image data and grabbed blocks of 64 bytes from three different places in the file. I wrote a small script to break each group of 4B into three 10b components, and verified that my group of 16 bytes (v210 encodes 6 pixels in 16 bytes) was proper by testing that bits [31:30] were always zero with the right alignment and wrong with any different alignment. It seems to be valid v210 pixel data.

However, each of the 10b components was simply an 8b value left shifted two bits. This has all been a goose chase. I plan on following up with the transfer house and ask if they knew that their "10b lossless" premium service is really only 8b.

Thanks to every one who has pitched in with ideas here.

[jmac698]

Hmm, I just finished my Deep Color tools. Do you have anyway of putting some 10bit colors bars into a v210 quicktime format?
http://forum.doom9.org/showthread.php?t=84481&page=5