Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 16th February 2012, 11:55   #1  |  Link
PALpilot
Registered User
 
Join Date: Dec 2007
Posts: 17
RAW like recovering highlight detail beyond clipping

I'm working on Magic Lantern to add in camera functionality to Canon DSLRs. It's is mostly aimed at the video side of the cameras
http://magiclantern.wikia.com/wiki/Unified

Currently we are working on a way to record full RGB information. Normally RGB highlight information is discarded at the point where the first channel clips. This is usually Green. Red and Blue are less sensitive to clipping and can contain detail up to 1EV extra. With the new feature, highlights tend to turn pink because only red and blue are registered.

With RAW photo's it's normal to register full RGB information and RAW converters have smart ways of interpolating missing color data or desaturate the pink to grey. We like to do this color conversion with avisynth and want to ask you guys to come up with a valid script.

here is the discussion thread and sample picture:VRAM25.tif
http://groups.google.com/group/ml-de...a4ce5e18f9140f

possible workflows:
- mask the beyond clipping highlights and desaturate them
- try to recreate missing color info out of the information that exists in the remaining channels.

Last edited by PALpilot; 16th February 2012 at 12:28.
PALpilot is offline   Reply With Quote
Old 16th February 2012, 19:51   #2  |  Link
Yellow_
Registered User
 
Join Date: Sep 2009
Posts: 378
Can you clarify what data it is you have as source? Assume this is 8bit 4:2:0 YCbCr h264AVC off the camera otherwise you'd just shoot RAW images?

If so is converting YCbCr to RGB really the way to go about it? I think it's quite an effort to clip chroma, easy to clip luma. The problems start when it's converted to RGB and how that's done?

Last edited by Yellow_; 16th February 2012 at 20:01.
Yellow_ is offline   Reply With Quote
Old 17th February 2012, 10:12   #3  |  Link
PALpilot
Registered User
 
Join Date: Dec 2007
Posts: 17
Hi Yellow,

I would use:
FFVideoSource("D:\MVI_2912.MOV")
ConvertToRGB(matrix="PC.601", interlaced=false)

You mention it is YCbCr. I always wondered what the initial color space was. Anyway the camera has a RGB histogram and exposure is made by reading RGB values. It signals when one channel clips.

I thought the picture of the duck would work as an example. I will provide a short sample in video later. If you look at the duck picture in a photo edit program with histograms you can see what happens.

If you have pink in areas that should be white. The workflow could be to find the value where the first channel clips and desaturate the higher values of the other channels. I never worked with YCbCr and don't know what the possibilities are. I'm more of a generalist with basic avisynth skills. There are plenty raw conversion programs that deal with this same problem. I was hoping that someone knew how raw converters handle these values.
PALpilot is offline   Reply With Quote
Old 17th February 2012, 10:37   #4  |  Link
Yellow_
Registered User
 
Join Date: Sep 2009
Posts: 378
I'm familiar with the camera and run ML on it etc. Ok, so is it YCC data and how does this highlighter++ thing work, how does it capture any additonal range to then 'recover' or is this just a salvage operation? In the YCC video file or supplementary additional image data?

Are the RGB data histogram and scopes in ML are based on a reverse engineered color space conversion of YCC data sent to the LCD in camera or RGB before the encoding to YCC happens, more upstream? or are you trying to recover YCC data after 'screwing' around with it in the YCC conversion to RGB?

There's numerous ways to convert to RGB via Avisynth, 3D LUT, 16bit. Avisynths built in ConvertToRGB() route may not be the most suitable. Maybe just a different weighting? Assuming RGB is the best route.

I couldn't make head or tail of how it works on the ML wiki thread, see no sign of it on the vimeo channel, don't twitter but I'm interested and happy to spend time searching others won't be without a direct link.

You'll need to get a source file up here or for any interest. Then hopefully some one with far more knowledge than me can help us out :-) Hoping to see a great psuedo HDR video solution from ML. :-)

Last edited by Yellow_; 17th February 2012 at 11:26.
Yellow_ is offline   Reply With Quote
Old 17th February 2012, 13:21   #5  |  Link
PALpilot
Registered User
 
Join Date: Dec 2007
Posts: 17
Sorry about the crippled information. I'm working on an article for the upcoming blogsite. Hope this makes things more clear:

For the color channels RGB (red,green,blue), some channels are more sensitive and will clip earlier than the others. When recording movies or shooting JPEG, the camera clips when the first channel hit the clipping point. It discards the extra data above the clipping point from the other channels. The camera does this to keep accurate colors at a certain white balance.



Usually green is the first channel to clip and blue and red channels can hold data up to an estimate of 1EV extra. White balance can greatly influence clipping point. Manipulating Kelvin values changes the relative positions of the three channels. It can shift one channel away from clipping point if needed. At the cost of overall warmer or cooler colors.

Highlights++ brings the ability to record extra highlight data. It effectively records 8-bit RAW, in terms of highlight headroom. In basic photo RAW mode all raw channel info is preserved and independent on white balance settings. It also records individual channels even if they clip. If green clips first, colors are only recorded on the blue and red channel and the highlights get a pink/purple color. Raw converters discard color information from overexposed pixels (whatever is above white levels) and interpolate color data from nearby pixels (which are not overexposed). Luma data is kept from the "pink" version.

Last edited by PALpilot; 17th February 2012 at 13:30.
PALpilot is offline   Reply With Quote
Old 17th February 2012, 15:10   #6  |  Link
Yellow_
Registered User
 
Join Date: Sep 2009
Posts: 378
When you upload a sample can you do a normal shot and a shot with the feature under same conditions to get a proper comparison?
Yellow_ is offline   Reply With Quote
Old 18th February 2012, 21:33   #7  |  Link
PALpilot
Registered User
 
Join Date: Dec 2007
Posts: 17
hold on we have to sort some technical issues first before coming to proper post processing. I get back to you as soon as possible.
PALpilot is offline   Reply With Quote
Reply

Tags
canon, clipping, dslr, highlight, raw

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:37.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.