Merging double captured (double exposed) files

[videoFred]

Sometimes the dynamic range of real film is impossible to capture to digital in one shot. A possible solution is: double exposure.

Like this: correct exposure:
http://users.telenet.be/ho-slotcars/DOOM/flea_dark.jpg

Exposure + 3 stops:
http://users.telenet.be/ho-slotcars/DOOM/flea_bright.jpg

My question is: how do I merge these frames? Only the blown out pixels from the bright image must be replaced with 'good' pixels from the dark image.

A simple Overlay() is not good enough. Perhaps I need MT_Masktools() for this? But I realy do not know where to begin, so all advice is very welcome.

A clip can be found here:
http://rapidshare.com/files/207442829/double_captured.avi

The dark and bright frames can be separated like this:
source= AviSource("double_captured.avi")
dark= source.SelectEven()
bright= source.SelectOdd()

The filmprojector was standing still while capturing twice. So there is no difference in position between frames 1-2, 3-4 etc.

I realy hope someone finds a good solution for this,
thank you all in advance,

Fred.

[smok3]

so basically you need bright parts from first exp and dark parts from 2nd exposure right?

would it be possible to use first as a mask for 2nd and vice versa?

slightly related;
http://www.luminous-landscape.com/tutorials/hdr.shtml
(you probably can't do that in avisynth thought)

edit: simulation in photoshop, good masking would be necesary, i have blured the mask here
http://somestuff.org/simulation.jpg
(the dark version is at the top, mask is actually grayscaled and blured bright version)

[videoFred]

so basically you need bright parts from first exp and dark parts from 2nd exposure right?

Yes!


would it be possible to use first as a mask for 2nd and vice versa?

Yes, I have tried this with overlay() already but the masking is not good enough. The good pixels from the 2nd are not as bright like they should be. They should be left untouched, if this is possible.

Fred.

[videoFred]

edit: simulation in photoshop, good masking would be necesary, i have blured the mask here
http://somestuff.org/simulation.jpg
(the dark version is at the top, mask is actually grayscaled and blured bright version)

Yes, something like this is what I want.
Thank you for the example! :)

Making a grayscale blurred mask from the bright version and use it in overlay() is very well possible.
But if I do this with the full res 1024x768 files, I still see to many artefacts.

I bet MT_Masktools can do this in a better way, but I do not know how to use it. :confused:

Fred.

[ncahammer]

What's wrong with merge(SelectEven(),SelectOdd(),0.6) ?

[rfmmars]

Ok I did this in Video Pro X, its fast and dirty, I will do again the right way and post steps.

You can either frameserve two "image sequences" from Vdub to Magix and then make the tracks Mix and with "ColorMill 2.1 VD plugin, make the final level adjustments or frameserve two video clips in and repeat the same process. Give me a day and I will post again.


http://img141.imageshack.us/img141/7251/videofredhdr.th.jpg (http://img141.imageshack.us/my.php?image=videofredhdr.jpg)

Richard
photorecall.net

[Sagekilla]

@ncahammer Nothing at all, that's just a very naive way of doing things. It blindly merges 40% of the overexposure into the normal exposure. Good exposure compensation (And this is basically HDR that he's doing) works on a variable basis to create an even tone throughout the whole image. Most people opt for that ugly high contrasty look, but good tone mapping will flatten the contrast in a way that it removes any of those "blown to white" or "crushed to black" scenarios.

What he needs is good per-frame tone mapping, which is something that's nearly impossible for computers to do given two input images. It gets even more difficult when you consider that it needs to look good over time, too! It's a very interesting idea that I'll have to think about and post back about any potential ideas :)

[LaTo]

Try this:
source=AviSource("double_captured.avi")
dark=source.SelectEven()
bright=source.SelectOdd()

var=1.0

mt_merge(bright,dark,mt_average(bright,dark,U=1,V=1).mt_lut("x 255 / "+string(var)+" ^ 255 *",U=1,V=1).removegrain(19,-1,-1),U=3,V=3,luma=true)


Here the result:

var=0.5 (http://www.imagebam.com/image/92484729244838)
var=1.0 (http://www.imagebam.com/image/e9484829244842)
var=1.5 (http://www.imagebam.com/image/71827929244845)

[McCauley]

Hi Fred,

very nice!
I had this idea some time ago too, but havent been able to try it, due to the absence of film material to scan or a scanner.

If you have lots of time, you could try processing the images (export as tiff, jpg, whatever) via batch with Photomatix, a highly sophisticated programm that has been written only for that special purpose. Though i doubt that the gain from using it instead of something like LaTo posted is worth it

Regards
McCauley

[Mug Funky]

use logarithmic mapping...

3 stops = 3*3dB

so convert both pics to log (mt_lut) using the 3dB as a hint, then maskmerge the two together over the area they cross over.

you should then have the equivalent of a cineon/dpx file but in 8 bits. you'll then have to convert this back to linear space, but the advantage here is that you can offset and gain in log before you do so, allowing you access to the film's full latitude.

[videoFred]

I'll have to think about and post back about any potential ideas :)

I thank you for the thinking and please post your ideas, Sagekilla ;)

Fred.

[videoFred]

Try this:
source=AviSource("double_captured.avi")
dark=source.SelectEven()
bright=source.SelectOdd()

var=1.0

mt_merge(bright,dark,mt_average(bright,dark,U=1,V=1).mt_lut("x 255 / "+string(var)+" ^ 255 *",U=1,V=1).removegrain(19,-1,-1),U=3,V=3,luma=true)


Hello LaTo,

I try to understand how Masktools2 works. I see you are merging the clips using mt_average as a mask.
Then you tweak the levels with mt_lut. And finaly you are removing the grain. Is this correct?

PS: why mt_average for the mask?

Fred.

[videoFred]

use logarithmic mapping...


I believe you :)
But how do I do this? :scared:

Fred.

[Sagekilla]

Only issue with logarithmic mapping (base 10 assumed here) is we're limited to integer precision outside of mt_lut. If you do a mt_lut on values from [0, 255], you'll get something like, with brackets show real value:

0 -> 0
1 -> 0
2 -> 0
10 -> 1
40 -> 1 [1.6]
50 -> 2 [1.69]
100 -> 2
255 -> 2 [2.4]

Even if you use natural log, you'd get [0, ..., 255] --> [0, ..., 6]

[rfmmars]

These are the steps that I used on VideoFred's sample avi.
First I separated the underexposed & overexposed frames with Virtualdub. Next I dragged the underexsposed folder onto track 1. Overexposed folder on track 2. Set both tracks to mix. Imported Vdub plugins Colormill 2.11 & RGBeq 2.11 to both tracks. Going back between tracks one and two, I adjusted both filters. For this test, the tracks 1 & 2 have their applied settings to the entire lenght of the film to be rendered.

The screen shot from mine is from the rendered movie, I think Lato's is from the two stills that VideoFred posted.

Not trying to say mine is better than Lato's. One of the adavantages of my way is its so easy to make sence by sence changes. Both VDfilters are being used in the non linera mode.


http://img17.imageshack.us/img17/9223/hdrdemo.th.jpg (http://img17.imageshack.us/my.php?image=hdrdemo.jpg)

Hope to see many other solutions.

EDIT: Forgot to say I post filtered with HDR & LSF in AVS-p

Edit: 2 If you look at by first post, they look different, set it to your own taste. Also I am sorry I didn't have the same frame as Lato's, I will pay more attension next time.

Richard
photorecall.net